Example of a Borel set that is neither FσF_\sigma nor GδG_\delta

There are many examples. Here’s one:

Observe first that the rational numbers $\mathbb{Q}$ are an $F_{\sigma}$. This is because they are a countable union of points. The irrational numbers $\mathbb{R} \smallsetminus \mathbb{Q} = \bigcap_{q \in \mathbb{Q}} \mathbb{R} \smallsetminus \{q\}$ are thus a $G_{\delta}$. Since both $\mathbb{Q}$ and $\mathbb{R} \smallsetminus \mathbb{Q}$ are dense and disjoint it follows from the Baire category theorem that $\mathbb{Q}$ cannot be a $G_{\delta}$. [Edit: See also this thread here containing several proofs that $\mathbb{Q}$ can’t be a $G_{\delta}$ in $\mathbb{R}$. These proofs explicitly avoid Baire].

The same reasoning shows that $F = \mathbb{Q}_{\geq 0}$ is an $F_{\sigma}$ in $[0,\infty)$, but isn’t a $G_{\delta}$, and that $G= \mathbb{R}_{\leq 0} \smallsetminus \mathbb{Q}_{\leq 0}$ is a $G_{\delta}$ in $(-\infty,0]$, but isn’t an $F_{\sigma}$. Their union $F \cup G$ is then an example of a Borel subset of $\mathbb{R}$ which is neither an $F_{\sigma}$ nor a $G_{\delta}$ because if it were an $F_{\sigma}$ then the same would hold for $G = (F \cup G) \cap (-\infty,0)$, for example. I leave it as an exercise to show that $F \cup G$ is both an $F_{\sigma\delta}$ and a $G_{\delta\sigma}$.

That’s probably the easiest example. A few more (both more interesting but also more involved ones) can be found in this MO thread.

For a much more in-depth discussion of such ideas, I recommend looking into one of the following books:

• A.S. Kechris, Classical Descriptive Set Theory, Springer GTM 156.

• S.M. Srivastava, A course on Borel sets, Springer GTM 180.

• J.C. Oxtoby, Measure and Category, Springer GTM 2.

Specifically, look up the sections on the Borel hierarchy.