Riemann zeta function at odd positive integers

Starting with the famous Basel problem, Euler evaluated the Riemann zeta function for all even positive integers and the result is a compact expression involving Bernoulli numbers. However, the evaluation of the zeta function at odd positive integers (in terms of getting a closed form sum) is still open. There has been some progress in the form of Apery’s theorem and other results such as “infinitely many of $\zeta(2n+1)$ are irrational” or “at least one of $\zeta(5),\zeta(7),\zeta(9)$ or $\zeta(11)$ is irrational”.

Question(s): Is there a high level understanding for this disparity between even and odd integers? Is it a case of there being a simple expression for $\zeta(3)$ that is out there waiting for an ingenious attack like Euler did with $\zeta(2)$? Or is the belief that such a closed form summation is unlikely? Where do the many many proofs powerful enough to evaluate $\zeta(2n)$ stumble when it comes to evaluating $\zeta(2n+1)$?

Motivation: The Basel problem and Euler’s solution are my all-time favorites for the sheer surprise factor and ingenuity of proof (what do $\pi$ and $\frac{sin(x)}{x}$ have to do with $\zeta(2)$??). However, I currently lack the more advanced analytical tools to appreciate the deeper results of this area. I have wondered for a while about the questions above and Internet search hasn’t helped much. I would greatly appreciate any answers/references. Thanks.

The zeta function is defined as a sum over the positive integers, but as far as actually evaluating it, it turns out to be more natural to think of it as a sum over all nonzero integers; thus we should really be thinking about $\sum_{n \neq 0} \frac{1}{n^k}$. For $k$ even this is just $2 \zeta(k)$ and there are various ways to evaluate this more symmetric sum, e.g. by writing down a meromorphic function with the right poles, or a Fourier series with the right coefficients, etc. But for $k$ odd this is equal to zero, since terms cancel with their negatives! Written in this way, the zeta function at even integers reveals its alter ego as an Eisenstein series in one dimension.
This cancellation phenomenon occurs in Euler’s classic “proof,” since the infinite product for $\sin z$ that he uses has zeroes at all integer multiples of $\pi$, not just the positive ones. It also occurs in the general proof that proceeds by considering the generating function $\frac{z}{e^z - 1}$ for the Bernoulli numbers. As you might know, the closed form of $\zeta(2k)$ involves Bernoulli numbers, and again $\frac{z}{e^z - 1}$ has poles at $2 \pi i n$ for all nonzero integers $n$, not just the positive ones. I describe how this works in slightly more detail here.
Another way to think about the difference between the even and odd cases is that one can think of the even cases as $L^2$ norms of appropriate Fourier series; this is precisely how a standard proof of the evaluation of $\zeta(2)$ works. But for the odd cases we don’t get an $L^2$ norm; instead we get a mysterious inner product.