I’ve been stumped by this problem:

Find three non-constant, pairwise unequal functions f,g,h:R→R such that

f∘g=h

g∘h=f

h∘f=g

or prove that no three such functions exist.I highly suspect, by now, that no non-trivial triplet of functions satisfying the stated property exists… but I don’t know how to prove it.

How do I prove this, or how do I find these functions if they do exist?

All help is appreciated!

The functions should also be continuous.

**Answer**

Here is a proof that no such continuous functions exist. I’ll use juxtaposition to denote function composition. As noted in the comments,

h2=fgh=f2=ghf=g2.

Let e=f4=g4=h4. Note that fgf=hf=g and gfg=gh=f, so

ef=g4f=gf2gf=gfg=f,

fe=fg4=fgf2g=gfg=f.

Similarly eg=g=ge and eh=h=he. In particular e2=ef4=f4=e, so e is idempotent. Consider its image X=ime⊆R, so e|X is the identity. We have imf⊆X since ef=f. Moreover f|4X=e|X=idX, so f|X is a permutation of X, and similarly for g and h. (aside: it now follows that f|X, g|X and h|X generate a quotient of the quaternion group).

Suppose |X|>1. Since e is continuous, X is a (possibly infinite) interval. Any continuous permutation of such a set is strictly monotone (either increasing or decreasing). Moreover the only strictly increasing involution of X is the identity. Indeed suppose σ is such a function. If σ(x)>x then

x=σ2(x)>σ(x)>x,

a contradiction. Similarly we cannot have σ(x)<x, so σ is the identity.

In particular f|X, g|X and h|X are strictly monotone. Since f|X=g|Xh|X, they can't all be decreasing. wlog suppose f|X is increasing. Then f|2X is increasing. Since f|4X=idX, applying the above result twice gives f|2X=idX and then f|X=idX.

If |X|=1, then we also have f|X=idX. In either case, for any x∈R we have g(x)∈X, so

h(x)=f(g(x))=g(x),

whence h=g.

**Attribution***Source : Link , Question Author : Franklin Pezzuti Dyer , Answer Author : stewbasic*