Answer: 676

Solution:

If $f$ has degree $m$ and leading coefficient $a,$ the leading coefficients of the two sides of $f(x)f(2x^2) = f(2x^3 + x)$ are $a^2 2^m$ and $a 2^m,$ so $a = 1.$ The equation also shows that whenever $\lambda$ is a root, $2\lambda^3 + \lambda$ is a root as well.

If some root had $|\lambda| \gt 1,$ then $|2\lambda^3 + \lambda| \ge 2|\lambda|^3 - |\lambda| \gt |\lambda|,$ and iterating would produce infinitely many distinct roots — impossible. Since $f$ is monic with $f(0) = 1,$ the product of the roots has modulus $1,$ so no root can have modulus less than $1$ either: every root satisfies $|\lambda| = 1.$ Then $2\lambda^3 + \lambda$ must also have modulus $1,$ so $|2\lambda^2 + 1| = 1.$ Writing $\lambda^2 = \cos\theta + i\sin\theta,$ we get $(2\cos\theta + 1)^2 + 4\sin^2\theta = 1,$ which simplifies to $\cos\theta = -1,$ so $\lambda^2 = -1.$

Thus every root is $\pm i,$ and real coefficients pair them up: $f(x) = (x^2 + 1)^n.$ The condition $f(2) + f(3) = 5^n + 10^n = 125$ gives $n = 2,$ so $f(5) = 26^2 = 676.$