Solution:

The six face centers form a regular octahedron, viewed as two congruent square pyramids sharing a base. Adjacent face centers are $\frac{\sqrt2}{2}$ apart, so the square base has area $\left(\frac{\sqrt2}{2}\right)^2 = \frac12.$

Each pyramid has height $\frac12,$ so its volume is $\frac13 \cdot \frac12 \cdot \frac12 = \frac{1}{12}.$ The octahedron has volume $2 \cdot \frac{1}{12} = \frac16.$

Thus, the correct answer is B.