Solution:

Let the block measure $p \times q \times r.$ A cube is hidden exactly when it touches none of the three visible faces, so the hidden cubes form a $(p-1) \times (q-1) \times (r-1)$ block, giving $(p-1)(q-1)(r-1) = 231 = 3 \cdot 7 \cdot 11.$

The ways to write $231$ as a product of three positive integers are $3 \cdot 7 \cdot 11,$ $1 \cdot 3 \cdot 77,$ $1 \cdot 7 \cdot 33,$ $1 \cdot 11 \cdot 21,$ and $1 \cdot 1 \cdot 231,$ giving blocks $4 \times 8 \times 12,$ $2 \times 4 \times 78,$ $2 \times 8 \times 34,$ $2 \times 12 \times 22,$ and $2 \times 2 \times 232,$ with volumes $384,$ $624,$ $544,$ $528,$ and $928.$

The smallest is $N = 384.$