Solution:

Since $6! = 720 = 2^4 \cdot 3^2 \cdot 5,$ the visible product uses only the primes $2, 3,$ and $5.$

Hiding $4$ leaves the fewest $2$'s, namely $2^2.$ Hiding $3$ or $6$ leaves the fewest $3$'s, namely one. Hiding $5$ leaves no factor of $5.$

Therefore $P$ is always divisible by $2^2 \cdot 3 = 12,$ but not necessarily by any larger number.

Thus, the correct answer is B.