Solution:

Divisibility by $16,$ $15,$ and $14$ means $N = 1680m$ where $1680 = \operatorname{lcm}(14, 15, 16) = 2^4 \cdot 3 \cdot 5 \cdot 7.$ Every positive integer less than $14$ divides $1680$ except $9,$ $11,$ and $13,$ and a divisor of $N$ leaves remainder $0,$ not $3.$ So necessarily $\{x, y, z\} = \{9, 11, 13\},$ and we need $1680m \equiv 3$ modulo each of $9,$ $11,$ $13.$

Since $1680 \equiv 6 \pmod 9,$ the first congruence is $6m \equiv 3 \pmod 9,$ i.e. $m \equiv 2 \pmod 3.$ Since $1680 \equiv 8 \pmod{11},$ we need $8m \equiv 3 \pmod{11},$ i.e. $m \equiv 10 \pmod{11}.$ Since $1680 \equiv 3 \pmod{13},$ we need $m \equiv 1 \pmod{13}.$ By the Chinese remainder theorem these combine to $m \equiv 131 \pmod{429},$ so the least $m$ is $131.$

Then $N = 1680 \cdot 131 = 2^4 \cdot 3 \cdot 5 \cdot 7 \cdot 131,$ and since $131$ is prime, the sum of the distinct prime divisors is $2 + 3 + 5 + 7 + 131 = 148.$