Solution:

A committee with $k$ men has $k + 1$ women, so $$N = \sum_{k=0}^{11} \binom{11}{k}\binom{12}{k+1} = \sum_{k=0}^{11} \binom{11}{k}\binom{12}{11-k} = \binom{23}{11}$$ by Vandermonde's identity (both sides count ways to choose $11$ people from all $23$).

Now factor $\binom{23}{11} = \frac{23!}{11!\,12!}.$ The primes $13,$ $17,$ $19,$ $23$ each appear in the numerator but not the denominator. By Legendre's formula the exponent of $2$ is $19 - 8 - 10 = 1,$ of $3$ is $9 - 4 - 5 = 0,$ of $5$ is $4 - 2 - 2 = 0,$ of $7$ is $3 - 1 - 1 = 1,$ and of $11$ is $2 - 1 - 1 = 0.$ Hence $N = 2 \cdot 7 \cdot 13 \cdot 17 \cdot 19 \cdot 23.$

The sum of the primes dividing $N$ is $2 + 7 + 13 + 17 + 19 + 23 = 81.$