Solution:

The second equation gives $|x|+|y|-4=\pm1$, so $|x|+|y|=3$ or $|x|+|y|=5$. Also the first equation gives $y=3-\frac{x^2}{3}$.

If $|x|+|y|=3$ and $y\ge0$, then $|x|+y=3$. For $x\ge0$, this gives $x+3-\frac{x^2}{3}=3$, so $x=0,3$. For $x\le0$, it gives $-x+3-\frac{x^2}{3}=3$, so $x=0,-3$. These give $3$ distinct points.

If $|x|+|y|=3$ and $y\lt0$, then $|x|-y=3$. For $x\ge0$, $x-3+\frac{x^2}{3}=3$, so $x=3$, which has $y=0$, not $y\lt0$. The case $x\le0$ similarly only gives $x=-3$, also already counted.

If $|x|+|y|=5$ and $y\ge0$, then $|x|+3-\frac{x^2}{3}=5$, which has no real solution in the required sign ranges. If $y\lt0$, then $|x|-3+\frac{x^2}{3}=5$. This gives $x=4$ for $x\ge0$ and $x=-4$ for $x\le0$, producing $2$ more points.

The total number of ordered pairs is $3+2=5$.

Thus, D is the correct answer.