Solution:

Note that we can express any $4$ digit number as $abcd.$ This can be expressed in long form as $$10^3a + 10^2b + 10c + d.$$ Since in a palindrome, we have that $a = d$ and $b = c.$ We can simplify this to get $$1001a + 110b.$$ Note that $1001$ is divisible by $7.$ This means that $110b$ must also be divisible by $7.$

The only way for this to happen is if $b$ is $0$ or $7$ since $110$ is not divisible by $7.$

There are $9$ options for $a$ and $2$ options for $b,$ for a total of $9 \cdot 2 = 18$ palindromes.

The total number of palindromes is $9 \cdot 10$ since there are $9$ options for the thousands digit and $10$ options for the hundreds digit.

The desired probability is then $$\dfrac{18}{9 \cdot 10} = \dfrac{1}{5}.$$

Thus, E is the correct answer.