Answer: 875

Small Hint:

The rotation is by $15^\circ,$ and since $\angle ABC = 75^\circ,$ the image $AB'$ of $AB$ is perpendicular to $BC$

Big Hint:

The overlap is $[AB'F] - [EB'D],$ where $D$ is the foot of $AB'$ on $BC$ and $E,$ $F$ lie on $B'C';$ use the law of sines and $\triangle EB'D \sim \triangle ABD$

Solution:

Since $AC$ makes a $75^\circ$ angle with the positive $x$-axis, the rotation is by $15^\circ.$ Let $B'$ and $C'$ be the images of $B$ and $C.$ Because $\angle B'AB = 15^\circ$ and $\angle ABC = 75^\circ,$ segment $AB'$ is perpendicular to $BC;$ let $D$ be their intersection, and let $E = BC \cap B'C'$ and $F = AC \cap B'C'.$ The common region is the quadrilateral $ADEF,$ whose area is $[AB'F] - [EB'D].$

In triangle $AB'F,$ $\angle FAB' = 75^\circ - 15^\circ = 60^\circ$ and $\angle AB'F = 75^\circ,$ so $\angle AFB' = 45^\circ,$ and the law of sines gives $B'F = \frac{20\sin 60^\circ}{\sin 45^\circ}$ $= 10\sqrt{6}.$ With $\sin 75^\circ = \frac{\sqrt{6} + \sqrt{2}}{4},$ $$\begin{aligned} [AB'F] &= \tfrac{1}{2} \cdot 20 \cdot 10\sqrt{6}\,\sin 75^\circ \\ &= 50(3 + \sqrt{3}). \end{aligned}$$

In right triangle $ABD,$ $AD = 20\cos 15^\circ$ and $BD = 20\sin 15^\circ,$ so $[ABD] = 200\sin 15^\circ\cos 15^\circ$ $= 100\sin 30^\circ = 50,$ and $B'D = 20(1 - \cos 15^\circ).$ Triangles $EB'D$ and $ABD$ are similar (right angles at $D,$ and $\angle EB'D = \angle ABD = 75^\circ$), so, using $\cos 15^\circ = \frac{\sqrt{6} + \sqrt{2}}{4},$ $$\begin{aligned} [EB'D] &= 50\left(\frac{1 - \cos 15^\circ}{\sin 15^\circ}\right)^2 \\ &= 50 \\ &\quad {}\cdot \left(15 + 8\sqrt{3} - 6\sqrt{6} - 10\sqrt{2}\right). \end{aligned}$$ Therefore $$\begin{aligned} [ADEF] &= 50(3 + \sqrt{3}) \\ &\quad {}- 50 \\ &{}\cdot (15 + 8\sqrt{3} - 6\sqrt{6} - 10\sqrt{2}) \\ &= 500\sqrt{2} - 350\sqrt{3} \\ &\quad {}+ 300\sqrt{6} - 600, \end{aligned}$$ so $(p, q, r, s)$ $= (500, -350, 300, -600)$ and $\frac{p - q + r - s}{2} = \frac{1750}{2} = 875.$