Answer: 83

Solution:

Place $A = (0, \sqrt{3}),$ $B = (-1, 0),$ $C = (1, 0),$ so $D = (0, 0),$ $E = \left(\frac{1}{2}, \frac{\sqrt{3}}{2}\right),$ $F = \left(-\frac{1}{2}, \frac{\sqrt{3}}{2}\right).$ Write $P=D+x(E-D),$ $Q=E+y(F-E),$ and $R=F+z(D-F),$ where $0\le x,y,z\le1.$ Computing the three two-dimensional cross products, the collinearities $C,P,Q;$ $A,Q,R;$ and $B,R,P$ give, respectively, $$\begin{gathered} x(1+y)=1,\\ y(1+z)=1,\\ z(1+x)=1. \end{gathered}$$

Let $f(u)=\frac{1}{1+u}.$ The equations say $x=f(y),$ $y=f(z),$ and $z=f(x),$ so $$x=f(f(f(x)))=\frac{x+2}{2x+3}.$$ Hence $x^2+x-1=0,$ and positivity gives $x=\frac{\sqrt5-1}{2}.$ The equations then force $y=z=x;$ call this common value $t.$ Thus the desired configuration really is the symmetric one, rather than merely being assumed to be.

With $P = \left(\frac{t}{2}, \frac{t\sqrt{3}}{2}\right)$ and $Q = \left(\frac{1}{2} - t, \frac{\sqrt{3}}{2}\right),$ both triangles are equilateral with center $G = \left(0, \frac{\sqrt{3}}{3}\right).$ Therefore the area ratio is $\frac{GA^2}{GP^2}.$ Using $t^2 = 1 - t,$ $$\begin{aligned} GP^2 &= \frac{t^2}{4} + 3\left(\frac{t}{2} - \frac{1}{3}\right)^2 \\ &= t^2 - t + \frac{1}{3} \\ &= \frac{7}{3} - \sqrt{5}, \\ GA^2 &= \frac{4}{3}. \end{aligned}$$

Hence $$\begin{aligned} \frac{[ABC]}{[PQR]} &= \frac{4/3}{7/3 - \sqrt{5}} \\ &= \frac{4}{7 - 3\sqrt{5}} \\ &= 7 + 3\sqrt{5}, \end{aligned}$$ so $a = 7,$ $b = 3,$ $c = 5,$ and $a^2 + b^2 + c^2 = 49 + 9 + 25 = 83.$