Video solution:

Click to load, then click again to play

Written solution:

Without loss of generality, allow $A$ to be a vertex of the triangle. Suppose we also have points $B,C$ of the triangle with $A,B,C$ being in clockwise order. Let $x$ be the number of vertices of the octagon between $A$ and $B,$ $y$ be the number of vertices between $B$ and $C,$ and $z$ be the number of vertices between $C$ and $A.$ We know $x+y+z = 5$ as it encompasses every vertex of the octagon except $A,B,C.$

If two sides form the sides of an octagon, the distance between them would be $0.$

Therefore, if we use complementary counting to find how many have $x,y,z > 0,$ we can deduce out how many triangles are formed with no sides of the triangle being a side of the octagon. This would make $x,y,z$ whole numbers whose sum is $5.$ Using the stars and bars method, we can see that there are $\binom {5-1}{3-1} = 6$ ways to place $B,C$ such that $x,y,z > 0.$ Now to find the total number of cases, since there are $7$ points that aren't $A,$ there are $\binom{7}{2} = 21$ ways to place $B,C$ in clockwise order.

This means there is a $\dfrac{6}{21} = \dfrac{2}{7}$ probability of the triangle not having sides on the octagon. Therefore, there is a $1- \dfrac{2}{7} = \dfrac{5}{7}$ probability of the triangle having at least one side on the octagon.

Thus, D is the correct answer.