2008 AIME I Problem 2

If the altitude $h$ were at most $10,$ the overlap would be the whole triangle, forcing $h = 16$ — impossible, so the apex pokes out above the square

The overlap is triangle $GEM$ minus a similar triangle above the square with height $h - 10$ and base $\frac{10(h - 10)}{h}$

Here $\overline{EM}$ is a side of the square. Let $h$ be the altitude of triangle $GEM.$ If $h \le 10,$ the triangle would lie entirely inside the square, and its area $\frac{1}{2} \cdot 10 \cdot h = 80$ would force $h = 16,$ a contradiction. So $h \gt 10$ and the apex $G$ lies outside the square; the opposite side $\overline{AI}$ cuts off a smaller triangle similar to $GEM$ with height $h - 10$ and base $\frac{10(h - 10)}{h}.$

The common region is triangle $GEM$ minus that small triangle: $$\begin{aligned} 80 &= 5h \\ &\quad {}- \frac{1}{2} \cdot \frac{10(h - 10)}{h} \\ &\qquad {}\cdot (h - 10) \\ &= 5h - \frac{5(h - 10)^2}{h}. \end{aligned}$$ Multiplying by $h$ gives $80h = 5h^2 - 5(h - 10)^2$ $= 5(20h - 100)$ $= 100h - 500,$ so $20h = 500$ and $h = 25.$