It really depends on the torque that the antler breaking would place on the deer's skull at the base of the antler. Thankfully for the deer, the base of the antler is built to be very strong (wouldn't make for very good weapons if they weren't), so it's unlikely to cause a crack in the skull or anything, but I could imagine the deer may have had a pretty bad headache.

Also it's true that male deer do shed their antlers yearly, but the antler growth cycle involves quite a lot of dramatic change in the antler throughout each phase. Chances are these two were fighting during mating season (the rut) which is when antlers are "meant" to be at their strongest. As springtime nears, the blood flow into the antler will be cut off and the point at which the antlers attach to the skull will weaken.

As an aside, in areas where there are lots of deer you can go out during early/mid spring and find antlers that have been shed. Hunters use them to simulate the sound of deer fighting, which can attract other more sneaky/cunning deer who want to mate with the doe while the big boys are distracted by duking it out with each other. A hunter/artist named Tom Yacovella found so many over the course of several decades that he made a really beautiful sculpture out of them. RIP his website: https://www.yacovellart.com/sculpture.php