Confused about Haymitch and Lenore?

I just read sotr and I'm just confused about Lenore and Haymitch. I'm pretty dumb so pls don't take this as me hating on the book cause I really did enjoy it. I just don't understand how Lenore is the thing that haunts Haymitch the most. Haymitch seems more gutted by her death then he does with the death of his baby brother and mother. I get that he loved her but it was also a teenage romance. The idea that he doesn't move on after 2-3 decades is insane to me since this was his girlfriend from when he was only 16.