AITA for staying at a birthday party when they explicitly asked me to stay?

My friend (35f) has been ill lately and severely under the weather, to lighten her mood I(39m) decided to bring her a bucket of popcorn from the store. I did ask If that was okay, but she hadn't read my message yet, when I arrived. I just wanted to leave it with her at the door but her mother opened and asked me to come inside (she does this frequently but normally I'm in a hurry and decline). I came inside and had a long Chat with them (approximately 2.5 hours) and wanted to leave because I knew they would have dinner guests. The guests arrived and starten to wish my friend's mother Happy Birthday! I was mortified and apologized profusely, because I had no idea and they obviously hadn't told me. As I wanted to leave her father asked me to stay and said there was enough food for everyone. I had a nice evening. A few days later after not having heard from my friend for a while I inquired about her health and she told me, she didn't contact me on purpose, because she was not fine with how I acted. I was surprised and asked what was wrong but she wanted to tell me personally and not on the phone. This took a few days, in which I tried to imagine what I could have done wrong, but she finally told me: She and all her family we're really weirded out that I stayed when the guests arrived, I had already overstayed my welcome before that and should have left then at the latest. They did not tell me any of this at the time and as mentioned before explicitly asked me to stay. This is we're I wonder If IATA, because they all thought I should have politely declined and left. I damit I thought it weird them not mentioning the birthday and would obviously not even have come in if I knew, but to decline an invitation seemed rude to me. So AITA?