How to play popular piano accompainments like these?

I've been a pianist for many years, primarily dedicated to classical music. I can read sheet music, understand harmony, and have solid musical perception. I also have no difficulty interpreting pieces of considerable complexity. I even studied music at the university level for a few semesters before switching majors. However, I still dedicate myself to music daily as a hobby.

That said, I’ve been struggling with a problem. For many years, I’ve been trying to deepen my understanding of popular music. One skill I have a hard time acquiring is accompanying singers or melodic instruments using only the piano. Even though I can play virtually any chord, I just can’t seem to grasp how to create those beautiful piano accompaniments we often see in piano and voice performances.

Whenever I need to accompany someone in a pop music performance, I find myself stuck in a very basic and uninspired way of playing. I end up just repeating a chord in a certain voicing throughout the entire measure, playing it once per beat, without much variation or musical interest.

It’s hard to explain. What I want to learn is how to create those accompaniments where the piano almost acts as a counterpoint to the voice, filling in spaces with melodic lines that make sense—sometimes using other notes from the scale. At times, it just marks the rhythm, like I do, but with some variations that make the accompaniment more interesting. I’ll leave some links below to illustrate what I mean.

https://www.youtube.com/watch?v=8mGQ3LL2EHo
https://www.youtube.com/watch?v=b6sKnKP9PX4
https://www.youtube.com/watch?v=6x_bSXZQ1Fs

I understand that patience is necessary to learn this. What frustrates me is that I hear these accompaniments and know that, technically, they are easy for me to play. However, I don’t understand the theory behind how they are built. How does one learn this? Is there a specific discipline in the field of music that studies this? Or is it something you only learn by copying what other pianists do?

Sometimes, it feels like the accompaniment is almost like a simple piano piece written from scratch. Is there a thought process that could guide me toward greater autonomy?

What I’ve been trying to do to overcome this is to learn transcriptions and hope that, over time, I develop a repertoire of resources and gradually move beyond just copying, eventually creating my own arrangements. However, when I look for teachers or lessons, it becomes even more frustrating. They often go over things I already know, while what I really want is a more applied and targeted approach to what I already master.

Additionally, many materials I find are just people playing instrumental versions of songs on the piano, which is different from actually accompanying someone.

Does anyone here know how to play like in the examples I shared above? If so, how did you learn it? Can anyone help me?

I've been struggling with this for a long time. Mastering this is my New Year's resolution for 2025.