Question about the Nvidia driver

Recently, I downloaded the driver for my graphics card to update it. However, I noticed that the Nvidia application was consuming some RAM on my computer, so I decided to uninstall the drivers. The games were running fine, but as I mentioned, it was using RAM that wasn’t needed before. After uninstalling, the games still run normally, and the RAM usage has returned to what it was before. Is there any problem for the graphics card if I play heavier games like Cyberpunk 2077 and The Last of Us without using the drivers?