Any solution for the random and seemingly permanent frame drops until restart on PC? Specs in post body.

i7 12700k

3070ti

64GB RAM

I'm getting 75-80 fps on High / Medium mixed settings with DLSS Quality at 1080p with no frame gen.

When I use Framegen I get a smooth, albeit slightly input delayed (worth it), 120fps locked. BUT when using Framegen I get a permanent and random drop to about 15-20 fps soon after playing for about 10-15 mins or if I enter a in-engine cutscene which drops the game to 30fps and is a stuttering 30fps mind you. I also get locked to the 15-20fps with framegen sometimes after just having a convo with any NPC.

After a restart of system and game I was playing fine at 65-75 fps without Framegen but then about an hour in I got the 15-20fps drop in the wilderness in an area I was JUST playing at 75fps in after a death and reload?

Drivers are all updated, I have no issues in any other games.

Any ideas?