Ethernet causing Mac mini m1 to freeze

So I just picked up a base model mac mini m1. Its connected to my local server through ethernet. Its connected to a gigabit network switch which is also connected to my server and to my Isp through a router.

Ive noticed that if I start writing or reading heavy files off my server it causes the mac mini to freeze up finder and then freezes up the whole desktop.

Troubleshooting that i've tried: -Safe mode -Guest account -reinstalled mac os (monterey) -format and reinstall of mac os

I came from an intel base MBP 13" which was also connected to my server and internet in the same fashion but I have never experienced the same issue.

any fixes that you guys could recommend?