I've discovered that wearing ear plugs gives me much peace. Are there any downsides in using them everyday/all day?

The world becomes silent and somehow this sooths me. It is also easier to focus on breathing, as this is the only thing I can hear/feel.

But I'm wondering if my eardrums would deteriorate over time when not being used as much?