Noob still unsure about purchase of Sit-in vs -on.

70% sure I want a sit in vs a sit on kayak. I've watched several vids on YouTube regarding one vs the other. Are there any situations I'm not considering where I'd be sorry I got a Sit-in? Other than being a little easier to get into, is there anything a Sit-on does better? I'm not a fisherman so it's just recreational. I have a calm lake right next to my home where it will get the most use, but I also want the option of taking it with me camping and doing some easy rivers.