Franky master Tom's story feels forced.

I'm currently reading it and I can't help but feel it's forced. I mean, the villagers who know Tom for 20 years believed he'd attack the port? He lived there 30 years so why on such important day would he carry an attack? Even though, they suddenly change their view completely. I think it'd have been better if someone murdered him but the current way just feels unnatural in my opinion