John Stuart Mill was a utilitarian, but he also believed in the harm principle. Is he being consistent?

In other words, is Mill really a thoroughgoing utilitarian, or does he sneak in another ethical framework for his harm principle?

I’ve never read Mill, so maybe he justifies the harm principle on utilitarian grounds, but the thought just occurred to me, and I’m curious to know.