How to talk about fears of never closing the distance?

I’ve been in a long-distance relationship for about a year and a half now, with my boyfriend who lives a four/five hour drive away. We met while living in different cities and eventually decided to start a relationship. When we first began, I didn’t think too much about the logistics; I just knew I really liked him and hoped we could figure things out eventually. He has a remote job and chose to live in his current city because he enjoys it and wanted to try it out. I live close to family and am in a in person master’s program and have a hybrid work schedule. I’ve told him that I don’t see myself living in his city and want to stay where I am, both for my work and to be near my family, and the city he lives in is pretty expensive. He has said he wouldn’t mind living in my city, but he really enjoys where he currently lives and isn’t sure when he would move. This uncertainty is stressing me out because it feels like there’s no clear timeline, and it’s hard to gauge if we’re working toward any kind of solution. I’m also frustrated because I’m the one who always brings up the distance and the idea of potentially closing it. When I’ve expressed this frustration, he’s said he doesn’t bring it up because there’s not much new to add—he likes where he is but would be willing to move, he just doesn’t know when. He does make effort to visit when he can since his job is remote, but I have a desire to live in the same city. I’m considering bringing up that if we don’t make any progress on our distance situation, I might want to consider parting ways. Is this a fair thing to say, or does it come across as too harsh? Is one and a half years too early to be thinking like this? If it’s not too harsh, what would be a good way to communicate this?