Why not move to Chicago? What are the real downsides?

Currently looking into Chicago and Philadelphia as options to move. I see so many great points and pluses but hearing the real negatives are hard to catch.