Pickleballs snubbed

At open play, when using a ball I supplied I can't tell if people are repeatedly inspecting the ball for damage or looking for a name brand. Recently someone switched out the ball (Diadem) for one of their own and apologized for doing it which made it weird. I have a mix of balls in my bag. Franklin X40s (both yellow and pink), Diadem Premier Power and some others with the name worn off (I use those mainly for drilling), I can't remember what they are. I never bring out the pink balls, those get stared at the hardest. I always prefer to play with someone else's ball just so they can't blame the ball I brought for their poor play. Is it just me?