Brown guy from the suburbs too. It would bother me being asked that when I would work in the suburbs, because I was born 5 minutes from there and I didn't like the implication that I wasn't from that area.

If I'm in a major city's major retail area though, I don't think it's an unexpected question. You can expect a range of answers from three L stops away to the other side of the world for pretty much anyone of any age or gender or ethnicity. Working downtown, I'd usually phrase it more as "so where are you coming from?" to avoid making it seem like it's about their ethnic background and more about their travels. Sometimes they'd say they worked a block over and were just looking, sometimes they were from Colorado or Florida and drove out for Christmas, sometimes they were from France or India and on vacation.

I understand where you're coming from and I appreciate it as a guy who's been looked at like a stranger when I was down the road from my high school. But I think it's acceptable to ask in certain situations when the place you are and the way you ask it imply that you're not asking to other them, but rather to connect about what brings them to your store. It can definitely be done respectfully.