Minor point… at any time a property owner or an agent thereof can ask you to leave and your trespassing at that point.

As far as civil liberties, the reasons can’t be that you’re a protected class. But that’s about it. Generally, it’s really hard to prove that the rules are being unfairly applied to begin with, and then there are a lot of ways around that. it’s really hard to prove racism for some of those ways (“suspicious”=hoodie, for example.)