Those aren’t the same situations. You’re allowed to discriminate against Nazis or people who own a ficus, but not gays. It’s not an arbitrary line, it’s a legally well defined distinction.

In both cases you don’t want to offer those people a service because of hatred. You’re allowed to hate people and discriminate against them for a variety of reasons. As a society we’ve legally decided that it’s not acceptable to hate (insofar as it leads to discrimination) for many reasons innate to a person (race, religion, sexual orientation, etc…). That’s the line.