Not really much different than the people who think they're not racist because they have a black friend.
It's like they're saying they're a nice guy because (a) there's women they love or (b) women who love them. Nope. Sorry. Women have been loving and loved by guys who act like jerks to women for hundreds of years. And yes, I know -- not all men. But too many.