I one hundred percent agree with your point--but statistically, this just isn't true. An awful lot of men out there do rape or hit their partners, and many more behave abusively in subtler ways.

That doesn't mean that all men or even most men do. But the problem is not confined to a tiny percentage of men.