When someone says to you, “You shouldn’t impose your morality on other people,” proceed as follows:

YOU:  “So you think it is wrong to impose one’s moral point of view on other people?”
THEM:  “Yes.”
YOU:  “Then why are you imposing your moral point of view on me?
THEM:  “What?”
YOU:  “To say it is wrong to impose one’s moral point of view on other people is itself a moral point of view, and you are imposing that moral point of view one me by morally condemning me for morally condemning the actions of other people.  You are guilty of doing the very thing you say should not be done.”

The fact of the matter is that we all have a moral point of view, and all of us apply that moral standard to others and judge them accordingly.  The question is not whether we have moral standards, or whether we will apply them to other people, but rather whether or not our moral standards are true.