Landlord verbally abusive, trespassing when we are not home?

When you take the house on rent, you are the de facto owners as long as the agreement lasts. You can book him for trespassing and misbehavior. Do that, and he would himself like to see you leave. Report him, and you will not lose your deposit. Just be assertive with him in the matter. He will buckle down.