As long as your pellets don't ricochet out of your boundary you are doing nothing wrong. Tell him to go and stick his head up his ass and leave you alone.