What follows is a quote from Luther's very last sermon, delivered on Feb 14th, 1546.
"True preachers must carefully and faithfully teach only God’s Word and must seek its honor and praise alone. In like manner, the hearers must say: We do not believe in our pastor; but he tells us of another Master, One named Christ. To Him he directs us; what His lips say we shall heed. And we shall heed our pastor insofar as he directs us to this true Master and Teacher, the Son of God."
As Pastors, we are called to teach only the Word of God. We are to point to Christ and instruct people to follow Him. In all things, whatever we teach, it must point solely to the Word of God, and no where else.
If someone ignores you, disagrees with you, besmirches what you teach, you must ask yourself a simple, yet difficult question. Is this person rejecting me and what I say, or are they rejecting what Christ and His Word says?
If you are downtrodden and burdened by rejection, and then you see that you have been speaking simply what Christ has commanded you to say - then know that you are not rejected, but Christ is. Remain faithful.
If you see that they reject and despise not what Christ says in His Word, but what you say, what you conclude, what you command, then repent and apologize. If you see that you have be teaching the way you wish things would be instead of what Christ does and brings and what He makes to be - repent.
As a Pastor you have no authority whatsoever other than the authority which Christ gives you - and that is to teach His Word of life and forgiveness - and you have no authority to add to it nor call to subtract from it.
Criticism is a chance, a call for you to reflect upon this. Are you speaking God's Word or instead your own thoughts on how best to please God? One is good, the other is not.