Okay, I agree, sometimes it is a pain in the neck, frustrating, tiring, thankless, discuraging - add your own frustrations to the list. But even these troubles are fun if you let them be. Most of us who have taken on leadership roles are dynamic, life-long learners. The worst leadership times are the best times for learning. Step out of the day-to-day and look into the growth potential. If you didn't have the trouble, you couldn't have the growth. Take the opportunity; learn from it; and enjoy it.
Another thing to keep in mind is that if there weren't any troubles, why would your team need a leader. Think hard about this. You don't have a role if there aren't any issues. That's a key point of leadership. If you don't thrive on the challenges, leadership might not be the right role for you. And if you don't feel the troubles, perhaps you aren't leading well enough.
So, next time you are frustrated by your leadership role you have two things focus on. First, what are you learning from it. Second, your team still needs you.