...it is my opinion, and as the Pastor my opinion is that my youth leader wont do it under my leadership,
No, it's a direct restriction on your youth pastor.
you're the pastor, and you have that right, but you can't call it just an opinion when you mandate it for someone under your authority. That is what I am calling contradictory. I may disagree with Poimen on this, but at least he is consistant in his beliefs.
( this phrase "half naked" isnt a derogatory comment, just a phrase I use to describe someone without all their clothes on

)
Yes, it is, it implies that the person is not appropriately covered. It's used not just as someone who has removed an acceptable amount of clothing, but for one who is inappropriately covered.
Sometimes (not often) I may be sitting around the house without my shirt on (just got up, not having put one on, getting ready for bed, etc. If anyone comes over I put it back on, it's not neccesary to have it off, and not really appropriate in that situation. Working outside and it gets hot, most of the time I still won't take my shirt off (I'm fat and it don't look good) but I don't see a problem with it either. If I were working outside without a shirt on, really it doesn't matter who comes by, I'm working, it's hot, and it's appropriate attire for that situation, now if I stop working to go sit around and talk, or something, then yeah, I'd probably put something on.
Swimming is the same situation, It's an appropriate situation for suitable clothing, that doesn't mean find the skimpiest string bikini, or tightest shorts.
I have done alot of swimming, and it has been mainly in public or semi public places, I have seen inappropriately dressed men and women, just as I have seen inappropriately dressed people at the mall, wal-mart, and just about anywhere.
Two things that are more important than the amount of skin shown.
1. Intentions - I have seen one girl coming into Walmart she had just come from church (I am familiar with the church) strict holiness dress, long denim skirt, hair down to her waist, no makeup, no jewelry, showing as much skin as she could get away with in that outfit by having the skirt unbuttoned up to her hip. She was much more covered than most people in the store, but IMHO her intentions made her dress more inappropriate than someone the next aisle over in shorts and a tank top.
2. Situation - Dress should be appropriate to the situation. Walking through the mall in a swimsuit, is inappropriate, swiming in the same suit would not be. In this way I can see you feeling comfortable being shirtless at work, but not say, in the pulpit
Who I am around shouldn't dictate different actions. Being shirtless is not a sexual thing for you when you are at work, and doesn't have to be in the pool either.
That being said, leaders do have to always be aware of the way those under them are perceiving them, and extra measures have to be taken IF there is a problem. This is regardless of amount of clothing. If there seems to be a crush, enfatuation, lusting towards you, then deal with it accordingly, but that's a whole different story than making blanket rulings that will limit even innocent activities.