Tell him how you feel. Take a time when you can do it privately, and tell him in person. If you can't do that, at least write him a letter, or an e-mail-- give him something he can read.

Don't do it over the phone, or in front of his friends. But you probably already know that ;)