I'm now trying to get rid of that BUT part, and it feels a lot better. Harder - yes. Because you kinda have the feeling that you didn't say everything you wanted. Didn't express your feelings, emotions, but If you think about it, its not about expressing your opinions. Its about showing that you are better than he is, because you don't have those problems. I haven't heard anyone say something bad about other person if he would think he has the same problem in his life.
Its a hard thing to do, to resist the temptation to say some things about other people. But it must be the right thing to do.