I've said it before and I'll say it again.
Canon Visser Three is a campy, Scooby-Doo-esque villain who spouts off cliche phrases and is evil just for the sake of being evil. Not a very appealing villain, really.
Now, if you look at the offshoot books (HBC, TAC and even Visser to some extent) you get a much better look at Esplin 9466 as a three-dimensional character - someone who was so driven to reach the top that he made significant sacrifices, especially moral ones.
Visser Three does a lot of cruel things in the series - I mean, he eats Elfangor and makes a joke about it immediately afterwards, while Taxxons are feeding off of the body parts that missed his mouth.
I love three-dimensional Visser Three, who has a motivation for being "evil", and who does some truly twisted things that aren't downright stupid. I think that a lot of the canon books have him doing evil things just for the sake of "being evil', with no real rhyme or reason.
Clearly as time went on he somewhat lost his mind. I think that this is in part due to the fact that he was inhabiting Alloran, who was a few crayons short of a Crayola box. Together they made a rather unbalanced team.