I think it was just a string of too many odd coincidences that finally made Visser Three look at the bigger picture and realize that they really are human. Let's look at them, shall we?
His advisers were telling from the start that the bandits might (emphasis on might, don't want to get decapitated!) be human. They saw a bunch of kids at a construction site, around an Andalite, and not long after that the "Andalite bandits" appeared. A group of Yeerks at the beach saw suspicious goings-on, with clothing being left behind and kids - kids once again - disappearing into the water. And when they were at the Yeerk pool, a group of "slime-covered things" burst out of a Taxxon, and could obviously morph, but looked nothing like Andalites, and they have to pass through an intermediate stage when they demorph and remorph. Also, Ax talked too much. ALL THE TIME. In fact, the Animorphs never once thought to talk more to Visser Three, to keep him thinking they're Andalites. And then in The Revelation, an "Andalite bandit" rescues a human, and once again, rescues Visser One, when there is nothing to be gained from it.
You see what I'm trying to get at? As Marco said: "The Yeerks can add two plus two."
