Perhaps they coexisted in a way, if ants even existed back then and I don't know if they did, it's possible that the Nesk considered them a good indicator that Earth would be a fine place to live, like how humans sent chimps into space. They could have regarded ants in general the same way humans regard chimps, i.e. a similar species but not actually equal that had best stay out of our way or we'll put them in a world of hurt but whom we otherwise don't care much about except to say "aww, he's like a wittle person!"
It's also possible that other Nesk "colonies" existed and might not have been evacuated before the planet was hit, if they were far enough away these Nesk could have found themselves without transportation if they were stranded, or if their ships were somehow damaged. They may have had a decline in civilization and been reduced to the level of ants, going from a sentient hive mind thingy to a more simple minded one might not be a devolution, but rather survival of the fittest after millions of years, simple minded survival colonies outdoing and thereby outlasting higher thinking colonies until the modern day ants were the result.
If you think a sentient species can't get dumber you clearly haven't met the people I've met.