The way I interpret it, the core issue is often hypersensitivity to external stimulation. Those of us with only mild forms only have trouble with things like social skills but more severe forms are are so problematic because the person can't process and make sense of the sights, sounds, smells, ect, their brain is recieving from the world around them. My brain only overloads when there are too many facts at once, like math, or I'm trying to remember a bunch of things at once. It often seems like I can't remember common sense things.
I hope I at least came close to being right there. That's how I've heard it described even though I don't feel that way, probably because I'm only mild.