Would you give a 2nd thought to a sleepover invitation if it was with a single dad instead of a single mum? Let's assume for argument's sake that you've known the kid in question for a few years and had heaps of school / social activity interaction with both parents.
If I'd known them for years and was comfortable with them, then i would give it a second thought, and id probably let them go. If they were someone I'd just met then yes I would give anyone (male or female) a second, third and fourth thought!
I wouldn't be at all surprised at someone ummming and ahhhing over their child sleeping over at a mans house. Your level of acceptance is personal. So is theirs.
The horrid fact of the matter is pedophiles live in the burbs with you and me. Not down dingy lane ways where we would never stray. And the fact the vast majority are men. You can't tell who is one by looking at them, and you never can know someone well enough IMO. So, to be human and normal about it without being neurotic, you have to rely on your instincts and go with your gut.
if someone is uncomfortable about it then so be it. They have their reasons. It may have more to do with them than the fellow.
I remember the pressure from the kids in early primary, (like kindly and y1 ) constantly asking for sleepovers. I was so uncomfortable with it, I think they're too young.and we knew no one that well from school. With my next schoolie it'll be a flat out NO. I remember some parents said "were not doing sleep overs this year" and that was that. No argument. I ll be that parent.