Silva- M always said that orphans make the best agents ... so noo.

Besides where would Silva be without parental issues?

Jim Moriarty - His backstory isn't really mentioned in canon but it's pretty obvious it wasn't a stable upbringing.

What with him being batcrap crazy and all.

I dont know if they're alive but we cant rule out the possibility Jim eat them.

Gary Sitterson- Headcanon says no they passed away six years ago.

He was sad about it but knew it to be a fact of life as you got older.