Let's play what if.
What if Sophia had 3 sons and 1 daughter?
Even though they're not, you could easily make Sam and Zach as Sophia's sons.
What amount gets too much?
I think 4 is on the edge of being too many for a story, I think 2 or 3 is best.
So let's say Sam, Dylan and Ellie were Sophia's children.
Do you think the story changes at all? in what way?
I don't think we see Alyssa and Neil as much. Julia would be an only child or there be another character, but to many characters as is, so the best thing would be to cut out the neighbours completely. Julia becomes Emma in Sophia's class. Ellie becomes friends with her and sees her crush on Sophia, there wouldn't be any Julia dating Dylan like we saw with Emma as Julia is not into guys.
I think some things change but most of Sam's scenes are in the main house, so wouldn't change much I think. Maybe some brotherly rivalry over Sophia but it be mostly scenes in the neighbour's house, the lessons would just move to the main house.