We have two kids in a 3 bed/2 bath 1350 sq ft home. We do have a full basement, but the kids aren’t really allowed down there (power tools, toy stash, etc). I guess I do hang out there some nights, but that’s only because my gaming computer moved downstairs years ago when our oldest started to be able to reach the keyboard and pull key caps off it.
In our experience, you’re probably not going to do a lot of entertaining while you have young kids. While one of your kids is under 3-4, and sometimes older, they’re going to need naps. They’re also going to have early bedtimes. Naps are mostly behind us, and we do have afternoon play dates, but the kids don’t really care what space they’re in as long as they’re engaged and have things to do. Having an adult gathering is… very rare. We have a nice sized yard, so we tend to have gatherings outside.
I don’t think we need extra bedrooms or bigger bedrooms/bathrooms. An office might be nice, but working from the basement works just as well. A toy room could be nice, but to me it would be wasted space as the kids get older and have fewer, but larger/more engaging, toys. At least around here, the extra room comes with extra walls that result in a space that’s not often used (think a formal dining room).
There’s also the financial side of things. We could afford a larger house, but would rather be putting any extra into 529s, our own 401ks, etc. Kid related expenses really add up before you start also thinking about a bigger mortgage payment.