A powerful summoner should have no issue binding enough mimics to make up for each brick, stone and interior detail of a castle, only needing to conjure some slimes for mortar and elementals for lighting, heat, AC (air and ice for summer, air and fire for winter) plus enough water elementals to have running plumbing. Just a matter of time and focus

Just a bit heavy on the initiative tracking, but the castle would probably not need any extra guards