It’s because all those words have the same root:

• Either from the Proto-West Germanic “burg”, which means fortification/fortified town (maybe goes even back further to the Proto-Germanic “burgz”)

• or possibly the Ancient-Greek word “pýrgos”, which was a tower fort

• or maybe something else. Proto-Celtic or if we go way back: the Proto-Indo-European bʰergʰ- “to protect”

There’s a lot of speculation involved to where exactly the roots are, but in the end it all means a small castle or fortified town in which the “burger” lived.