I really like that notion.
I think the reason it wouldn’t work (at least as you’ve described) is the myriad of sub-governments (and therefore smaller elections) that can exist for each voter.
My city does town council elections, my county does its board of supervisor elections, plus an occasional county ordinance vote, plus state elections and ballot initiatives, and then our federal president and congress elections all on the same ballot. If I move to a new city, up to half of the relevant people to vote for could change - probably closer to 3/4 if I moved states.
It’s always the same for every kind of election, from EU parliament down to local initiatives: The government has and does the job of telling and inviting you.
But, so how does that work for people who move right around election time?
They get their information to the address that is valid at the date when the invitations are being sent.
One can order postal services to send letters from old to new addresses for a certain time (did months or so). This services are pretty commonly used so no letters get lost.