I agree and it’s one of the biggest places where OSM is completely outclassed.
I hope OSM reaches a point where house number datasets are imported on a large scale, even if only for major cities. Adding numbers by hand is possible but very tedious, and unlike most mapping done for OSM you can’t just go off of what sat imagery shows.
15 or so years ago I did a rough and dirty implementation of approximate addresses using the idea of just dividing street segments up by the address numbers on them and going from there. For instance, in the Canadian Road Network Files, they provide smallish segments of streets that usually line up to things like cross streets in metro areas, and they come with the ranges of the street numbers in the metadata, so you’d get something like a starting value of say 200 and and an ending value of 212 for a section of, say, Yonge St, and you could just divide that segment up across those values directly. You’d generally get within a few metres of the correct address. Close enough at the time for our use cases, at least. For more rural areas it didn’t work out so well, but for metro areas it was actually pretty decent. This could all be done via a single Postgres/PostGIS query with the right inputs and address parsing in front of it.
It wasn’t perfect and later came various APIs and whatnot for doing this sort of stuff, but it was pretty decent for such a relatively simple implementation.