The problem is having a competent team to manage your infrastructure. You can do a lot with a handful of people - but you need competences spanning a lot of areas, and finding that is pretty hard.
If you can get a competent team the only advantage cloud still has is the ability to quickly scale up and down - but if there might be a need for that it’d still be better to go hybrid, most on your own hardware, and just the prepared ability to quickly bring up cloud workers if needed. The cost savings of properly doing it yourself are so huge that it still might be cheaper to just have some pre-provisioned standby hardware for that, though.
Depends on your needs. If you expect to grow fast and unpredictably, or have extreme burst workloads (at my company it fluctuates between requiring ~10 cpus to ~50,000, and between 0 GPUs and dozens) or if you need several complex types of services and no people at hand who can manage them, it can be way cheaper. If you just need a few servers, a tape backup and a database, actual hardware has always been cheaper.
