Why do people host LLMs at home when processing the same amount of data from the internet to train their LLM will never be even a little bit as efficient as sending a paid prompt to some high quality official model?
inb4 privacy concerns or a proof of concept
this is out of discussion, I want someone to prove his LLM can be as insightful and accurate as paid one. I don’t care about anything else than quality of generated answers
“Why do people do X, when in my opinion if you disregard the two top reasons for doing X, it’s pointless? Prove to me that it would be better!?”
I’m not saying in advance that it’s pointless, it’s just your far-fetched preconception. I want to know if there are any further arguments for hosting LLMs besides the two I ruled out in advance for being too obvious. Furthermore, I think that since you moved so quickly to mockery, there are no other reasons than privacy and tinkering
Why would you need a reason other than privacy? Or for that matter any reason you want or nobe at all.
To be fair, he is just asking for more reasons than privacy. No reason not to give him them, and also okay to say we don’t think there is any.
Privacy would be the main concern. Every single one of your words, documents, pictures will probably end up in some large database over at OpenAI. I don’t like that at all. And as a company for example, it might be against the law to share some information about clients with third parties.
Then you don’t get any of the freedoms we got with Free Software. It’s a service you rely on with very little opportunities to customize, or look inside and tinker. There is little control for the user whatsoever. Additionally we already had companies cease service. So it might become unavailable tomorrow, which is a bad thing if you’re attached to it, invested or built things around it.
And since “the internet is for porn”… We also have a noteworthy community doing those kinds of things. And well… go ahead and ask the big services to generate a lewd story. Most of them even refuse to write a murder mystery story for me, instead they’ll lecture me on how it is not ethical to murder someone. So that would be use-cases where local AI outperforms any of the market leaders.
Personally, I’m a bit opposed to the entire concept of letting other people’s algorithms dictate my life. I don’t want to rely on them. I also don’t want them to pick the bias for my perspective on the world. The algorithms in social media are dwarfed by how dangerous it’s gonna be once people rely on AI more and more. And it gets to choose which information to show and which to drop. What kind of bias to introduce in summaries etc. Teach people how to think. And I already don’t like the way all big AI chatbots talk to me with a lot of emojis and in a “Explain like I’m 5 yo” way.
So to go back to the original question… I think the more “useful” AI is, the more reasons there are to retain some control yourself. What do you think?
Why do people host LLMs at home when processing the same amount of data from the internet to train their LLM will never be even a little bit as efficient as sending a paid prompt to some high quality official model?
inb4 privacy concerns or a proof of concept this is out of discussion, I want someone to prove his LLM can be as insightful and accurate as paid one. I don’t care about anything else than quality of generated answers
If you ask other people for their reasoning and opinions, it doesn’t really make any sense to put something “out of the discussion”, does it? :P
But no, if you have no qualms about sharing your innermost feelings, sexual preference or illegal plans with those that have an explicit desire to exploit that information then there is little reason to attempt something as complicated and wasteful as self-hosting your own LLMs.
want someone to prove his LLM can be as insightful and accurate as paid one.
The full DeepSeek model is available for download, and should generate about the same quality answers as the official one, with the bonus of less censorship. I pretty trivially got it to talk about the Tiananmen Square, and they can’t even ban me for it.
That said, that’s rarely the point. It’s usually because you can, a cost saving measure, sometimes you plainly just don’t need a good model, sometimes you want privacy, sometimes you need privacy at the cost of quality.
If your business is shoving customer reviews into a model, you really don’t need the best model for it to tell you how angry the customer is.
Personally I just do it for fun and because I can. Sometimes you just do things for no other reason than because you can.
Poor internet connection/no internet at all, network latency too high for their needs, specific fine tuned LLM ?
Off course, main reason is privacy. My company host its own GPT4 chatbot, and forbid us to use public ones. But I suppose there are other legit use case to host its own LLM.
