I mean, maybe that is a possibility. I just never thought of it. I would recommend checking the official documentation. Might help shed some light on this.

I don’t think so, no.

Awesome

I see. I don’t think there there are many solutions on that front for Android. For PC there are a few, such as LM Studio.

Thanks for your comment. That for sure is something to look out for. It is really important to know what you’re running and what possible limitations there could be. Not what the original comment said, though.

This is all very nuanced and there isn’t a clear cut answer. It really depends on what you’re running, for how long you’re running it, your device specs, etc. The LLMs I mentioned in the post did just fine and did not cause any overheating if not used for extended periods of time. You absolutely can run a SMALL LLM and not fry your processor if you don’t overdo it. Even then, I find it extremely unlikely that you’re going to cause permanent damage to your hardware components.

Of course that is something to be mindful of, but that’s not what the person in the original comment said. It does run, but you need to be aware of the limitations and potential consequences. That goes without saying, though.

Just don’t overdo it. Or do, but the worst thing that will happen is your phone getting hella hot and shutting down.

For me the biggest benefits are:

• Your queries don’t ever leave your computer
• You don’t have to trust a third party with your data
• You know exactly what you’re running
• You can tweak most models to your liking
• You can upload sensitive information to it and not worry about it
• It works entirely offline
• You can run several models

I am not entirely sure. My best piece of advice here would be: check their in depth documentation on GitLab. It might answer your questions.

That’s great news! I’d love for it to be added to a wiki. Just make sure that whatever version of this post is added to the wiki is the most updated one.