Do we know if local models are any safer or is that a trust me bro?
well we’re talking about data across a company. Tho apparently it does send stuff back to MS as well, because of course it does.
- don’t use any of this stupid garbage
- if you’re forced to deploy this stupid garbage, treat RAG like a poorly-secured search engine index (which it pretty much is) or privacy-hostile API and don’t feed anything sensitive or valuable into it
- document the fuck out of your objections because this stupid garbage is easy to get wrong and might fabricate liability-inducing answers in spite of your best efforts
- push back hard on making any of this stupid garbage public-facing, but remember that your VPN really shouldn’t be the only thing saving you from a data breach
Local models are theoretically safer, by virtue of not being connected to the company which tried to make Recall a thing, but they’re still LLMs at the end of the day - they’re still loaded with vulnerabilities, and will remain a data breach waiting to happen unless you make sure its rendered basically useless.
You can download multiple LLM models yourself and run them locally. It’s relatively straightforward;
Then you can switch off your network after download, wireshark the shit out of it, run it behind a proxy, etc.