Disclaimer: I wrote this article and made this website.
There was some talk of this issue in the recent fediverse inefficiencies thread. I’m hopeful that in the future we’ll have a decentralized solution for file hosting but for now I deeply believe that users should pay for their own file hosting.
We totally need sustainable file hosting. Freedom!
Wait… the fuck did you just upload? Oh god. Oh god no. Do I have to call the cops on you? Oh no. Wait, does this count as possession? FUCK!!!
We need someone else to handle the totally sustainable file hosting. Freedom!
Yep, there needs to be moderation tools that can be quickly deployed to stop the illegal/immoral/evil stuff from spreading and taking over self-hosted servers.
And moderation of this kind of content almost always sounds like torture when you hear about what facebook and the like are outsourcing.
Theoretically, this is a good problem for computer vision/machine learning. But there are a LOT of false positives (I think it was Aftermath who did an article on a study of when a nipple becomes female?). And… what ethical responsibility do you have to report on the fiftieth time that SheIsReallyAnEightThousandYearOldDragon_6969 uploaded CSAM? And how quick do you think people are going to lose faith in you and start wondering if you’ll also report on the rampant piracy?
And… there are also false negatives. At which point you find out you have been hosting something truly heinous for the past few months… possibly when local law enforcement tells you.
Like a lot of things: it sounds great. But nobody in their right mind is going to host this for free. And once you start accepting money you start opening yourself up to a LOT of regulations.
Illegal I can begrudgingly agree with. Even though I am a proponent for piracy, I will conceed that for growth’s sake, the tools need a clear well defined path to moderation.
That being said, who’s to say what IS immoral and evil?
In the republicans minds, porn is evil and should be banned. Trans rights are evil and should be banned. Abortion is evil and should be banned.
I disagree with all those claims. I do not think any of them are immoral, or evil.
I think pineapple on pizza is wrong, and evil. Some agree, others don’t. If I had my way, promoting of pineapple on pizza would be banned.
Now, who’s to say what is, and what isn’t evil? I think the only clear line to a moderation approach is to have a clear, unquestionable set of rules. These rules are to be based on public laws.
Everything else, I feel you should have the freedom to do as you wish. But also, I believe other people that you don’t agree with should be free to do as they wish.
You may never know how someone feels, or understand their perspective, but as long as they aren’t breaking laws, I feel they should have the ability to feel that way consequence free.
I may not like that you put pineapples on your pizza, but I feel that you should have the right to enjoy it. Even if it goes against MY views as to what constitutes a REAL pizza! Much to my surprise, pineapple on pizza ISN’T illegal. So you should have the right to enjoy it…
And yes. I did take the most pandtentic example I could think of, in order to display the absurdity of the concept of how easy it is to accept others rights in this world that don’t affect you.
Now just apply that same concept to every other example in the world. Then take into consideration that by using vague undefined terms to define your rules, you create grey area that’s easy to exploit. Who’s to say what IS evil? Adults told their teenagers in the 1950s that Elvis was evil. Parents in the 1920s told their teenagers that jazz was evil.
We need to define the terms that define our rules.
Seems to me that this is a use-case Freenet Hyphanet would be good for, both because it distributes the problem of file storage load and because it eliminates responsibility for each host to police his node by making it impossible for anyone to know which file chunks said node is hosting.
Nothing solves the problem of CSAM quite like… making everyone partially culpable in the storage and distribution of CSAM.
You can’t prove I was hosting child porn. Statistically, we all only had a 70% probability of having it on our computers
Stuff that isn’t accessed eventually gets deleted. If the Lemmy instances (which are clearnet, of course) delete the references to it, it would go away.
Ok, hear me out.
We find the users with the slowest internet and start sending them all the data. They don’t have to keep anything on disk. Then they send it all back and forth between each other. Any time a user makes a request, we just wait for one of the slow nodes to come across the data and send it out.
We use the slowest wires for all the storage. It’s fool proof.
Somebody actually did make this as a joke years ago haha https://github.com/yarrick/pingfs
I was brushing my teeth when reading this comment and inadvertently ended up swallowing all my toothpaste.
You jest but… delay line memory
Too wet for server racks in the forest.
Have you considered providing something like this: https://jortage.com/ and maybe contribute to their efforts to develop a specific API for that? Source code is here: https://github.com/jortage
Jortage is a really interesting approach. It definitely helps reduce the impact of the file hosting problem but it doesn’t fully address the underlying cost issue. The cost of storing files grows every month indefinitely while donations typically don’t.
I would like to see a file hosting pool come to lemmy though. So I will look into it. :)
Pict-rs that is used by Lemmy to store images already supports S3 type storage, so in theory it should work with Jortage, but I don’t think anybody has tested that yet. The people behind Feddit.org might have experimented with it as they expressed interest a while back.
I think the major advantage is the deduplication - when an image goes viral across Mastodon (or Lemmy) it’s currently stored hundreds or thousands of times, each with its own cost. Do you dedupe (for either your customers’ benefit or your own)?
Are the images duplicated when shared? My understanding is that only a link to the file is replicated across servers and duplication comes from users manually uploading the same file to another server.
My website does not do any deduplication at this time.
IPFS?
as I stated in this comment it’s not really feasible as to ~5s delay that was tested some time ago.
What would an IPFS solution look like here? That’s a genuine question. I don’t have much experience with IPFS. It seems like it isn’t really used outside of blockchain applications.
The sustainability of it is questionable. If I’m not mistaken, IPFS is based on Ethereum, which has gone over to proof of stake rather than proof of work, but it’s still a pretty cumbersome system.
We’re talking about something that needs to compete with Quic and CloudFlare. I’m not sure that Ethereum or even crypto itself is efficient enough as a content delivery method, that IPFS - though a nice idea - is unrealistic.
But that’s just speculation from someone who has zero knowledge behind IPFS as a technology and protocol, so take it with a grain of salt.
EDIT: honestly, why qualify with “I’m not sure” when besserwissers and their alts roam the fediverse instead of going to therapy. Smh. Give the people a Tl;Dr at least. I’m not here for long form content.
IPFS has absolutely nothing whatsoever to do with Ethereum, or indeed any blockchain. It is a protocol for storing distributing and addressing data by hashes of the content over a peer to peer network.
There is however an initiative to create a commercial market for “pinning*”, which is blockchain based. It still has nothing to do with Ethereum, and is a distinct project that uses IPFS rather than being part of the protocol, thankfully. It is also not a “proof of work” sort of waste, but built around proving content that was promised to be stored is actually stored.
Pinning in IPFS is effectively “hosting” data permanently. IPFS is inherently peer to peer: content you access gets added to your local cache and gets served to any peer near you asking for it—like BitTorrent—until it that cache is cleared to make space for new content you access. If nobody keeps a copy of some data you want others to access when your machines are offline, IPFS wouldn’t be particularly useful as a CDN. So peers on the network can choose to pin some data, making them exempt from being cleared with cache. It is perfectly possible to offer pinning services that have nothing to do with Filecoin or the blockchain, and those exist already. But the organization developing IPFS wanted an independent blockchain based solution simply because they felt it would scale better and give them a potential way to sustain themselves.
Frankly, it was a bad idea then, as crypto grift was already becoming obvious. And it didn’t really take off. But since Filecoin has always been a completely separate thing to IPFS, it doesn’t affect how IPFS works in any way, which it continues to do so.
There are many aspects of IPFS the actual protocol that could stand to be improved. But in a lot of ways, it does do many of the things a Fediverse “CDN” should. But that’s just the storage layer. Getting even the popular AP servers to agree to implement IPFS is going to be almost as realistic an expectation as getting federated identity working on AP. A personal pessimistic view.
