Authors using a new tool to search a list of 183,000 books used to train AI are furious to find their works on the list.
Trick is educating the octogenarians in the senate to understand any of what you just wrote.
I’d say they should have to follow the most-restrictive license of all of their training data, and that existing CC/FOSS licenses don’t count because they were designed for use in a pre-LLM world.
It seems like a pretty reasonable request. But people like free stuff, and when they think about who will get screwed by this they like to imagine that they’re sticking it to the biggest publishers of mass media.
But IRL, those publishers are giddy with the idea that instead of scouting artists and bullying them into signing over their IP, they can just summon IP on demand.
The people who will suffer are the independents who refused to sign over their IP. They never got their payday, and now they never will either.
The people I’m seeing outraged are artists and authors who did not sign their ideas over for public access or for disingenuous use. not a faceless publisher with cloth bags and dollar signs painted on them. Also I don’t think you understand what public and private ownership means. A person is allowed to privately own their own creation. They don’t owe that to the world. The world isn’t entitled to it.
Here’s an idea, legally force companies like OpenAI to rely on opt-in data, rather then build their entire company on stealing massive amounts of data. That includes requiring to retrain from scratch. Sam Altman was crying for regulations for scary AI, right?
Would search engines only be allowed to show search results for sources that had opted in? They “train” their search engine on public data too, after all.
They aren’t reselling their information, they’re linking you to the source which then the website decides what to do with your traffic. Which they usually want your traffic, that’s the point of a public site.
That’s like trying to say it’s bad to point to where a book store is so someone can buy from it. Whereas the LLM is stealing from that bookstore and selling it to you in a back alley.
First: There are mechanisms to opt out (robots.txt and meta noindex)
Second: There is some foreknowledge on the part of the web author. Even in the early days of the web — before you could’ve predicted the concept of search engines — in order to distribute anything you had to understand the basics of hypermedia, among which is the idea that anything can link to anything else and clients can be users or machines alike.
Third: Even though you are correct that search engines are tokenizing text and doing statistical analysis to recombine the tokens into novel forms in order to rank against queries, those novel forms are never presented to the user. Only direct quotes. So a user never gets a false reference to the supposed content of a page (unless the page itself lies to crawler requests).
Fourth: All of the technical points above are pretty much meaningless, because we are social creatures and our norms don’t stem from a mechanical flow chart divorced from real-world context.
Creators are generally okay with their content being copied into search DBs, because they know it’s going to lead to users finding the true author of those words, which will advance their creative pursuits either through collaboration or monetary support.
Creators are complaining about content being copied into LLMs, because their work will be presented out of context, often cited incorrectly, keep people away from the author of those words, and undermine the lifeblood of their creative pursuits – be it attracting new collaborators or making sales.
Whether it technically counts as IP infringement or not under current law? Who really cares? Current IP law is a fucking scam, designed to bully creators out of their own creations and assign full control to holding companies who see culture as nothing more than a financial instrument to be optimized. We desperately need to change IP law anyway – something that I think even many strident “AI” supporters agree with – so using it as a justification for the ethics of LLMs reveals just how weak the group’s position truly is.
LLM vendors see an opportunity for profit, if they can get away with it. They are offering consumers a utopian vision of infinite access to content while creating an IP chokepoint that they can enshittify once it blows past critical mass. It’s the same tactics the social media companies used 15 years ago, and it weighs heavy on my heart that so many Lemmy users are falling for it once again while the lesson is still so fresh.
This is no different than every other capitalist enterprise. The whole system works on taking a public resource, claiming private ownership of it, and then selling it back to the public for profit.
First it was farmland, then coal and minerals, oil, seafood, and now ideas. Its how the system works and is the whole reason people have been trying to stop it for the past 150 years.
The people making the laws are there because they and/or their parents and/or grandparents did the exact same thing. As despicable and corrupt as it is you won’t change it by complaining and no-one is going to make a law to stop it.
I certainly hope that none of these authors have ever read a book before or have been inspired by something written by another author.
That would be a much better comparison if it was artificial intelligence, but these are just reinforcement learning models. They do not get inspired.
just reinforcement learning models
…like the naturally occuring neural networks are.
The brain does not work the way you think… (I work in the field, bio-informatics). What you call “neural networks” come from an early misunderstanding of how the brain stores information. It’s a LOT more complicated and frankly, barely understood.
Tell you what, you get a landmark legal decision classifying LLM as people and then we’ll talk.
Until then it’s software being fed content in a way not permitted by its license i.e. the makers of that software committing copyright infringement.
That’s an interesting take, I didn’t know software could be inspired by other people’s works. And here I thought software just did exactly as it’s instructed to do. These are language models. They were given data to train those models. Did they pay for the data that they used to train for it, or did they scrub the internet and steal all these books along with everything everyone else has said?
Well, now you know; software can be inspired by other people’s works. That’s what AIs are instructed to do during their training phase.
Does that mean software can also be afraid, or angry? What about happy software? Saying software can be inspired is like saying a rock can feel pain.
Software cannot be “inspired”
AIs in their training stages are simply just running extreme statistical analysis on the input material. They’re not “learning” they’re not “inspired” they’re not “understanding”
The anthropomorphism of these models is a major problem. They are not human, they don’t learn like humans.
They weren’t given data. They were shown data then the company spent tens of millions of dollars on cpu time to do statistical analysis of the data shown.
A computer being shown data is a computer being given data. I don’t understand your argument.
These are machines, though, not human beings.
I guess I’d have to be an author to find out how I’d feel about it, to be fair.
If an AI “reproduces” a work it was trained on it is a failure of an AI. Why would anyone want to spend millions of dollars and devote oodles of computing power to build something that just does what a simple copy/paste operation can accomplish?
When an AI spits out something that’s too close to one of the original training set that’s called “overfitting” and it is considered an error to be corrected. Most overfitting that’s been detected has been a result of duplication in the training set - when you hammer an AI image generator in training with thousands of copies of the Mona Lisa it eventually goes “alright, I get it already, when you say ‘Mona Lisa’ you want that exact pattern!” And will try its best to replicate that pattern when you ask it to later. That’s why training sets need to be de-duplicated.
AIs are meant to produce new things.
Did you write a comment on Reddit before 2015? If so, your copyrighted content was used without your permission to train today’s LLMs, so you absolutely get to feel one way or another about it.
The idea that these authors were somehow the backbone of the models when any individual contribution was like spitting in the ocean and model weights would have considered 100 pages of Twilight fan fiction equivalent to 100 pages from Twilight is honestly one of the negative impacts of the extensive coverage these suits are getting.
Pretty much everyone who has ever written anything indexed online is a tiny part of today’s LLMs.
Thank you for your reply.
On a completely separate note, it’s funny to think that there exists Twilight fan fiction when Twilight itself started as fan fiction work.
Edit: I dun goofed.
These are machines, though, not human beings.
What’s the difference? On the most fundamental level it’s all the same.
A human, regardless of how many books they read, will have personal experiences that are undeniably unique to themselves. They will interpret the works they read differently from each other based on their worldly experiences. Their writing, no matter how many books they read and get inspired on, will always be influenced by their own personal lives. They can experience love, hate, heartbreak, empathy, sadness, and happiness.
This is something a LLM does not have, and in my opinion, is a massive distinguishing factor. So on a “fundamental” level, it is not the same. It is no where near the same.
Unless you think theres no difference between killing a person and closing a program, I think we can agree they should be treated differently in the eyes of the law.
And so theres a difference between a person reading a book and being inspired by it, and someone writing a program that automatically transforms the book in data that can create new books.
The same thing as with tooooooons of things: scale.
Nobody cares if one dude steals office supplies at work. Now, if everyone stats doing it, or if the single guy steals everything, then action is taken.
Nobody cares if a random person draws in the same style and with same characters as you, but if they start to sell them, or god forbid, out-sell you, then there is a problem.
Nobody cares (except police I guess) if a random driver drives double the speed limit and annoys people living next to the road on the weekends, but when tons of people do it, you get speed bumps.
Nobody cares if few people pirate movies, but when it gets to mainstream and companies notice that there might be money being lost. Then you get whatever we have now.
Nobody cares if the mudhill behind your house erodes a bit and you get mud on your shoes. Have a bunch of that erode and you realise the danger…
You have been fine-tuning your own writing style for a decade and random schmuck starts to write similarly, you probably don’t care. No harm done. Now, get an AI to write 10 000 books in a weekend and someone starts to sell them… well now you have a completely different problem.
On a fundamental level the exact same thing is happening, yet action is only taken after a certain threshold is step over.
Are you saying the writers of these programs have read all these books, and were inspired by them so much they wrote millions of books? And all this software is doing is outputting the result of someone being inspired by other books?
Clearly not. He’s saying that other authors have done the same as the software does. The software creators implemented the same principle into their llm. You are being daft on purpose.
It’s not the same principle. Large language models aren’t ‘inspired’ to write new works. Software can’t be inspired. It follows instructions. Even though large language models might feel like somebody is talking back to you and giving you new information, it’s just code following instructions designed to predict output based on the input provided and the data supplied. There’s no inspiration to be had, and to attribute inspiration to language models is a huge mischaracterization of what’s happening under the hood. Can a language model, without being told what to do, actually use any of the data it was fed to create something? No. Every single large language model requires some sort of input from a user to act as a seed before any sort of response can begin.
This is why it’s so stupid to call this shit AI, because people start thinking it’s actual intelligence. Really, It’s just a fancy illusion.
They purchased their books to get inspiration from, the original author gets paid, and the author consented to selling it. That’s the difference.
Also the LLM can post entire snippets or chapters of books, which of course you’ll take at face value even if it hallucinates and makes the author look like a worse author then they are.
Generally they probably bought the books they read though.
If George RR Martin torrented Tolkien, wouldn’t he be infringing on the copyright no matter how he subsequently incorporated it into future output?
I completely agree that the training as infringement argument is ludicrous.
But OpenAI exposed themselves to IP infringement by sailing the high seas in how they obtained the works in the first place.
I hate that the world we live in is one where so much data is gated behind paywalls, but the law is what it is, and if the government was going to come down hard on Aaron Swartz for trying to bypass paywalls for massive amounts of written text, it’s not exactly fair if there’s a double standard for OpenAI doing the same thing in an even more closed fashion.
But yes, the degree of entitled focus on the premise of training an AI as equivalent of infringing is weird as heck to see from authors drawing quite clearly from earlier works in their own output.
I have to assume that openAI also paid for the books. if yes then i consider it the same as me reciting passages from memory or coming up with derivative text.
if no, then by all means, go after them and any model trainer for the cost of one book.
Asking an LLM to recite an entire novel isn’t even vaguely a thing yet.
Well, here’s straight from one of the suits against them:
“The OpenAI Books2 dataset can be estimated to contain about 294,000 titles. The only ‘internet-based books corpora’ that have ever offered that much material are notorious ‘shadow library’ websites like Library Genesis (aka LibGen), Z-Library (aka B-ok), Sci-Hub, and Bibliotik. The books aggregated by these websites have also been available in bulk via torrent systems.”
I’m not even sure how they would have logistically gone about purchasing 294,000 books in bulk in digital form to be fed into training. Using the existing collections seems much more likely, but I suppose we’ll see what turns up in litigation.
Also, the penalty for downloading copyrighted material if willful infringement is up to $250,000 per work. So it’s quite a bit more than the cost of one book on the line…
I hope they can at least get compensated.
Did you ever comment on Reddit before 2015? If so, your copyrighted material was used to train the modern LLMs even if your published book wasn’t used at all.
The database is here. You’ll have to sign up for a free trial if you’re not a subscriber to The Atlantic already. https://www.theatlantic.com/technology/archive/2023/09/books3-database-generative-ai-training-copyright-infringement/675363/
What about my Reddit history?
Arguably there’s more of my text there that was used to train these LLMs than most authors in that list.
The comment elsewhere in this thread about models built on broad public data needing to be public in turn is a salient one.
IP laws were designed to foster innovation, not hold it back.
I’d much rather see a world where we have open access models trained broadly and accelerating us towards greener pastures than one where book publishers get a few extra cents from less capable closed models that take longer for us to reach the heyday where LLMs can do things like review the past 20 years of cancer research in order to identify promising trends in allocation of future resources.
OpenAI should probably rightfully be dinged for downloading copyrighted media the same way any average user would be sued when caught doing the same.
But the popular arguments these days for making training infringement are ass backwards and a slippery slope to a far more dystopian future than the alternative.
they were compensated when the company using the book, purchased the book. you can’t tell me what to do with the words written in the book once I’ve purchased it. nor do you own the ideas or things I come up with as a result of your words in your book. of course this argument only holds up if they purchased the book. if it was “stolen” then they are entitled to the $24.95 their book costs.
That’s the thing – they weren’t.
The case has two prongs.
One is that training the AI on copyrighted material is somehow infringement, which is total BS and a dangerous path for the world to go down.
The other is that copyrighted material was illegally downloaded by OpenAI, which is pretty much an open and shut case, as they didn’t buy up copies of 100k books, they basically torrented them.
And because of ridiculous IP laws bought by industry lobbyists in the dawn of the digital age, the damages are more like $250,000 per book if willful infringement, not $24.95.
Had they purchased them, these cases would very likely be headed for the dumpster heap.
That said, there’s a certain irony to Lemmy having pirate subs as one of the most popular while also generally being aggressively pro-enforcement on IP infringement.
Training AI on copyrighted material is infringement and I’ll die on that hill. It’s use of copyrighted material to create a commercial product. Doesn’t get any more clear cut than that.
I know as an artist/musician/photographer I’d rather not put my creations out there at all if it means some corporation is going to be able to steal it.
