Greg Rutkowski, a digital artist known for his surreal style, opposes AI art but his name and style have been frequently used by AI art generators without his consent. In response, Stable Diffusion removed his work from their dataset in version 2.0. However, the community has now created a tool to emulate Rutkowski’s style against his wishes using a LoRA model. While some argue this is unethical, others justify it since Rutkowski’s art has already been widely used in Stable Diffusion 1.5. The debate highlights the blurry line between innovation and infringement in the emerging field of AI art.
I don’t fully understand how this works, but if they’ve created a way to replicate his style that doesn’t involve using his art in the model, how is it problematic? I understand not wanting models to be trained using his art, but he doesn’t have exclusive rights to the art style, and if someone else can replicate it, what’s the problem?
This is an honest question, I don’t know enough about this topic to make a case for either side.
Do you know how they recreated his style? I couldn’t find such information or frankly have enough understanding to know how.
But if they either use his works directly or works created by another GAI with his name/style in the prompt, my personal feeling is that would still be unethical, especially if they charge money to generate his style of art without compensating him.
Plus, I find that the opt-out mentality really creepy and disrespectful
“If he contacts me asking for removal, I’ll remove this.” Lykon said. “At the moment I believe that having an accurate immortal depiction of his style is in everyone’s best interest.”
I still have trouble understanding the distinction between “a human consuming different artists, and replicating the style” vs “software consuming different artists, and replicating the style”.
But if they either use his works directly or works created by another GAI with his name/style in the prompt, my personal feeling is that would still be unethical, especially if they charge money to generate his style of art without compensating him.
LORA’s are created on image datasets, but these images are just available anywhere. It’s really not much different from you taking every still of The Simpsons and using it. What I don’t understand is how these are seen as problematic because a majority of end users utilizing AI are doing it under fair use.
No one charges for LORA’s or models AFAIK. If they do, it hasn’t come across the Stable Diffusion discords I moderate.
People actually selling AI generated art is also a different story and that’s where it falls outside of fair use if the models being used contain copy-written work. It seems pretty cut and dry, artists complained about not being emulated by other artists before AI so it’s only reasonable that it happens again. If people are profiting off it, it should be at least giving compensation to the original artist (if it could be adjusted so that per-token payments are given as royalties to the artist). However, on the other hand think about The Simpsons, or Pokemon, or anything that has ever been sold as a sticker/poster/display item.
I’m gonna guess that a majority of people have no problem with that IP theft cause it’s a big company. Okay… so what if I love Greg but he doesn’t respond to my letters and e-mails begging him to commission him for a Pokemon Rutkowski piece? Under fair use there’s no reason I can’t create that on my own, and if that means creating a dataset of all of his paintings that I paid for to utilize it then it’s technically legal.
The only thing here that would be unethical or illegal is if his works are copywritten and being redistributed. They aren’t being redistributed and currently copy-written materials aren’t protected from being used in AI models, since the work done from AI can’t be copywritten. In other words, while it may be disrespectful to go against the artists wishes to not be used in AI, there’s no current grounds for it other than an artist not wanting to be copied… which is a tale as old as time.
TL;DR model and LORA makers aren’t charging, users can’t sell or copywrite AI works, and copywritten works aren’t protected from being used in AI models (currently). An artist not wanting to be used currently has no grounds other than making strikes against anything that is redistributing copies of their work. If someone is using this LORA to recreate Greg Rutkowski paintings and then proceeds to give or sell them then the artist is able to claim that there’s theft and damages… but the likelihood of an AI model being able to do this is low. The likelihood of someone selling these is higher, but from my understanding artistic styles are pretty much fair game anyway you swing it.
I understand wanting to protect artists. Artists also get overly defensive at times - I’m not saying that this guy is I actually am more on his side than my comment makes it out, especially after how he was treated in the discord I moderate. I’m more just pointing out that there’s a slippery slope both ways and the current state of U.S. law on it.
they charge money to generate his style of art without compensating him.
That’s really the big thing, not just here but any material that’s been used to train on without permission or compensation. The difference is that most of it is so subtle it can’t be picked out, but an artist style is obviously a huge parameter since his name was being used to call out those particular training aspects during generations. It’s a bit hypocritical to say you aren’t stealing someone’s work when you stick his actual name in the prompt. It doesn’t really matter how many levels the art style has been laundered, it still originated from him.
It is unconditionally impossible to own an artistic style. “Stealing a style” cannot be done.
Generally speaking, the way training works is this:
You put together a folder of pictures, all the same size. It would’ve been 1024x1024 in this case. Other models have used 768z768 or 512x512. For every picture, you also have a text file with a description.
The training software takes a picture, slices it into squares, generates a square the same size of random noise, then trains on how to change that noise into that square. It associates that training with tokens from the description that went with that picture. And it keeps doing this.
Then later, when someone types a prompt into the software, it tokenizes it, generates more random noise, and uses the denoising methods associated with the tokens you typed in. The pictures in the folder aren’t actually kept by it anywhere.
From the side of the person doing the training, it’s just put together the pictures and descriptions, set some settings, and let the training software do its work, though.
(No money involved in this one. One person trained it and plopped it on a website where people can download loras for free…)
Do you know how they recreated his style? I couldn’t find such information or frankly have enough understanding to know how.
I don’t, but another poster noted that it involves using his art to create the LoRA.
Plus, I find that the opt-out mentality really creepy and disrespectful
I don’t know about creepy and disrespectful, but it does feel like they’re saying “I know the artist doesn’t want me to do this, but if he doesn’t specifically ask me personally to stop, I’m going to do it anyway.”
TL;DR The new method still requires his art.
LoRA is a way to add additional layers to a neural network that effectively allow you to fine tune it’s behaviour. Think of it like a “plugin” or a “mod”
LoRas require examples of the thing you are targeting. Lots of people in the SD community build them for particular celebrities or art styles by collecting examples of the that celebrity or whatever from online.
So in this case Greg has asked Stable to remove his artwork which they have done but some third party has created an unofficial LoRA that does use his artwork to mod the functionality back in.
In the traditional world the rights holder would presumably DMCA the plugin but the lines are much blurrier with LoRA models.
