Google has a deal with reddit as well. https://www.reuters.com/technology/reddit-ai-content-licensing-deal-with-google-sources-say-2024-02-22/?utm_source=reddit.com
But I don’t think it’s just an issue with the dataset. It’s the false promise of these LLMs having a fucking clue what a good search result is and what is not. They don’t. They are just good at creating text that sounds plausible. That’s not what searching for factually correct information is about though.
Thank you for the link. I didn’t realise that Google had a deal with Reddit as well, which explains why it was clearly indexed from Reddit.
I agree that AI doesn’t have a clue what an accurate response is. It’s just not sentient enough to differentiate between shitposting and fact. I also totally agree that an answer given from a search result HAS to be accurate, and we’re heading down a path of a misinformation super highway if LLMs are trained on incorrect data.
