So I signed up for a free month of their crap because I wanted to test if it solves novel variants of the river crossing puzzle.
Like this one:
You have a duck, a carrot, and a potato. You want to transport them across the river using a boat that can take yourself and up to 2 other items. If the duck is left unsupervised, it will run away.
Unsurprisingly, it does not:
https://g.co/gemini/share/a79dc80c5c6c
https://g.co/gemini/share/59b024d0908b
The only 2 new things seem to be that old variants are no longer novel, and that it is no longer limited to producing incorrect solutions - now it can also incorrectly claim that the solution is impossible.
I think chain of thought / reasoning is a fundamentally dishonest technology. At the end of the day, just like older LLMs it requires that someone solved a similar problem (either online or perhaps in a problem solution pair they generated if they do that to augment the training data).
But it outputs quasi reasoning to pretend that it is actually solving the problem live.
Yeah, exactly. There’s no trick to it at all, unlike the original puzzle.
I also tested OpenAI’s offerings a few months back with similarly nonsensical results: https://awful.systems/post/1769506
All-vegetables no duck variant is solved correctly now, but I doubt it is due to improved reasoning as such, I think they may have augmented the training data with some variants of the river crossing. The river crossing is one of the top most known puzzles, and various people have been posting hilarious bot failures with variants of it. So it wouldn’t be unexpected that their training data augmentation has river crossing variants.
Of course, there’s very many ways in which the puzzle can be modified, and their augmentation would only cover obvious stuff like variation on what items can be left with what items or spots on the boat.
It’s just overtrained on the puzzle such that it mostly ignores your prompt. Changing a few words out doesn’t change that it recognises the puzzle. Try writing it out in ASCII or uploading an image with it written or some other weird way that it hasn’t been specifically trained on and I bet it actually performs better.
oh look it’s a loadbearing “just” in the wild. better hope you can shore that fucker up with some facts
Try writing it out in ASCII
my poster in christ, what in the fuck are you on about. stop prompting LLMs and go learn some things instead
some other weird way that it hasn’t been specifically trained on and I bet it actually performs better
“no no see, you just need to prompt it different. just prompt it different bro it’ll work bro I swear bro”
god, every fucking time
All along my mistake was that I was prompting it in unicode instead of latin1, alphameric BCD, or “modified UTF-8”.
Well has anyone tried prompting it in EBCDIC? How do we know doing so won’t immediately create the super intelligence that "or whatever"s us to silicon Valhalla? Asking for a friend.
write it out in ASCII
My dude what do you think ASCII is? Assuming we’re using standard internet interfaces here and the request is coming in as UTF-8 encoded English text it is being written out in ASCII
Sneers aside, given that the supposed capability here is examining a text prompt and reason through the relevant information to provide a solution in the form of a text response this kind of test is, if anything, rigged in favor of the AI compared to some similar versions that add in more steps to the task like OCR or other forms of image parsing.
It also speaks to a difference in how AI pattern recognition compared to the human version. For a sufficiently well-known pattern like the form of this river-crossing puzzle it’s the changes and exceptions that jump out. This feels almost like giving someone a picture of the Mona Lisa with aviators on; the model recognizes that it’s 99% of the Mona Lisa and goes from there, rather than recognizing that the changes from that base case are significant and intentional variation rather than either a totally new thing or a ‘corrupted’ version of the original.
Exactly. It’s overtrained on the test, ignoring the differences. If you instead used something it recognises but doesn’t recognise as the test pattern (having the same tokens/embeddings) it will perform better. I’m not joking, it’s a common tactic to get around censoring. You’re just going around the issue. What I’m saying is they’ve trained the model so much on benchmarks that it is indeed dumber.
Not really. Here’s the chain-of-word-vomit that led to the answers:
Note that in “its impossible” answer it correctly echoes that you can take one other item with you, and does not bring the duck back (while the old overfitted gpt4 obsessively brought items back), while in the duck + 3 vegetables variant, it has a correct answer in the wordvomit, but not being an AI enthusiast it can’t actually choose the correct answer (a problem shared with the monkeys on typewriters).
I’d say it clearly isn’t ignoring the prompt or differences from the original river crossings. It just can’t actually reason, and the problem requires a modicum of reasoning, much as unloading groceries from a car does.
