Classic case of “that’s not what LLMs are made for”?
It indeed is one of these cases. Individual letter recognition doesn’t work well, since they tend to be clumped up into larger “tokens”, which then can’t be differentiated anymore. That’s also the same reason why it can’t count the letters in words, It sees words as just a single, or at most three “tokens”
So a game of Hangman is the next level to compare AI to humans, above the Turing test.
Oh, how we’ll long for the good old days of stupid AI after the revolution.
Considering the code it spurts out, future AI will be doubly perplexed while still getting stuff wrong.
There’s one thing you guys are missing, yes it’s stupid but it also doesn’t have a memory. It cannot play hangman with a word it doesn’t really know. If you ask it to pre-select a word and keep it in chat then it plays fine
Perplexity:
MOENY