I was recently reading Ben Aaronvitch’s ‘Rivers of London’ and I was struggling to follow the plot. So I engaged in discussions with both ChatGPT and Gemini and both really struggled to not make lots of mistakes. Both AIs would regularly give inaccurate info about major plot points, and when I questioned them about those issues, they would apologize profusely and then try to continue on. I basically had to coach them through to get them to remember important points of the plot! They also struggled to not reveal spoilers past a certain chapter. It was interesting to see the AIs repeatedly fail.
That’s literally how the work: They can’t reason or think. They don’t know or understand anything. They just output strings of words that are statistically likely to be the response to your question based on strings of words that they’ve been trained on.
I’ve heard it referred to as spicy autocomplete and also next word extruder
I did have a much better discussion of ‘Moby Dick’ a while ago, but I imagine there is lots of info that AI has scraped from scholarly articles and the like about classics.
They don’t do well with big sources like books unless they’re discussed. If there’s no online references or scholarly articles it won’t know the contents of the book. Same with movies or music.
Right and even if it “knew” the contents it still couldn’t interpret them or have a discussion about the meaning.
Probability machine prone to hallucinations is how I call them.
No need to humanize them, they are Markov machines. That is all.
Yeah, they’re basically predicting the responses.
It’s always sad to see people anthropomorphising LLMs.