To be fair, the monster was artificial and intelligent.
You don’t create a thriving breakfast cereal brand without top tier smarts
Or Smurfs.
Loved it when he met The AI Bride of Frankenstein.
The Al Bride of Frankenstein?
I postulated exactly these issues with detecting LLM code in software projects, making “no LLM code”-type policies unenforceable. It doesn’t work most of the time now, and it will only get worse in the future.
It’s a 100% match on what the model was trained on.
AI slop checking AI slop.
We’re using AI to determine what is AI, even though AI doesn’t know what AI is.
Perfectly reasonable, no careers will be destroyed, surely
Yeah, the AI based “AI detectors” are themselves slop.
Shouldn’t it be easy with today’s computing power to simply search all content first for a match to known sources?
That would make sense if you wanted to do what was best for the user rather than push the Chosen Product.
Wouldn’t necessarily tell you if that known source is AI. So now we have to go through everything by hand anyway.
Of course it can, if it’s from 1859 the chances of it being AI are … I leave that as an exercise for the reader.
And how do you know the source for your “hit” isn’t lying? You are not describing a trivially easy problem at all.
It seems trivially easier than whatever the hell AI is.
And this is the “unseen” loss of value. Can you imagine writing a book (by hand the old-fashioned way – maybe with just a text editor) and submitting it to publish and the publisher rejects it as “written by AI?”
This is absolutely the biggest source of anxiety I have right now as someone who has been writing a really heartfelt novel for over a decade. (Things got delayed due to… personal stuff. Also I started drinking beer like a proper author. It didn’t help, actually.)
Back when I started this, I had been publishing some short stories under CC licences. I guess that has to be an option once again (and as much as I hate it as an open source nerd, -NC might make an appearance.)
Damn, GPTZero also flags it as 57% AI.
Uses Ai to detect the use of Ai
Ahead of her time. But, then again, we all knew that.
AI is like: “i made this”
confidently incorrect
Does it think it’s AI because it was trained on it so it recognizes it as being part of AI now?
It was posted like some kind of gotcha but…this book was in the dataset.
I’m aware that LLMs don’t keep the dataset in memory but it “knows” that this is an existing work but these sites aren’t doing any wild calculations to figure out if it was AI-generated. They basically just check to see if the sentences exist elsewhere and they do. In the original dataset.
So it was wrong to say it’s AI-generated but it correctly identified that it’s not original.
I’m sorry, what? It’s correctly identified that it’s what, now?
It’s original in the book. What the user entered is an exact copy.
“Original” is not the correct work but you know what I mean.
The little scale thing doesn’t say it is 100% not original, it says it is AI/LLM generated.
Yes…because the pasted text has some of the exact patterns that are part of the dataset that the website trained on…because that dataset contains text generated from a dataset containing the exact paragraph…
I feel like you’re being deliberately obtuse.
I feel like you just aren’t making sense and just need to maybe ask google for how any of this works
I feel like you don’t understand the difference between ‘AI generated’ and ‘something that existed over 200 years ago’.
I feel like you don’t understand that LLMs train on text that existed 200 years ago…
Yes, but if that were true, then every single book ever published would be flagged. The training data is for teaching it patterns, not just cross-referencing.
They analyze statistical patterns, they don’t cross reference the training data.
It’s picking up the text as generated because it’s mostly guess work and constantly spits out false positives, especially with non native speakers.
Honestly I didn’t expect them to flag non-native speakers. LLMs don’t often make gramatical mistakes (or at least not ones your average joe could identify)
Weirdly enough i saw a massive increase in little typoes on reddit this year. More typoes and such in a few months than i saw in years.
It feels like LLMs started making mistakes on purpose.
Foreign speakers often know English better than most Americans
Oooh yeah, the most common dumb mistakes are (by my own checking) done by Americans in the VAST majority cases. Stuff like not knowing how to use your/you’re, there/their/they’re, cloths/clothes, definitely/defiantly, where/were/we’re, its/it’s etc. correctly and the many other failings of basic grammar like “would/could of” and so much else.
I don’t think there’s another country in the world where the native speakers are so fucking bad at even the most basic shit.
The most common response I’ve gotten and seen when others correct stuff like that is “calm down, English probability isn’t their main language” yet when checking it’s been Americans who made the mistakes 99% of the time.I was going to say:
The difference is between growing up, learning a language by hearing and making the sounds, and learning it as a foreign language, having to understand and memorize the rules.
“They’re, their, there” all sound the same as well as “could’ve, could of”.But interestingly other English speakers don’t make those mistakes as often as people emerging from the american education system.
It’s plausible that Americans are worse at this than foreigners, but English spelling is a crime and everyone who makes mistakes ought be excused on account of how ridiculous the “rules” are.
I know how to spell most of the words I use and I resent the fact that my memory is polluted with this otherwise useless knowledge.
In many other languages if you know how to say a word, you automatically know how to spell it. We need a world language with grammar as simple as English and spelling as straightforward as German.
And are also less prone to use slang unless they go more deep in the culture
The decline I’ve seen in my time on the internet is genuinely shocking
ya we used 2 jst tlk lyk dis b4 t9 bcam irlvnt
Abrevs 4evr
Everything you said was hallucinated. Nothing you said was even slightly coherent or connected with reality. Absolutely nothing was correct. If you let your cat dance on the keyboard more sense would come out.
That must have sounded really clever and powerful in your head for you to hit send, huh? Couldn’t decide which zinger to go with so you sent all 4?
I’d have given you more credit if you just posted the quote from Billy Madison. Try harder lol
No it just annoys me when people have no understanding of how shit works and just make up an explanation instead of looking it up. Hell you could have asked chatGPT and probably got a correct answer.
That is not in the least bit how a tool like this works.
All AI detection is extremely unreliable, but they operate on principles which, if the assumptions supporting them were true, would be sound. The way you imagine they work is different: you’re describing a “novel text detector” which is not at all the same thing as an “AI text detector”.
AI detection works, at a very high level, by throwing a bunch of examples of AI text, and a bunch of examples of non-AI text, into a machine-learning model, and training it until it is able to recognise the AI examples as such. It doesn’t work by throwing in all existing text including novels written before AI, because that would never do what you want.
(The reason, if you’re interested, why this ends up not working is because the features such a model identifies as indicative of AI text are very sensitive: if you train it on Claude and ChatGPT, it will not work on Gemini output. If you train it on Gemini, when Gemini updates it will get worse. If someone generates text with a weird prompt, it may slip by. If someone writes in a weird way, it may get flagged. And if any AI company wanted to defeat AI detectors, it could trivially feed its output through one during the training process and give that output as examples to avoid in the training, so that the model learns to create output which doesn’t “look like” AI output to those detectors.)
I was being overly simplistic - I meant more that the patterns it’s trying to detect were created by an LLM trained on the data they’re inputting.
It’s like how reddit comments from 2016 look generated. If you stick one of those into an AI detector, it’ll give a false positive for the same reason
You weren’t simplifying what you said was just wrong what the person you are responding to is just correct. Just admit when you are wrong
They do keep it.
The phenomena of regurgitation is a strong evidence on it, for example.
Those detector sites are really evil. They simply do not work. They can’t work. But they are advertising and people are using them to “check” texts. Which then totally fucks students when their idiotic teacher uses such a site and accuses them of using AI tools.
I had to let out a depressed chuckle when someone referred to Pangram as “industry standard”. …just because that particular service had been the centre of the Shy Girl controversy and they had dragged themselves to the limelight again when another novel got accused of AI use.
The Shy Girl controversy is a doozy. People raised their eyebrows and thought the novel was written with AI. Pangram CEO elbowed his way to the limelight and proved, using his fine AI product, that this novel is [insert some absurdly specific high percentage] AI. The author was like “Well I handed my novel to someone else, and they used AI to copyedit the book!” The publisher was like “So you used someone else afterall, after declaring you were the sole author? Consider the publishing agreement revoked.” People were like “Due to AI use, right? 😀 Due to AI use, right? 😕” The publisher was like “…No, we have a few AI projects cooking.”
I was having a conversation with someone here who claimed to be a professor (I believe it) and went on and on about how they are a subject matter expert and that AI detection actually works really well these days. I told him I felt bad for his students and he had a full blown insult laden meltdown at me then deleted everything.
I just feel bad for those students.
I wonder if everything uploaded to them gets used as LLM training data
i worry about this. all my writing in grad school, likely fed to a machine.
