Seems like we as a society have a couple of options here:
Take this story as obvious marketing; downgrade Altman’s credibility yet again and move on
Take it seriously and demand that OpenAI cease operations until they can function responsibly
Take it seriously and throw yet more money at them
Unfortunately #3 seems like a strong contender in the current climate.
Just remind them that it’s natural to be curious about crab literacy, but they need to find a more healthy outlet for these feelings.
Ok, thanks for the detailed explanation. I guess if your goal is to make your model sound like another model that makes perfect sense.
The article is not clear on what a “distillation attack” is… what exactly is Alibaba supposed to be getting away with here? The article mentions using many different connections through obfuscation networks and proxies… so that would get them around rate limiting, and maybe enable them to submit many queries on free accounts… just spin up a new account whenever you hit the token limit of an unpaid account. So basically it’s a terms of service violation?
I don’t see why it’s necessarily a huge leg up for a competitor… they are just using the outputs of another model as training data. They still need to train their model, which is the expensive and energy intensive part.
It sounds to me like Anthropic just wants the US Government to help enforce its TOS internationally and force Alibaba to pay for those precious tokens? Because apart from that piece, the “attack” just seems like normal use of the service. If Anthropic’s service has an inherent vulnerability, that’s their problem.
Of course all the other comments about how they stole all their training data in the first place are spot on.
No no really it’s about protecting girls, really it is! Now go on over to the doctor’s office and take off your pants.
+1 for Secret of Mana. Awesome game.
I think this probably fits into Doctorow’s concept of “centaurs” in that the use of the AI tool is driven by highly skilled workers who tune the tool and carefully review its output. He contrasts this with the lazy mandates to use LLMs for everything in the corporate world.
I do think this kind of medical research is the best argument for the benefits of machine learning, but it’s also not what’s driving the massive scale of the AI infrastructure build-out and not really generalizable to the idea that organizations can replace workers for every knowledge job