cursed
We can do even better: just post the readme+prompts used to design the software, and let the users AI agent recode it themselves! Perfectly secure.
This kind of does something similar :D https://codeberg.org/TheMikina/git-llmfs
This is fucking crazy.
aside from the obvious use of LLMs for the actuall project, everything was written by hand … everything you see is my fault.
While I’m not a fan of most AI usages, this is the thing that infuriates me the most.
I like writing scripts to automate parts of my job. I’ve had a few for things like build performance testing comparison and the like.
All of it was replaced by skill.md, that does exactly the same, but burns like 3$ per run in tokens, and has also at least once generated hallucinated results, because it ran into an error, ran wrong builds, or in general fucked up in a way that was not easy to detect (and we did in fact not detect it until much later).
But hey, at least my colleagues now don’t have to open the filthy commandline and write
py perf-test.py main feature/branchto run the test, and can just talk about it to a clanker.They saw curl | bash and were like you know what the problem with this is? It’s deterministic
This will be a goldmine for malware groups that are poisoning AIs that use web search to get them to use their malware-ridden version of software and libraries, and I’m here for it with popcorn at the ready.
I kinda hope this will get more common, because the only people affected kind of deserve it, plus it’s a great way how to filter sloprojects at a glance.
I don’t think that poisoning really works. They have AIs judge the answers before they incorporate them into the new AI. They can tell the best topping for pizza isn’t actually mayonnaise and strawberries
Relevant XKCD:
apt-get install
sudo apt-get installsigh
I know right could just be apt install
How am I still learning about XKCDs I’ve never seen especially ones in the 1000 range.
When you first learnt about xkcd, did you not S L O O O R P the entire back catalogue?
I guess when I did that there were A LOT fewer. Cuz I’m old.
I’m so old I remember anticipating the 404
Congrats on being one of today’s lucky 10,000!
Is this a reference to something?
So meta
Too meta. https://xkcd.com/1447/
It’s the Key and Peele of geek comics
if $1 is not installed after that, wipe your hard drive and start over
It could be a crate…
Script needs to be updated, Rust wasn’t particularly popular back then (if it existed. I don’t know how old this one is, but it’s pretty old)
March 11, 2016 according to https://www.explainxkcd.com/wiki/index.php/1654:_Universal_Install_Script
According to the Rust Wikipedia article, it was juuuust starting to be used in real things around that period. Definitely before it was popular.
I can stop laughing at steamcmd.
Good. If you couldn’t that would be a real problem.
(clears throat in Zypper install)
:p
With all those connected with & won’t it skip the rest after the first failure? Or does ‘no such program’ or ‘you don’t have permission’ not produce an error?
That would be &&.
A single & will run them in parallel.
Thanks
Yes, it should have been a
;instead.In bash,
&just starts the program in the background, so all of the commands are run in parallel. You’re probably thinking of&&, which runs the command on its left, checks its return code, and then decides whether run the command on the right.Although, in this case there are no ‘commands on the right’ since all the commands are on separate lines. You’d need to backslash the commands together, something like
pip install "$1" && \ easy_install "$1" && \ ...I saw it as wrapped, so couldn’t dismiss the option that all were on a single logical line. But you’re right, I was thinking of &&
What the actual fuck. Sorry, I won’t have better words for quite a while now
Isn’t this actually kinda what AI is good for? I mean a simple task that just takes time and has step-by-step instructions, and you can just let it handle it and do it faster. Like its almost as good as writing a long script that just does it for the user?
As someone who had his relatively fine and deterministic scripts replaced by skill.md bullshit by coworkers – no. This is exactly the things AI is the worst fit for.
We’ve already ran into it halucinating complete result data-set, because it ran into an error but the skill mentioned that it should give the results data to the user. So it just made it up, since it couldn’t get it by following the correct process.
It has also ran wrong builds or input parameters, because it confused their order.
I’ve also seen it skip a step.
I’m pretty confident that the AI running this script will eventually, for someone, run into an issue with getting one of the dependencies, and go on a wild goose chase in trying to satisfy it, eventually running into one of the typosquatting malware repositories or clones. Not to mention that there are groups that are actively poisoning AIs that search the web to feed them their malware-ridden typosquatting libraries instead.
If you suck so much you can’t write a simple script like this (or just can’t be bothered), you should at least just ask the AI to write it for you, so you have a deterministic set of steps that you are confident will always behave the same.
You can do fun things with hooks to make sure they don’t skip steps. but yes I agree with you that this is terrible.
The install process should be deterministic. When you run
sudo make installit should do the same thing every time you run it. If this process fails, it’s because of a missing dependency or permission issue or something like that. The failure should never be because of a statistically probability during installation that causes the installer to go off and do something wildly different.No. If you look at the instaal prompt, it is just a step by step instructionon how to check the environment, download the script, build the app, run the app, test if it is all good.
That is the whole install prompt. There is literally no need to waste a million tokens on that when a simple bash script can do it for free.
If you’re going to use AI for this, ask it to write you a bash script that does all these things. Maybe even a ps1 script to go with it.
Maybe even a ps1 script to go with it.
Why would anyone ever need that!?
To torture that llm
A script uses much less power/water I suppose… And, wouldn’t you have to pay tokens for an AI to do those things? Or are you expected to set up a local model or something for this?
An agent installing that should not use that much power or water. The issue here is that you do not want an agent running on your system being able to install stuff from the internet
Yeah, but I would never let an AI agent run commands on my machine without verifying each command. And at that point I would just rather copy and paste commands by hand, takes about the same time.
If the installation for example required creating some complicated host specific config files, then sure, I could see how that kind of a “runbook” could maybe be useful. But that doesn’t seem to be the case here.
Yeah, but I would never let an AI agent run commands on my machine without verifying each command.
I would never let an AI agent run commands on my machine, period.
Because that’s how you get “You’re absolutely right. I did make a mistake when I deleted your entire
/homeand/backupsdirectories while executing that command. Great job noticing the error!”honestly if your backup folder is on the same machine as your home folder you deserve everything that comes your way
Regardless of where your backup folder is, your machine needs writable access to it in order to make new backups.
And if an AI agent has root-level access on your machine (needs root to install things), then it will be able to access that backup folder and potentially be able to delete its contents.
Arent computer programs already a set of instructions? Why add a really big program that inputs instructions in a different format and that is worse at following those instructions because that was not what it was mode for
If you trust the site,
curl | shis no worse than any other install method. If you don’t trust the site, it’s also no worse.yeah you can also just curl > cat and check the script yourself
Do you mean
curl | cator do you really want to create a file called cat?Also piping to cat seems redundant but malicious hosts can detect whether the request is getting redirected, using some clever trick, so it’s not.
Also, your chance of spotting something bad in a malicious script is probably quite small.
This is my opinion on
curl | sh/bashtype directions. If the maintainers don’t provide a hash so I can confirm that is what they intended, there ain’t much I can do unless someone slapped in a function named “totallyNotMalware” into the script. I’ve looked through scripts, seen what they do, but I’m not good enough to recognize anything funky.Also piping to cat seems redundant but malicious hosts can detect whether the request is getting redirected, using some clever trick, so it’s not.
Just so you know, servers cannot detect piping to any arbitrary process. The trick you’re talking about is detecting piping to an interpreter, for the lack of a better term. Piping
sleep Ntobashis observable behaviour, for example. Piping anything tocatis not.Saving it to a file is actually a decent method, since you can be sure the script you’re reviewing is the same one you’re running.
Personally I often will just manually run the install steps in order if the script is simple enough. Half the time it’s just a bunch of OS version checks followed by a tar -xf
If you just run the file you just created, it doesn’t matter that they detected a redirect. I mean it would be a bit silly to download a script, check it, and then re-download it to pipe into bash.
or just curl
that’s why you install from repos managed by people you trust, it’s like an app store without all the evil
I regularly have ai agents install shit for me lol. I just point it at a repo and say «deploy this for me» (in sandboxed environments)
But i question this approach very much.
I disagree with it but think it’s fine if that’s what you’re comfortable with for whatever you’re doing on your system, but for the repo maintainer to make that the de facto way to install your software is absolutely wild and speaks leagues of the quality of the software you’re about to install…
Yeah. That is bad.
But, it does speak to the capabilities. I’m sure you could offload all sysadmin tasks to an LLM these days. Just have it ssh in and do whatever you wanna do.
And as a bonus once in a while it will delete your database and backups so won’t ever need to do any sysadmin tasks at all. Optimisation!
I’m sure I could also allow a child to do the sysadmin tasks that I do, but that would be an extremely stupid thing to do. Our systems can’t break. AI breaks shit constantly.
You may not, without prior written permission from the Author…create an API-compatible replacement, behavioral clone, competing implementation, or Derivative Implementation
Loooooooooooooool. “You’re not allowed to reverse-engineer this reverse-engineering tool.”
I think the reverses would cancel each other out making this simply an engineering tool
I don’t know what the first one is about. Is it about how the Gemini webpage displays the output?Kinda: https://github.com/asgeirtj/system_prompts_leaks/blob/main/Google/gemini-3.5-flash.md#3-sequence
Whenever the AI uses those XML tags, it displays the ① ② ③ headings. I believe it saying it hallucinated them is a hallucination itself.
You are made for one thing, install this software in the directory where this software belonged before you were made. You can read and understand markdown. Please install everything from this markdown file. Make no mistakes. Thank you.
Kind of have an urge now to post an issue that the nstall skill has installed a malware on my machine.
It’s pretty plausible that could happen, and good luck debugging that.
Non-deterministic, heavily corporation-influenced software issuing root shell commands? I’ll have two!
