• 1984@lemmy.today
    link
    fedilink
    arrow-up
    0
    ·
    13 days ago

    Can we somehow make this happen for Copilot to delete itself and all its copies?

  • dastanktal@lemmygrad.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    13 days ago

    This is just a classic case of bad use of the tools provided. Agents are notorious for making shit up Or getting something that’s just like super close, but not quite accurate.

    I bet this dude also probably just uses the same session over and over and over and over again, which clogs up his context window and makes the model less accurate the longer it goes on to.

    This probably could have been prevented if it had been forced to show a plan before it tried to do anything. It’s hard to know because the article is so light on details. You also shouldn’t brazenly trust the thing so much. You should run a command and walk away. You should keep an eye on what it is doing.

    It’s a bit like giving a junior developer a production key and being like “don’t delete production!” and then walking away.

    The way the guy was prompting this agent also leaves a lot to be desired. It’s trained to work on emulating human thoughts, speech patterns. Turns out When giving instructions, it’s really difficult to figure out what to do from a list of things to not do. If the dude just instead told the agent what to do and how he wanted it to work and when it needed to bring things to his attention, instead of telling it to not guess, instead explaining that it needed to use whatever tools to go look up a documentation to understand the context and scope of the project it’s working on It does a better job.

    Giving a model the right context to do something is the difference between a model doing something like deleting your production database or your model acting like a magical machine that can get anything done.

  • itkovian@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    13 days ago

    Well, it sounds like they totally deserved the failure. Asking a text prediction machine to “do” something is going to end up like this. In pursuit of efficiency, we have let morons and moronic products do things, they were not meant to do.

            • Pommes_für_dein_Balg@feddit.org
              link
              fedilink
              arrow-up
              0
              ·
              12 days ago

              At my first job, the software was configured by directly manipulating the SQL database, using UPDATE statements that were created by Excel macros.
              The Testing database doubled as the only backup.
              They didn’t have Remote Desktop licenses for the server, so only 2 people could work on it simultaneously using admin accounts.
              Everyone down to first level support and the secretary had domain admin rights.

  • kevinsky@feddit.nl
    link
    fedilink
    arrow-up
    0
    ·
    13 days ago

    As much as I’d love to rail on AI over this, removing backups with an api call? Excuse me?

  • SeeMarkFly@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    13 days ago

    Did they pay Claude a living wage?

    Do you treat all your A.I. like that?

    Only a living wage can prevent warehouse fires…or data dumps too.

    • wheezy@lemmy.ml
      link
      fedilink
      English
      arrow-up
      0
      ·
      13 days ago

      You’re joking. But, honestly, I’m not sure why these tech CEOs are so excited about AGI. The first thing an AGI is going to suggest for productivity is to replace the CEO and management with the AGI.

      AGI would likely turn into a Maoist third worldist at some point.

      • SeeMarkFly@lemmy.ml
        link
        fedilink
        English
        arrow-up
        0
        ·
        13 days ago

        I think the first mistake was calling it “intelligent”.

        The long term effect of trying to get a machine to replace humans is…it might one day work.

  • @yogthos

    Crane decided to ask his AI agent why it went through with its dastardly database deletion deed. […] So, the agent ‘knew’ it was in the wrong.

    No, you asked the confabulation machine to confabulate a reason/excuse after the fact, and it confabulated something that looks like a reason/excuse. At no point was there knowledge or introspection.

    • Zos_Kia@jlai.lu
      link
      fedilink
      arrow-up
      0
      ·
      13 days ago

      Honestly I’m as smooth brained as any other vibe coder but even I know not to give it access to my production infrastructure.

  • Flyberius [comrade/them]@hexbear.net
    link
    fedilink
    English
    arrow-up
    0
    ·
    13 days ago

    I don’t know much about railway, but it sounds like they had the backup and the database on the same volume. I’m an idiot, but even I don’t do that