Would be a terrible shame if lots of people opted out.
If you are a EU citizen you might also want to write a complaint to privacy@huggingface.co because they are collecting your personally identifiable information in machine-readable form which they are distributing to third parties.
From the linked webpage readme:
2.1 Classify. Each file is labelled permissive (at least one permissive license detected, no conflicting non-permissive license), no_license (no licenses detected, or only non-license legal texts such as CLAs), or non_permissive. The permissive allowlist follows the Blue Oak Council list plus licenses categorized as Permissive or Public Domain by ScanCode. Files classified as non_permissive are excluded from both released datasets.
From https://www.bigcode-project.org/docs/about/the-stack/:
v1.1: The three copyleft licenses (MPL/EPL/LGPL) were excluded and the list of permissive licenses extended to 193 licenses in total. The list of programming languages was increased from 30 to 358 languages. Also opt-out request submitted by 15.11.2022 were excluded from this ersion of the dataset. The resulting near-deduplicated dataset is 6TB in size.
So MPL/EPL/LGPL are already not part of the dataset.
So… why were they in there? Was this added for v1.1?
“one permissive license” - So if my project includes a lib and I include the license file for that for the license notice…?
Scraping without consent steals developer personalIP.
Good thing I never finished a project!
I never really needed any justification, but ever since AI companies just take stuff illegally, and that openly, it has become my justification to just pirate the shit out of everything.
(Excluding indie games of developers I like).
This specifically isn’t illegal though. They’ve only scraped public repos.
That depends on the license. Music is also available on Youtube without authentication requirements but I don’t think you can just download those and do whatever you want with them.
Find me a repo on GitHub with a license that disallows cloning it. I’ll wait.
Any repo without a license disallows reproducing it in any way, which includes cloning it for business purposes.
The only exception is that public repositories can be forked on GitHub as new GitHub repositories. Forking doesn’t give you a license to use or reproduce the code, though.
Could be wrong, but the act of downloading music isn’t what gets people in trouble, it’s the uploading. When people torrent both happen, but afaik it’s only the uploading portion that is actually bad.
When people torrent only a part of the file is being shared by all peers. Technically only the original seeder shared the whole thing and after a few seeders it’s a tiny portion of the file.
This depends, but not because of piracy. If you have to bypass DRM to download the music, you may be violating DMCA §1201 (even if you are otherwise allowed to use the music).
Edit: This obviously doesn’t apply to GitHub, but might to YouTube.
Downloading is illegal.
Have you heard of a thing called copyright?
Wait, are we pro-copyright now?
Never
It should have been anyways. Like what the fuck were you all doing on the internet. Trying to sell your stupid art. Helping corporations steal data and build pay walls so you can earn $500/month selling your kitchy Garfield key chains. The internet should have always been about data replication and sharing. It’s too late to put that cat back in the bag. It was suppose to be stopped 20 years ago. Now we have all this stuff ruining the world and bringing back nazis. But at least the lefty capitalist artist sold some key chains
I don’t see how people selling keychains (physical items) helped corporations steal data.
Keychain just a stand in for anything. More like how someone making videos of making key chains used patent trolls to build tools to scan all the videos they can in order to find users they can take down. Just one example. The little lefty entrepreneur was used to beat the old school leftists into the dirty on behalf of the corporate interests. Now everything is owned by corporate and we lost the internet to racist nazis. There’s a moment 20 years ago things could have been different that we’ll never get back. Frontier was new and we needed to rally around the wagon to defend the frontier. Instead we sold out.
And now all the lefties are in the smallest corner of the internet and the rest of the internet and social media is owned by the capitalist and nazis. It all started by forgetting the roots and in favor of selling our key chains and trying to build a market rather than a collaborative space.
What’s with the obsession with “lefties” vs nazis?? Sounds like somebody stayed on the propaganda carousel a bit too long.
You choose what parts of the internet you want to use, so who cares if 99% is profit oriented garbage, honestly.
People choose to publicly share their stuff for free, so of course it’s gonna be taken advantage of. That’s literally capitalism.
Yea it’s so crazy. Lol
Everything is fine. Nazis are marching in streets again primarily driven by online recruitment. Billionaires are investing in removing leftist from digital spaces to facilitate this. But because I care, I’m obsessed.
I’ll just choose to hang out here with the people who say they oppose this stuff but really, they’re just what, posting about beans and moths while nazis take over the government, March in the street and feed children their content through multiple pipelines that are growing because there’s fuck all resistance. After all the “resistance” is busy trying to decide which hole they want to hide in.
But at least the lefty capitalist artist sold some key chains
Selling your shit and getting paid doesn’t make you into a capitalist. At most you’re an artisan, if you own your tools and you get paid for your final product rather than your labour hours.
To be a capitalist you necessarily need to own capital. In the example you picked you’d need to own a factory where you underpay people to and keep the surplus from all the keychains they produce.
And regardless of what you meant, you’re absolutely barking at the wrong tree here, market consolidation and monopolies predate the internet, and they would have always had the final say in how we shape our infrastructure and share shit online under capitalism.
You would be a supporter of capitalist. The capitalist needed to take the internet away from the left. It was hostile to making capital. Early days people were much more aware of defending it against capitalist which like you said, people were tracking how capitalist ruined all other frontiers and left them dried out husks that stifled originality, creativity and actual innovation. Look at the leftovers from the early days. Websites from them were focused on free use and sharing information freely, gnu, Winamp, VLC, sites like reddit, digg, Wikipedia wouldn’t exist with today’s internet if they were created today. Capitalist took over.
I fought hard for net neutrality, a lot of us did. We don’t matter.
Which goes to my point. You fought wrong. What i see is that the left is very likely manipulated by right wing interest groups way more than people want to admit. I think for decades now, they have targeted leftist online for issues we traditionally dominated and redirected people into areas that were very ineffective. Meanwhile they socially engineered the right side and MAGA to actually effective social engagement. After decades the left is just a dead in the water idea with people who no longer know how to use the internet correctly for social engineering and engagement. The only time the left rallies is when the effort is the most high effort, high energy, low reward movement people could possibly imagine.
The left in the past were intelligent intellectual building and innovating things. That is no longer the case. We’re resting on the laurels of the best and looking around at lemmy should make it very obvious how true this is.
The sad reality i think people need to understand its the MAGA isn’t winning just because of money. They’re wining because they’re putting effort in areas that the left don’t even realize are important. They are playing chess and the left aren’t even playing because competition makes the left feel anxious.
I have multiple people in my life on the right who spend all their time online posting and networking and planning. They will shit post all day. Between the 3 of them, they probably create tons of content. Let alone sharing and promoting others content.
People on the left I know will read an article. Can’t be bothered to share or even upvote. Maybe they’ll write a comment like “This sucks, Trump is stupid” and then they go look for star trek memes all day. Give this behavior 20 years for the kids today to grow up. The global cultural image of the left right now is a weak, gutless screaming loser who nobody wants to admit to being outside of small circles. Kids today hate the left. It’s going to get so much worse.
I hope you’re wrong, but not much hope :(
Me too.
Me too.
My point here isn’t to shit on the left just to shit on them. It’s to at least get discussion if not find others who see it too. I think we’re asleep. Maybe hoping if we just play the ethical long game that maybe it’ll all return. In my mind we’re standing on the beach watching the tide recede after an earthquake. Like most thing stuff is weird, but they’re not really tracking that they need to start acting now if it isn’t already too late.
My hope is people get a fire under there ass and first admit that we have fucked up and it’s time to start firing up our own grass roots efforts. Download gimp. Get back on social media since we were never going to make a dent in their profits. Start using all these tools even AI to resist and present a front against these modern nazis. Play fucking dirty if we have to. People bitch about money but that money is only doing what we do for fun. Instead of obsessing about beans and moths, let’s get back to being political and taking ground.
Lemmy could be a staging ground with the way it is built. Because when a site is a threat, they’ll use every capitalist lever they can to squash it. They’ll bring in moderation teams. They’ll bring in ads. They’ll bring in the tools we didn’t resist. And if they do, we stand up a new instance without that shit. But we gotta do something to fire us up or it is only going to get worse.
Some of these are good points, and I’ve got a tool to recommend: Fedica.
I use it to schedule my posts in advance and post on social media without actually having to be on it all the time.
A little bit infuriating since huggingface itself requires login to access a large portion of the content on their site
When I hear this company’s name I can’t get the idea out of my head that it’s related to the face huggers from Alien.
Much like… -reads notes- Microsoft is via Github… huh…
Gothub.com is a thing now for OpenBSD’s Game of Trees project.
It’s gothub.org. You linked to some dumb AI startup.
It’s a sure sign of a bubble that a URL typo coincidentally brings one to an in-beta AI startup with a dumb idea.
if I opt out, will the Roko’s basilisk come after me?
Reported for cognitohazard. Delete this immediately.
(I kid, but someone really did report your comment.)
lmao
I think its hilarious that some people seem to actually take this science fiction variation on Pascal’s wager seriously.
As long as you haven’t been told what you’re opting out of, you’re good.
“Oh no, people are using information I put publicly available on the internet for everyone to see!”
Morons. The lot of you.
Sure. Let’s see if they use it for endeavours in the same spirit.
Or are you fine with they using this data for-profit without benefitting the public by also making it open?
(not talking about hf, I know starcoder. just in general)
Maybe you should’ve released your code under a license with that stipulation.
In all honesty, you’re just coming up with reasons to fit in with the crowd.
Give me a crystal ball next time and I’ll do it genius, use your common sense.
Seems fitting the one thinking about “the crowd” the most is exactly the one trying to be the most distant from it. Projecting much are we?
Don’t hurt yourself with those mental gymnastics.
Hey, sure. I’m doing all kinds of gymnastics to think the ones profiting off our public data are in the wrong, I’m just in it to fit in and be in the crowd sure! Whatever makes you feel different. Hope your mirror works someday.
See? Even now you’re still trying to save face.
You need help.
Oh God why are all your replies in the same structure, am I talking to an LLM? I can’t decide whether that would be preferrable or not
I did it to invite collaboration and connect with other developers with similar interests. FOSS is more about building communities than building software, after all.
I did not anticipate that it could be (legally) used to dismantle the kinds of communities I wanted to build. (I did anticipate that it could be illegally used to that end, but historically that has tended to cause a Streisand Effect, so that risk seemed worth it.)
If people still want to be part of the community, they can.
You’re just coming up with reasons to fit in with the crowd.
Love you too, kitten. ♥️
Am I alone in not wanting to put my username into that field? If they don’t have it will they then just decide that it’s now a good time to scrape it? Or are they going to record that it was searched?
They’re not able to scape private repos, surely?
No, it seems to only be a subset of public repo’s.
I have like 65 repo’s and only 13 were scraped. I don’t get why they specifically scraped those though. They don’t have the most stars, they aren’t the oldest or newest, not the ones with the most forks, nor do I see a pattern based on programming language.Microsoft has access, and everything is for sale. So, maybe? Probably? 🤷
Embrace extend extinguish
Since they stole my paper on ethics in computer science, maybe the model will learn to act better than its owners
Is there a form to ask to be included in the next stack? they seem to have missed me this time
Pretty sure all my repos are MIT licensed for the betterment of everyone, but I’m not on the list! So I guess I’m not good enough, or they are failing to follow the attribution clause of it.
My dotfile repo is there and it doesn’t have a licence. Meaning it’s technically not open source. Didn’t stop them
Strange because all of my MIT repos are listed - all the GPL ones are excluded.
My list includes GPL licensed repo’s
Mine are all GPLv3 or forks of other repos and they are listed. I’m sanguine because the license allows for study and if it’s good enough for humans I don’t see why it’s not for clankers.
For me, and a few other projects I checked, it only has non-GPL repos. But it also does not have everything that isn’t GPL, despite the repos being much older than the cut-off date. But it does have repos without a license, which they are simply not allowed to copy.
I wonder if those repos have been deduplicated, and one of the forks (on some other person’s account) is included instead. Unfortunately you can only search the first 5M records via the website, and I don’t have time to play around with the API at the moment, so I could neither confirm nor deny that possibility
The best way to opt out is by not using GitHub. Also opts you out of Copilot and a bunch of other stuff.
note that former users would have needed to remove their GitHub data before August 2025 to not be in this dataset
When I back something up, I save a copy and add “backup” to the name… Because I’m advanced.
If I’m feeling really good and healthy, I’ll even put it on a usb stick.
Other git hosts are also getting scraped, and have had to implement counters because of it. For example, this is the kind of thing Codeberg shows crawlers. I’ve even seen people who self-host complaining about getting overloaded because of bots scraping their forge
I’ve put Anubis before most of my website, including my forgejo instance. For the projects hosted there, which is not all, I can only hope that that’s enough.
I like to have the visibility and CI of GitHub. But this sucks ass.
Already did that some time ago but I am still in the dataset.
