I really don't get those takes. AI is the best thing that ever happened to the Free Software world, it is basically turning any software into Free Software. You can just throw file formats, protocols or even plain binaries at the AI and it'll reverse engineer everything in a pinch. Users can finally modify software themselves, which was always the goal of the Free Software world, but very rarely happened in actuality, since it was just so damn complicated. AI lowered the barrier of entry tremendously, not just in terms of required knowledge, but especially time. Same with Creative Commons, sharing art and stuff, was a nice gesture, but rarely useful, since the level of work to modify a work to fit your project was pretty close to just doing it from scratch anyway. With AI everybody can toy around with image generators and get what they want.
Is a social contract being broken? Yeah, kind of, but the problems that that contract existed to solve are no longer a thing. Creation is now easy and commodified. We finally have computer we can interact with in natural language, something people tried to do for at least 70 years and never made any significant progress on until LLM arrived.
If you want to gatekeep or only create stuff to boost your own ego or portfolio, then AI might be an issue, if you actually want to build stuff, AI is godsend. We are essentially living in the StarTrek future with Holodecks and replicators and people still find reason to complain.
> If you want to gatekeep or only create stuff to boost your own ego or portfolio, then AI might be an issue, if you actually want to build stuff, AI is godsend.
Exactly this. I've been very confused by the free software advocates that seemed to hate AI until I realized their reasons for releasing software under an open source license were very different than what I assumed they were.
Tech has always been breaking social contracts. It’s how we roll. We did the same thing with taxis and travel accommodations to name a few.
We hid our “we know better” hubris under the term “disruption” because the reality (breaking everything) was a little too unsettling for us.
We ignore laws and regulations where we know better of course. Don’t you love having a homey place to stay while travelling that has just a few weird rules, a small to-do list and stays spotless thanks to that cleaning fee?
And look at all the good we did! A whole new world of slaves (oops gig workers - sorry!)
And of course we’re doing it again with information and content. We should be in charge of monetizing all your work because you’re not responsible enough to do it right. It furthers our need for power and control. Oops we meant to help make the world a better place (We keep doing that, sorry.)
Somewhere at the very beginning of my career, the second year I think, I was part of the team that visited a location of the company we were selling our SaaS solution to, in a branch that was back then still dependent on Excel sheets.
The owner of the company lead us to a large open office with about 30 people working in there, mostly women, and whispered to us that he expects he'll only need three of them after subscribing to our product.
That's when I realised the brutality of what we are doing, and the sales guy I was with realised we should be charging significantly more.
As much as it would suck to be one of the 27 unemployed, I don't think the answer is to throw away our calculators and start doing our bookkeeping by hand again. I don't think you need to feel too bad about this. The accountants didn't all disappear when Excel showed up; they just got to work on the more interesting parts of the job, instead of rectifying arithmetic.
The unnecessary hyperbole ruins your argument. Not all of tech is about breaking social contracts. Whose livelihood is destroying the engineer that design better protein folding software? Linus made the world better and richer for everyone. Even the humble startup shipping yet another calendar tool isn’t harming anyone or being the direct cause of layoffs of other workers.
All true, indeed. However, it does feel especially sobering realizing tech is hitting its own ouroboros point: a point where it starts to break social contracts that it designed to begin with.
> We hid our “we know better” hubris under the term “disruption” because the reality (breaking everything) was a little too unsettling for us.
That's not quite right. For a decade, the silicon valley mantra was "Move fast and break things", and it was largely accepted as a good thing. The prevailing wisdom was that this "internet" thing was going to fundamentally change the structure of human society, and nothing would ever be the same again. We ignored the laws, and society broadly accepted that those laws weren't fit for this new world of abundance and information freedom anyway. Big tech was offered broad protection from any responsibility through laws such as Section 230, which basically says they can't be blamed for anything, as long as they structure their services as a "market".
After 20 years, I think we've found that actually the structure of human society has remained largely the same. The things we all need are still the same. The only thing we actually disrupted was the economy, such that we now can't produce and disburse the things we need.
>That's not quite right. For a decade, the silicon valley mantra was "Move fast and break things", and it was largely accepted as a good thing.
By the people not involved in the things breaking maybe. If you didn't drive a taxi or run a hotel the tech companies "disrupting" things by breaking laws didn't affect you directly. Now with LLMs, they do.
I'm a cybersecurity 'expert' and my 40 person team and I make our money by providing GPL software to the world that I wrote.
You have the same incentive you had to share your work as before, and that is to get attention, and customers, assuming that's your game. Open code means no vendor lockin for a lot of customers, so they can pay you but also trust you to not extort them. And if you're hit by a bus your product lives on, and someone else can adopt it.
Supply chain risks and vulnerabilities existed before LLMs came along. They're easier to handle now that we have LLMs helping us. In our org the tsunami of updates we need to do weekly is far more easy to handle with LLMs.
Open code has always been easier to find vulns in vs closed. AI didn't change that.
Yeah the slopfest is real, but easier to deal with thanks to agents/LLMs so it kind of offsets itself.
Your licenses are still enforceable in court. Agreed that not being able to reverse the fact that AI trained on your code and is selling that capability kind of sucks. But humans were doing that before AI.
Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due. But on the flip side, we get AI! Which I frikkin love. I feel like a kid in a candy store. It's training on my code too and it's training on my content. And when people ask about what the best product is for our space, the AI tells them its our product. Woohoo!
Regarding the future: We have some really really big problems that need solving - stuff that creates a massive amount of misery in dark stuffy hospital rooms with crying relatives saying goodbye to their 9 year old child. I've been in those spaces and I'd give up the previous generation of open source ethics in a heartbeat to make just an ounce of that misery stop. The training that my code provides AI is a tiny little part of that solution, and I'm proud of that. Whatever I can do to push our capabilities to the point where we can make the major breakthroughs this species needs, I'm happy to provide.
Hey thanks for WordFence! It made my early days as a programmer a bit less crazy. Having clients constantly installing plugins (backdoors) in their websites was a never ending battle. Don’t really touch Wordpress stuff anymore but it was definitely my favourite plugin back then.
And frankly, I think most of the source of outrage is this:
> Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due.
Turns out, a lot of people didn't really do things in the open to benefit the others, in pay-it-forward style. They just did it for selfish gains. Which is fine, just like keeping source closed and selling licenses is fine. The problem is with lying - doing something for personal gain, while claiming it's for greater good, thus getting more gains through dishonesty.
LLMs just shone a light onto it. People who had betterment of others on their minds, don't have a reason to consider LLMs training on their output as taking anything from them. On the contrary, their outputs now contribute to a general-purpose problem solving tool that will (and already does) help humanity with way more problems that anyone imagined.
I personally am more than happy to know LLMs may have trained on my content. I don't begrudge the companies the $0.000001/year they probably owe me for my relative contributions. I get orders of magnitude more value for myself from LLMs every day, in my personal life alone.
There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it. The OSS subset of those, just paint themselves as being grifters all along.
You have no proof that the world will be a better place because of this stuff but there is plenty of proof already that it will be worse, possibly significantly worse but the jury is still out on that.
That is kind of net loss at this point. Hey, on the bright side we get onslaught of slop, ai psychosis, constant stream of doom trolling.
And ideology of pointlessness where any time you do or learn anything, you get told that why bother you should have used AI.
The kid forever locked in candy store is happy for a bit, but then they get hungry and feel bad. And no matter how much sugar you eat, it wont get better.
If we come together to create a new digital renaissance, they’ll just feed that into the machine as well.
We are not entering a new dark age so much as we are being pushed, by specific actors who make no secret of their actions, and are identifiable and locatable.
>If I make my code available, AI can be used to more easily discover vulnerabilities to abuse my coding errors,
Keeping your source code private doesn't save you. These LLMs are really good at decompilation efforts. I've seen an insane number of clean room ports of games the last few weeks (just saw a Zelda 2 PC port this morning). Even with that, I don't share the gloomy perspective of the author.
"We all have our own set of ethical and moral guidelines, which are also lost once having been absorbed into the colossus. While existing licenses and agreements are imperfect, they have been shown to be enforceable in a court of law allowing me, the creator, power over my own creation."
This touches on the tension between artistic integrity vs. open source and the creative commons. There's been a shift from releasing software to benefit the public domain or "for the lulz" towards treating software more like a piece of artwork you as a creator have exclusive control over, even if it does have an open-source licence.
Sadly or not sadly, the freedom 0 in the four freedoms (freedom for any use) prohibits stuff like "anti-AI" licences, and even if it didn't, the mainstream argument is that you don't need a copyright licence because training upon a work doesn't constitute copying. Of course not everyone agrees with that, but once this is socially acceptable to say, it's also more acceptable to "more directly" copy existing projects, such as by directing LLMs to read the existing code and reimplement it, sometimes overtly, sometimes under the fig leaf of a "cleanroom" implementation.
Fundamentally, AI doesn't change anything if you wanted to do things the previous way - you can still publish your software, others can still use it, you can still forbid AI contributions or AI usage while interacting with your work - but it does surface the hidden implications. Many people do open source for the clout, for their portfolio and for approval from others, and those are things AI threatens, because it devalues software from requiring a huge amount of mechanical engineering skills to create to knowing how to prompt for good results. (see "without respect or credit for those who did the real labour.")
I wouldn't be so pessimistic about the new state of affairs - if creating software is much easier without the existing gatekeepers, a lot more cool things will be created :) People usually create things on their own even when not directly incentivised to do it with the power of law. In a way, this fulfills Stallman's dream - if you can easily modify or create any kind of software then he could have fixed his printer driver.
I really don't see how this would decrease openness, copyright itself decreases openness, and if everyone starts disregarding it then the field of software reverts to belonging to the shared commons.
I don't see why freedom for any use could not be 'freedom for any use except AI ingestion'. Copyright rests on control of your work product, customizing the license is definitely an option.
We’re sharing a world with cultures that do not respect or enforce patents. The internet has helped us stay competitive by making information a lot cheaper to share. AI is the same thing but more potent. It will help us stay competitive even longer.
There were some hidden downsides to the internet and they are worse with AI. Upsides are better, and downsides are worse. Still largely the same deal we already accepted though. “Information wants to be free” never said information wasn’t harmful.
> The past 35 years of openness have created nearly everything we benefit from online, which has transformed the world in a mostly positive and empowering way.
> We are entering a digital dark age...
It's interesting (and a bit disheartening) that it took modern AI for so many technologists to wake up to the fact that technology does not always help society.
Yes, the internet has brought many benefits, but "transformed the world in a mostly positive and empowering way"?
What about all the harms we know are associated with social media, especially among young people, including anxiety and depression, bullying, body image issues, sleep disturbances, social media, dopamine loops, the rapid spread of disinformation to manipulate elections, etc.? What about the effects of porn? Sex trafficking? The hundreds of billions of dollars that are lost every year to internet-based fraud? The loss of personal privacy and wholesale theft and sale of people's personal data? The tech oligopoly that now has its hands in politics? The "better to ask for forgiveness than permission" startups that have pushed legitimate businesses to the brink, turned neighborhoods into clusters of illegal hotels, etc.? The countless people who have been pushed into gig work? And so on.
The internet has been mostly positive and empowering to the people who profited in some way from the above, and many of them are now only upset because the latest technological "advancement" is coming to eat them. For a ton of people, the "digital dark age" started two decades ago.
I like this article. We've read a lot of takes quite similar to this, and I think it's worth continuing to write and share reflections just like this.
> Yet, through much effort and 40 years of debate we had reached an equilibrium. Now AI has thrown that out the window.
The only thing I'd like to add here is that "AI" in this sentence should really be read as "the companies building frontier AI models have thrown this out the window". I don't necessarily think the author should have used a phrase like that, because it gets quite pedantic to read posts where such precise language is used all the time. But when we talk about where to go as an industry and as a society, we need to keep that distinction in mind.
The tech is here to stay, but the tech itself is not what's thrown our norms out the window. It's the people running these companies who are putting their vision and their profits above everyone else in the world. Most of us are exploring this tech and trying to figure out what it's useful for. It's a much harder question to try and figure out what to do about the people using this tech to consolidate wealth and power. We can't hold a technology responsible for anything, but we can hold people responsible for their impact on everyone else.
> I now have every incentive to not share my work, while also being wary of anything I find online that has been shared. If it hasn’t been polluted by slop code, it might have been poisoned with a malicious library, send data off to third parties, or itself be comprised of someone else’s stolen work, implicating me in the crime.
All of this was a concern before.
> If I make my code available, AI can be used to more easily discover vulnerabilities to abuse my coding errors, while if I keep my code closed it is far more difficult to find those same mistakes.
This has always been a tradeoff.
> If I publish my code on a public service like GitHub, I am likely to be inundated with pull requests generated by AI bots and inexperienced users, flooding me with mostly useless slop and taking all of my spare time away just triaging it.
Then disable PRs?
This guy has four public repos on GitHub, two of which are forks. I get the impression he does more philosophizing about writing code than actually writing code.
Does publishing it matter? You would almost have to keep it out of the cloud to avoid it being ingested as policies change. So even sharing it may become higher friction.
Is a social contract being broken? Yeah, kind of, but the problems that that contract existed to solve are no longer a thing. Creation is now easy and commodified. We finally have computer we can interact with in natural language, something people tried to do for at least 70 years and never made any significant progress on until LLM arrived.
If you want to gatekeep or only create stuff to boost your own ego or portfolio, then AI might be an issue, if you actually want to build stuff, AI is godsend. We are essentially living in the StarTrek future with Holodecks and replicators and people still find reason to complain.
Exactly this. I've been very confused by the free software advocates that seemed to hate AI until I realized their reasons for releasing software under an open source license were very different than what I assumed they were.
We hid our “we know better” hubris under the term “disruption” because the reality (breaking everything) was a little too unsettling for us.
We ignore laws and regulations where we know better of course. Don’t you love having a homey place to stay while travelling that has just a few weird rules, a small to-do list and stays spotless thanks to that cleaning fee?
And look at all the good we did! A whole new world of slaves (oops gig workers - sorry!)
And of course we’re doing it again with information and content. We should be in charge of monetizing all your work because you’re not responsible enough to do it right. It furthers our need for power and control. Oops we meant to help make the world a better place (We keep doing that, sorry.)
The owner of the company lead us to a large open office with about 30 people working in there, mostly women, and whispered to us that he expects he'll only need three of them after subscribing to our product.
That's when I realised the brutality of what we are doing, and the sales guy I was with realised we should be charging significantly more.
The unnecessary hyperbole ruins your argument. Not all of tech is about breaking social contracts. Whose livelihood is destroying the engineer that design better protein folding software? Linus made the world better and richer for everyone. Even the humble startup shipping yet another calendar tool isn’t harming anyone or being the direct cause of layoffs of other workers.
Not everybody works for Uber.
That's not quite right. For a decade, the silicon valley mantra was "Move fast and break things", and it was largely accepted as a good thing. The prevailing wisdom was that this "internet" thing was going to fundamentally change the structure of human society, and nothing would ever be the same again. We ignored the laws, and society broadly accepted that those laws weren't fit for this new world of abundance and information freedom anyway. Big tech was offered broad protection from any responsibility through laws such as Section 230, which basically says they can't be blamed for anything, as long as they structure their services as a "market".
After 20 years, I think we've found that actually the structure of human society has remained largely the same. The things we all need are still the same. The only thing we actually disrupted was the economy, such that we now can't produce and disburse the things we need.
By the people not involved in the things breaking maybe. If you didn't drive a taxi or run a hotel the tech companies "disrupting" things by breaking laws didn't affect you directly. Now with LLMs, they do.
You have the same incentive you had to share your work as before, and that is to get attention, and customers, assuming that's your game. Open code means no vendor lockin for a lot of customers, so they can pay you but also trust you to not extort them. And if you're hit by a bus your product lives on, and someone else can adopt it.
Supply chain risks and vulnerabilities existed before LLMs came along. They're easier to handle now that we have LLMs helping us. In our org the tsunami of updates we need to do weekly is far more easy to handle with LLMs.
Open code has always been easier to find vulns in vs closed. AI didn't change that.
Yeah the slopfest is real, but easier to deal with thanks to agents/LLMs so it kind of offsets itself.
Your licenses are still enforceable in court. Agreed that not being able to reverse the fact that AI trained on your code and is selling that capability kind of sucks. But humans were doing that before AI.
Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due. But on the flip side, we get AI! Which I frikkin love. I feel like a kid in a candy store. It's training on my code too and it's training on my content. And when people ask about what the best product is for our space, the AI tells them its our product. Woohoo!
Regarding the future: We have some really really big problems that need solving - stuff that creates a massive amount of misery in dark stuffy hospital rooms with crying relatives saying goodbye to their 9 year old child. I've been in those spaces and I'd give up the previous generation of open source ethics in a heartbeat to make just an ounce of that misery stop. The training that my code provides AI is a tiny little part of that solution, and I'm proud of that. Whatever I can do to push our capabilities to the point where we can make the major breakthroughs this species needs, I'm happy to provide.
And frankly, I think most of the source of outrage is this:
> Yeah on the one hand opening your code got you credit which was nice for the ego and for getting paid, and AI trains on it and doesn't give credit where it's due.
Turns out, a lot of people didn't really do things in the open to benefit the others, in pay-it-forward style. They just did it for selfish gains. Which is fine, just like keeping source closed and selling licenses is fine. The problem is with lying - doing something for personal gain, while claiming it's for greater good, thus getting more gains through dishonesty.
LLMs just shone a light onto it. People who had betterment of others on their minds, don't have a reason to consider LLMs training on their output as taking anything from them. On the contrary, their outputs now contribute to a general-purpose problem solving tool that will (and already does) help humanity with way more problems that anyone imagined.
I personally am more than happy to know LLMs may have trained on my content. I don't begrudge the companies the $0.000001/year they probably owe me for my relative contributions. I get orders of magnitude more value for myself from LLMs every day, in my personal life alone.
There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it. The OSS subset of those, just paint themselves as being grifters all along.
That is kind of net loss at this point. Hey, on the bright side we get onslaught of slop, ai psychosis, constant stream of doom trolling.
And ideology of pointlessness where any time you do or learn anything, you get told that why bother you should have used AI.
The kid forever locked in candy store is happy for a bit, but then they get hungry and feel bad. And no matter how much sugar you eat, it wont get better.
Who cares? Anyone can tell you anything online.
We are not entering a new dark age so much as we are being pushed, by specific actors who make no secret of their actions, and are identifiable and locatable.
Keeping your source code private doesn't save you. These LLMs are really good at decompilation efforts. I've seen an insane number of clean room ports of games the last few weeks (just saw a Zelda 2 PC port this morning). Even with that, I don't share the gloomy perspective of the author.
This touches on the tension between artistic integrity vs. open source and the creative commons. There's been a shift from releasing software to benefit the public domain or "for the lulz" towards treating software more like a piece of artwork you as a creator have exclusive control over, even if it does have an open-source licence.
Sadly or not sadly, the freedom 0 in the four freedoms (freedom for any use) prohibits stuff like "anti-AI" licences, and even if it didn't, the mainstream argument is that you don't need a copyright licence because training upon a work doesn't constitute copying. Of course not everyone agrees with that, but once this is socially acceptable to say, it's also more acceptable to "more directly" copy existing projects, such as by directing LLMs to read the existing code and reimplement it, sometimes overtly, sometimes under the fig leaf of a "cleanroom" implementation.
Fundamentally, AI doesn't change anything if you wanted to do things the previous way - you can still publish your software, others can still use it, you can still forbid AI contributions or AI usage while interacting with your work - but it does surface the hidden implications. Many people do open source for the clout, for their portfolio and for approval from others, and those are things AI threatens, because it devalues software from requiring a huge amount of mechanical engineering skills to create to knowing how to prompt for good results. (see "without respect or credit for those who did the real labour.")
I wouldn't be so pessimistic about the new state of affairs - if creating software is much easier without the existing gatekeepers, a lot more cool things will be created :) People usually create things on their own even when not directly incentivised to do it with the power of law. In a way, this fulfills Stallman's dream - if you can easily modify or create any kind of software then he could have fixed his printer driver.
I really don't see how this would decrease openness, copyright itself decreases openness, and if everyone starts disregarding it then the field of software reverts to belonging to the shared commons.
There were some hidden downsides to the internet and they are worse with AI. Upsides are better, and downsides are worse. Still largely the same deal we already accepted though. “Information wants to be free” never said information wasn’t harmful.
> We are entering a digital dark age...
It's interesting (and a bit disheartening) that it took modern AI for so many technologists to wake up to the fact that technology does not always help society.
Yes, the internet has brought many benefits, but "transformed the world in a mostly positive and empowering way"?
What about all the harms we know are associated with social media, especially among young people, including anxiety and depression, bullying, body image issues, sleep disturbances, social media, dopamine loops, the rapid spread of disinformation to manipulate elections, etc.? What about the effects of porn? Sex trafficking? The hundreds of billions of dollars that are lost every year to internet-based fraud? The loss of personal privacy and wholesale theft and sale of people's personal data? The tech oligopoly that now has its hands in politics? The "better to ask for forgiveness than permission" startups that have pushed legitimate businesses to the brink, turned neighborhoods into clusters of illegal hotels, etc.? The countless people who have been pushed into gig work? And so on.
The internet has been mostly positive and empowering to the people who profited in some way from the above, and many of them are now only upset because the latest technological "advancement" is coming to eat them. For a ton of people, the "digital dark age" started two decades ago.
> Yet, through much effort and 40 years of debate we had reached an equilibrium. Now AI has thrown that out the window.
The only thing I'd like to add here is that "AI" in this sentence should really be read as "the companies building frontier AI models have thrown this out the window". I don't necessarily think the author should have used a phrase like that, because it gets quite pedantic to read posts where such precise language is used all the time. But when we talk about where to go as an industry and as a society, we need to keep that distinction in mind.
The tech is here to stay, but the tech itself is not what's thrown our norms out the window. It's the people running these companies who are putting their vision and their profits above everyone else in the world. Most of us are exploring this tech and trying to figure out what it's useful for. It's a much harder question to try and figure out what to do about the people using this tech to consolidate wealth and power. We can't hold a technology responsible for anything, but we can hold people responsible for their impact on everyone else.
> I now have every incentive to not share my work, while also being wary of anything I find online that has been shared. If it hasn’t been polluted by slop code, it might have been poisoned with a malicious library, send data off to third parties, or itself be comprised of someone else’s stolen work, implicating me in the crime.
All of this was a concern before.
> If I make my code available, AI can be used to more easily discover vulnerabilities to abuse my coding errors, while if I keep my code closed it is far more difficult to find those same mistakes.
This has always been a tradeoff.
> If I publish my code on a public service like GitHub, I am likely to be inundated with pull requests generated by AI bots and inexperienced users, flooding me with mostly useless slop and taking all of my spare time away just triaging it.
Then disable PRs?
This guy has four public repos on GitHub, two of which are forks. I get the impression he does more philosophizing about writing code than actually writing code.