Interesting essay even though I don't like the whole part about singularity. I found this section was a distraction to the more interesting arguments. Indeed, I liked a lot the concept of decoherence and its ties to complexity.
The essay also makes a good job of showing how it's something we have to deal with and likely the worst kind of technical debt. I also shows how it impact productivity obviously. It's all valuable and things I encounter on a regular basis.
And of course, since everything has to talk about LLM these days, it build an interesting case on how and why using those tools especially with harnesses to pump out more code is actually increasing decoherence. Exactly what we don't want.
This is a rant yes but well deserved. Those companies selling giant LLMs are just DDoSing everyone...
It makes sense to me, it's better to not fall prey to the marketing induced FOMO. Instead projects have a chance to do the right thing.
Kind of an unusual read. But I think the message is powerful. We should definitely favor community over tools controlled by the new robber barons.
The slippery slop seems clear... If you start generating a bit you'll be pushed to loose full control in the end. Do you think you can learn this way? Improve? I doubt it.
Since I've got my PhD in AI almost twenty years ago now, this piece strikes hard. This fellow kept doing research while I didn't (the reasons to be told another time but it's linked to this article). Clearly AI researchers who started around that time or before have a very different vision to what "AI" means as a field (although I've a few difference I'm fairly aligned with this article)... Clearly we've been robbed of the term by the industrial complex.
The hype is very high right now indeed. This can lead only to misguided decisions. But really it's mostly distraction from the massive ethical concerns this industry carries. It would require for policymakers to step up but I'm afraid it unfortunately won't happen or they'll come up with misguided attempts if they don't first consult with field experts.
Interesting initiative. I wish it'll be popular and many people will follow it. Maybe it'll cure some of them of their AI psychosis.
Answering a letter with important questions. LLMs can indeed be disruptive to learning, to be handled with care.
Clearly there's a political project behind the design of those tools. Clearly our industry is lacking enough class consciousness to push back against the "AI overlords" agenda.
Clearly the situation of copyright around LLM uses is complex. It's easy to fall into code which can't be copyrighted if there's not enough human input to produce the work (and keep in mind that "prompting" is likely not enough input). This also raises questions about potential copyright violations. Looks like following the guidelines of the SFC for contributing to Free Software projects is getting especially important.
The executive summary is good. This is also a nice source of reference papers and studies on the topic. It shows quite well why the work is moving to the harnesses rather than the models. Indeed models probably won't improve much in terms of reliability. It all points once again towards a needed shift of people attention from the shiny toys and towards team practice instead.
Very good opinion piece going back to Naur's and Brook's important papers about software production. It's a sobering take on how LLM can't bring the promised productivity gains. This is not saying it can't bring productivity gains of course but it'll never be as dramatic as what the current hype cycle pushes people to believe.
Interesting policy. I think it strikes a good balance.
Funny, I was using that exact same term recently: "brute-force driven development". It describes fairly well how the harnesses work.
In other words: the security boundaries of coding agents are very porous and not where you expect. Handle with extreme care until they have a proper security model.
The hardware prices are nuts right now... and there's no end in sight yet.
An illustration of the political and ethical acumen being low in our profession in general and in some (most?) Open Source projects in particular...
There's clearly an issue with the security and privacy practice of those companies...
Really this kind of AI push is a bad move for employers, especially when interacting with FOSS communities so much. This forces people to pass the ethical issues onto volunteers...