The climate constraints are currently not compatible with the ongoing arm race on large neural networks models. The training seems kinda OK, but the inferences... and it's currently just rolled out as shiny gadgets. This really need to be rethought.

Now, this is interesting research. With all that complexity, emergence is bound to happen. There's a chance to explain how and why. The links with the training data quality and the prompts themselves are interesting. It also explains a lot of the uncertainty.

The lack of transparency is staggering... this is purely about hype and at that point they're not making any effort to push science forward anymore.

Well, people asking relevant questions slow you down obviously... since the goal about the latest set of generative models is to "move them into customers hands at a very high speed" this creates tension. Instead of slowing down they seem hell bent at throwing ethics out of the window.

This is an excellent piece. Very nice portrait of Emily M. Bender a really gifted computational linguist and really bad ass if you ask me. She's out there asking all the difficult questions about the current moment regarding large language models and so far the answers are (I find) disappointing. We collectively seem to be way too fascinated by the shiny new toy and the business opportunities to pay really attention to the impact on the social fabric of all of this.

When they changed their statutes it was the first sign... now it's clear all ethics went through the window. It's about fueling the hype to drive money home.

That's a good set of questions to ask ourselves when in contact with a product claiming the use of "AI".

Interesting and surprising limitation. This makes a lot of sense when you think about the set of images used for training though. Also says something about our own art history.

Or why they are definitely not a magic tool for programming. Far from it. This might help developers a tiny bit, at the expense of killing the learning of students falling for it and the creation of a massive amount of low quality content.

Excellent piece as usual from Cory Doctorow. It quite clearly point out why Google is anxious and running off the chatbot cliff

Interesting strategy, shows a fascinating blind spot in the typical AIs used for Go nowadays. It kind of hints to the fact that the neural networks abstract knowledge much less than advertised.

Inaccuracies, contradicting itself, conflating events, misquoting sources... all of that mixed with some correct facts, it's a perfect misinformation spreading machine. This just can't be trusted at this point. Those experiments should be stopped in my opinion, better do proper homeworks first, then relaunch when this can be better trusted.

Are we surprised? Not really no... you don't own any of the data you're feeding it. Keep it away from your secrets.

So transformer models produce things that look plausible... and that's it. What would it look like if we started to make hybrid models in which a transformer model is also tied to proper computation tools with general knowledge? This piece is a good illustration of what it could provide.

For all the conversations about how chat GPT might displace jobs, there's a big untold: how much of copyright is violated in the process? It's also very concerning about how much data it collects when interacted with.

Or why you can't trust large language model for any fact or knowledge related tasks...

It's limits and biases are well documented. But, what about the ideologies of the people behind those models? What can it tell us about their aims behind those models? Questions worth exploring in my opinion.

Interesting work, trying to get back to the source material used by a generative model. This is definitely necessary as well.

A few interesting points in there. Too much hype and important points are glanced over, we'd all benefit from them being more actively explored.

Very nice summary of the architecture in the latest trend of transformer models. Long but comprehensive, a good way to start diving in the topic.