More signs of the generative AI companies hitting a plateau...

It shouldn't be, but it is a big deal. Having such training corpus openly available is one of the big missing pieces to build models.

This is an interesting and balanced view. Also nice to see that local inference is really getting closer. This is mostly a UI problem now.

This is what you get by making bots spewing text based on statistics without a proper knowledge base behind it.

More marketing announcement than real research paper. Still it's nice to see smaller models being optimized to run on mobile devices. This will get interesting when it's all local first and coupled to symbolic approaches.

This is still an important step with LLM. It's not because the models are huge that tokenizers disappeared or that you don't need to clean up your data.

Using the right metaphors will definitely help with the conversation in our industry around AI. This proposal is an interesting one.

More signs of the current bubble being about to burst?

Now this is an interesting paper. Neurosymbolic approaches are starting to go somewhere now. This is definitely helped by the NLP abilities of LLMs (which should be used only for that). The natural language to Prolog idea makes sense, now it needs to be more reliable. I'd be curious to know how many times the multiple-try path is exercised (the paper doesn't quite focus on that). More research is required obviously.

Now the impact seems clear and this is mostly bad news. This reduces the production of public knowledge so everyone looses. Ironically it also means less public knowledge available to train new models. At some point their only venue to fine tune their models will be user profiling which will be private... I've a hard time seeing how we won't end up stuck with another surveillance apparatus providing access to models running on outdated knowledge. This will lock so many behaviors and decisions in place.

Of course I recommend reading the actual research paper. This article is a good summary of the consequences though. LLMs definitely can't be trusted with formal reasoning including basic maths. This is a flaw in the way they are built, the bath forward is likely merging symbolic and sub-symbolic approaches.

Indeed, we should stop listening to such people who are basically pushing fantasies in order to raise more money.

OK, this paper picked my curiosity. The limitations of the experiments makes me wonder if some threshold effects aren't ignored. Still this is a good indication that the question is worth pursuing further.

The arm race is still on-going at a furious pace. Still wondering how messy it will be when this bubble bursts.

If you run the number, we actually can't afford this kind of generative AI arm race. It's completely unsustainable both for training and during use...

This is a short article summarizing a research paper at the surface level. It is clearly the last nail in the coffin for the generative AI grand marketing claims. Of course, I recommend reading the actual research paper (link at the end) but if you prefer this very short form, here it is. It's clearly time to go back to the initial goals of the AI field: understanding cognition. The latest industrial trends tend to confuse too much the map with the territory.

I definitely agree with this. I'm sick of the grand claims around what is essentially a parlor trick. Could we tone down the marketing enough so that we can properly think about making useful products again?

People are putting LLM related feature out there too hastily for my taste. At least they should keep in mind the security and safety implications.

This is clearly less high profile than the Scarlett Johanssen vs OpenAI one. Still this shows it has the potential to become a widespread (even though shady) practice. This might need some regulation fairly soon.

This is indeed important to be able to run such models locally. Will still require more optimization but it's slowly getting there. The reproducibility it brings is especially necessary for science.