Anatoly, Stephen, Dan, Alex, and every subscriber to these lists,

I want to emphasize two points:  (1) I am extremely enthusiastic about LLMs and what they can and cannot do.  (2) I am also extremely enthusiastic about the 60+ years of R & D in AI technologies and what they have and have not done.  Many of the most successful AI developments are no longer called AI because they have become integral components of computer science.  Examples:  compilers, databases, computer graphics, and the interfaces of nearly every appliance we use today:  cars, trucks, airplanes, rockets, telephones, farm equipment,  construction equipment, washing machines, etc.  For those things, the AI technology of the 20th century is performing mission-critical operations with a level of precision and dependability that unaided humans cannot achieve without their help.

Fundamental principle:  For any tool of any kind -- hardware or software -- it's impossible to understand exactly what it can do until the tool is pushed to the limits where it breaks.  At that point, an examination of the pieces shows where its strengths and weaknesses lie.

For LLMs, some of the breaking points have been published as hallucinations and humorous nonsense.  But more R & D is necessary to determine where the boundaries are, how to overcome them, work around them, and supplement them with the 60+ years of other AI tools.

Anatoly>  When you target LLM and ANN as its engine, you should consider that this is very fast moving target. E.g. consider recent work (and imagine what can be done there in a year or two in graph-of-thoughts architectures) . . .

Yes, that's obvious.  The article you cited looks interesting, and there are many others.  They are certainly worth exploring.  But I emphasize the question I asked:    Google and OpenAI have been exploring this technology for quite a few years.   What mission-critical applications have they or anybody else discovered and implemented?

So far the only truly successful applications are in MT -- machine translation of languages, natural and artificial.   Can anybody point to any other applications that are mission critical for any business or government organization anywhere?

Stephen Young>  Yup.  My 17yo only managed 94% in his Math exam.  He got 6% wrong.  Hopeless - he'll never amount to anything. 

The LLMs have been successful in passing various tests at levels that match or surpass the best humans.  But that's because they cheat.  They have access to a huge amount of information on the WWW about a huge range of tests.  Bur when they are asked routine questions for which the answers or the methods for generating answers cannot be found, they make truly stupid mistakes. 

No mission-critical system that guides a car, an airplane, a rocket, or a farmer's plow can depend on such tools. 

Dan Brickley> Encouraging members of this forum to delay putting time into learning how to use LLMs is doing them no favours. All of us love to feel we can see through hype, but it’s also a brainworm that means we’ll occasionally miss out on things whose hype is grounded in substance. 

Yes, I enthusiastically agree.   We must always ask questions.  We must study how LLMs work, what they do, and what their limitations are.   If they cannot solve some puzzle, it's essential to find out why.     Noticing a failure on one problem is not an excuse for giving up.  It's a clue for guiding the search.

Alex>  I'm researching how LLMs work.  And we will really find out where they will be used after the hype in 3-5 years.

Yes.  But that is when everybody else will have won the big contracts to develop the mission-critical applications. 

Now is the time to do the critical research on where the strengths and limitations are.  Right now, the crowd is having fun building toys that exploit the obvious strengths.  The people who are doing the truly fundamental research are exploring the limitations and how to get around them.

John