3 Oct
2023
3 Oct
'23
10:53 p.m.
Anatoly, Stephen, Dan, Alex, and every subscriber to these lists,
I want to emphasize two points: (1) I am extremely enthusiastic about LLMs and what they
can and cannot do. (2) I am also extremely enthusiastic about the 60+ years of R & D
in AI technologies and what they have and have not done. Many of the most successful AI
developments are no longer called AI because they have become integral components of
computer science. Examples: compilers, databases, computer graphics, and the interfaces
of nearly every appliance we use today: cars, trucks, airplanes, rockets, telephones,
farm equipment, construction equipment, washing machines, etc. For those things, the AI
technology of the 20th century is performing mission-critical operations with a level of
precision and dependability that unaided humans cannot achieve without their help.
Fundamental principle: For any tool of any kind -- hardware or software -- it's
impossible to understand exactly what it can do until the tool is pushed to the limits
where it breaks. At that point, an examination of the pieces shows where its strengths
and weaknesses lie.
For LLMs, some of the breaking points have been published as hallucinations and humorous
nonsense. But more R & D is necessary to determine where the boundaries are, how to
overcome them, work around them, and supplement them with the 60+ years of other AI
tools.
Anatoly> When you target LLM and ANN as its engine, you should consider that this is
very fast moving target. E.g. consider recent work (and imagine what can be done there in
a year or two in graph-of-thoughts architectures) . . .
Yes, that's obvious. The article you cited looks interesting, and there are many
others. They are certainly worth exploring. But I emphasize the question I asked:
Google and OpenAI have been exploring this technology for quite a few years. What
mission-critical applications have they or anybody else discovered and implemented?
So far the only truly successful applications are in MT -- machine translation of
languages, natural and artificial. Can anybody point to any other applications that are
mission critical for any business or government organization anywhere?
Stephen Young> Yup. My 17yo only managed 94% in his Math exam. He got 6% wrong.
Hopeless - he'll never amount to anything.
The LLMs have been successful in passing various tests at levels that match or surpass the
best humans. But that's because they cheat. They have access to a huge amount of
information on the WWW about a huge range of tests. Bur when they are asked routine
questions for which the answers or the methods for generating answers cannot be found,
they make truly stupid mistakes.
No mission-critical system that guides a car, an airplane, a rocket, or a farmer's
plow can depend on such tools.
Dan Brickley> Encouraging members of this forum to delay putting time into learning how
to use LLMs is doing them no favours. All of us love to feel we can see through hype, but
it’s also a brainworm that means we’ll occasionally miss out on things whose hype is
grounded in substance.
Yes, I enthusiastically agree. We must always ask questions. We must study how LLMs
work, what they do, and what their limitations are. If they cannot solve some puzzle,
it's essential to find out why. Noticing a failure on one problem is not an excuse
for giving up. It's a clue for guiding the search.
Alex> I'm researching how LLMs work. And we will really find out where they will
be used after the hype in 3-5 years.
Yes. But that is when everybody else will have won the big contracts to develop the
mission-critical applications.
Now is the time to do the critical research on where the strengths and limitations are.
Right now, the crowd is having fun building toys that exploit the obvious strengths. The
people who are doing the truly fundamental research are exploring the limitations and how
to get around them.
John