On Sunday, CBS 60 Minutes presented a segment about Khanmigo, an AI tutor that is powered by LLM technology. It has shown some very impressive results, and the teachers who use it in their classes have found it very helpful. It doesn't replace teachers. It helps them by offloading routine testing and tutoring,
As I have said many times. there are serious limitations to the LLM technology, which requires evaluation to avoid serious errors and hallucinogenic disasters. Question; How can Khanmigo and related systems avoid those disasters?
I do not know the details of the Khanmigo implementation. But from the examples they showed, I suspect that they avoid mistakes by (1) Starting with a large text that was written, tested, and verified by humans (possibly with some computer aid); (2) For each topic, the system does Q/A primarily by translation; (3) And the LLM technology was first developed for translation and Q/A; (4) if the source text is tested and verified, a Q/A system that is based on that text can usually be very good and dependable.
But the CBS program did show an example where the system made some mistakes.
Summary: This example shows great potential for the LLM technology. But it also shows the need for evaluation by the traditional AI symbolic methods. Those methods have been tried and tested for over 50 years, and they are just as important today as they ever were.
As a reminder: LLMs can be used with a large volume of sources to find information and to generate hypotheses. But if the source is very large and unverified for accuracy, it can and does find and generate erroneous or even dangerously false information. That is why traditional AI methods are essential for evaluating what they find in a large volume of source data.
Danger; The larger the sources, the more likely that the LLMs will find bad data. Without evaluation, bigger is definitely not better. I am skeptical about attempts to create super large volumes of LLM data. Those systems consume enormous amounts of electricity with a diminishing return on investment.
There is already a backlash by employees of Google and Elon M.
John