[CG] How far can we push any technology? (was Generative AI systems...

Andrea, Dan, Doug, Alex, As I keep repeating, I am enthusiastic about the ongoing research on generative AI and the LLMs that support it. But as I also keep repeating, it's impossible to understand the full potential of any computational or reasoning method without understanding its limitations. I explicitly address that issue for my own work. In my first book, Conceptual Structures, the final chapter 7 had the title "Limits of Conceptualization". Following is the opening paragraph: "No theory is fully understood until its limitations are recognized. To avoid the presumption that conceptual mechanisms completely define the human mind, this chapter surveys aspects of the mind that lie beyond (or perhaps beneath) conceptual graphs. These are the continuous aspects of the world that cannot be adequately expressed in discrete concepts and conceptual relations." One of the reviewers, who wrote a favorable review of the book, said that he was surprised that Chapter 7 refuted everything that went before. But actually, it's not a refutation. It just itemizes the many complex issues about human thought and thinking that go beyond what can be handled by conceptual graphs (and related AI methods, such as semantic networks and knowledge graphs). Those are very important research areas, and it's essential to understand what can and cannot be done with current technology. For a copy of that chapter, see https://jfsowa.com/pubs/cs7.pdf As another example, the AI Journal devoted an entire issue in 1993 to a review of a book on Cyc by Lenat & Guha. Lenat told me that my review was the most accurate, but it was also the most frustrating because I itemized all the difficult problems that they had not yet solved. Following is a copy of that review: https://jfsowa.com/pubs/CycRev93.pdf Lenat did not hold that review against me. In 2004, the DoD, which had invested a great deal of funding in the Cyc project held a 20-year evaluation of the Cyc project to determine whether and how much should they continue to invest. And Lenat recommended me as one of the members of the review committee. Our unanimous review was that (1) Cyc had developed a great deal of important research, which should be documented and made available to the public; (2) future development of Cyc should be funded mostly by commercial applications of Cyc technology; (3) government funding should be continued during the documentation stage and during the transition to funding by applications. Those goals were achieved, and Cyc continued to be funded by applications for another 19 years. So when I write about the limitations of generative AI and the LLM technology, I am writing exactly what must be done in any review of any project of any kind. A good review of any development must ALWAYS evaluate the strengths and limitations. But many (most? all?) of the people who are working on LLMs don't ask questions about the limitations. For example, I have a high regard for Geoffrey Hinton, who has been one of the most prominent pioneers in this area. But in an interview on 60 Minutes last Sunday, he said nothing about the limitations. He even suggested that there were no limits. For that interview, see https://www.cbs.com/shows/video/L25QUOdr6apMNr0ZWqDBCo9uPMd_SBWM/ As a matter of fact, many of the limitations I discussed in cs7.pdf also apply to the limitations of LLMs. In particular, they are the limitations of representing and reasoning about the continuous aspects of the world and their translations to and from a discrete, finite vocabulary of any language, natural or artificial. Andrea> I agree with the position of using LLMs wherever they are appropriate, researching the areas where they need more work, supplementing them where other technologies are strong, and (in general) "not throwing the baby out with the bath water". Yes indeed. Dan> The ability of these systems to engage with human-authored text in ways highly sensitive to their content and intent is absolutely stunning. Encouraging members of this forum to delay putting time into learning how to use LLMs is doing them no favours. All of us love to feel we can see through hype, but it’s also a brainworm that means we’ll occasionally miss out on things whose hype is grounded in substance. I certainly agree. I'm not asking anybody to stop doing their R & D. But I am asking people who promote LLMs to look at where they are running up against the limits of current versions and what can be done to go beyond those limits. Doug F> Note that much of the left hemisphere has nothing to do with language. In front of the language areas are strips for motor control of & sensory input from the right side of the body. The frontal lobe forward of those strips do not deal with language. The occipital lobe at the rear of the brain does not deal with language, either. The visual cortex in the temporal lobe also does not deal with language. This means that most of the 8 billion neurons in the cerebral cortex have nothing to do with language. I agree with that point. But I believe that the LLM proponents would also agree. They would say that those areas of the cortex are necessary for mapping language-based LLMs to and from perception and action. What they fail to recognize is the importance of the 90% of the neurons that do not do anything directly related to language. Alex> My proposal: let’s first agree that ANN is far from being only an LLM. LLM is by far the most noisy and unexpected of ANN applications. The question can be posed this way: we know about the Language Model, but what other models using ANN exist? I agree that we should explore the many ways that artificial NNs relate to the NNs in various parts of the brain. It's also important to recognize that there are many difference kinds of NNs in different areas of the brain, and they are organized in ways that are very different from the currently popular ANNs. In summary, there is a lot more research that remains to be done. I'm not telling anybody to stop what they're doing. I'm just recommending that they look at what more needs to be done before claiming that LLMs can do everything. John