Alex,
The article you cited is interesting, and I recommend it as a useful technique for limiting the possible hallucinations of LLMs. But the cartoon I copy below is just as applicable to this article as it is to any other application of LLMs. There is no breakthrough here.
Furthermore, the authors' claim that their system understands or comprehends anything is absurd. What their system generates is "a large pile of linear algebra" that does not differ from the pile in the cartoon in any essential way. The only reason why their system performs better than the huge pile generated by OpenGPT is that it's restricted to peer-reviewed scientific articles. The main reason why the results are fairly good is that different scientific disciplines use very different terminology. Therefore, the texts from different articles do not mix or interfere or pollute one another.
Please note what the authors have done: They created a collection of LLMs from a collection of published scientific articles from multiple disciplines and used LLMs to represent both English text and diagrams in those texts.
Mixing the two different kinds of syntax is a useful enhancement, but there is nothing new in the underlying technology. You can get the same or better enhancement by mixing data in three very different linear syntaxes: English, SQL, and OWL. IT's useful to use the same spelling for the same concepts. But if there are enough examples, the LLMs are able to detect the similarities and do the equivalent translations.
The information in the diagrams is expressed in the same words as the English text, but the two-dimensional syntax of the diagrams represents a second language. There is nothing new there, since LLMs can relate languages with different syntax. For any language processor, the difference between a linear string and a 2-D diagram is trivial. When you send the diagram to another system, the 2-D syntax is mapped to a 1-D syntax that uses the same kinds of syntactic markers as describing a 2-D diagram in English.
Fundamental principle: For the LLMs, it's irrelevant whether the source is a linear language or a system of diagrams that were mapped to a linear string. The result is a pile of linear algebra. See the cartoon.
John
From: "Alex Shkotin" <alex.shkotin@gmail.com>
Peter,
Thanks for that link. That cartoon is a precise characterization of how LLMs process data. It was drawn in the 1990s when linear algebra usually meant something computed with matrices. LLMs go one step farther by using tensors, but the results are in the same ballpark (or sewer).
Fundamental principle: any machine learning system must be used with a system for evaluating or checking the answers. For simple factual questions, a database can be used. For more complex questions, logical deduction is necessary. For any kind of system, ontology can detect obvious hallucinations, but ontology by itself is insufficient to detect incorrect details that happen to be in the correct category.
John
