New subject: Insights into three of the important problems associated with LLMs/Generative AI

11 Oct 2023

Amit and anybody who did or did not attended today's talk at the Ontology Summit
session,

All three of those questions below involve metalevel issues about LLMs and various
reasoning issues with and about generative AI.  The first and most important is about
anything generated by LLMs:  Is it true, false, or possible?  After that  are How?  Why?
and How likely?

The biggest limitation of LLMs is that they cannot do any reasoning by themselves.  But
they can often find some reasoning by some human in some document from somewhere.  If they
find something similar, they can apply it to solve the current problem.  But the word
'similar' raises critical questions:  How similar?  In what way is it similar/  Is
that kind f similarity relevant to the current question or problem?

For example, the LLMs trained on the WWW must have found textbooks on Euclidean geometry. 
If some problem is stated in the same terminology as the books on geometry, the LLMs might
find an answer and apply it.  

But more likely, the problem will be stated in terms of the subject matter, such as
building a house, plowing a field, flying an airplane, or surveying the land rights in a
contract dispute.  In those cases, the same geometrical problem may have few or no words
in common with Euclid's description of the geometry and the terminology of each of the
applications.

For these reasons, a generative AI system, by itself, is unreliable for any
mission-critical application.  It is best used under the control and supervision of some
system that uses trusted methods of AI and computer science to check, evaluate, and
supplement whatever the generative AI happens to generate.

As an example of the kinds of systems that my colleagues and I have been developing, see
https://jfsowa.com/talks/cogmem.pdf , Cognitive Memory For Language, Learning, and
Reasoning, by  Arun K. Majumdar and John F. Sowa.

See especially slides 44 to 64.  They show three applications for which precision is
essential.  There are no LLM systems today that can do anything useful with those
applications or anything similar.  Today, we have a new company, Permion.ai LLC, which has
developed new technology that takes advantage of BOTH LLMs and the 60+ years of earlier AI
research. 

The often flaky and hallucinogenic LLMs are under the control of technology that is
guaranteed to produce precisely controlled reasoning and evaluations.  Metalevel reasoning
is its forte.  It evaluates and filters out whatever may be flaky, hallucinogenic, or
inconsistent with the given facts.

John

----------------------------------------
From: "Sheth, Amit" &lt;AMIT(a)sc.edu&gt;

There has been a lot of discussion on LLMs and GenAI on this forum.
I would like to share papers related to three major challenges:

1 Is it Human or AI?
Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think — 
Introducing AI Detectability Index

2. Measuring, characterizing and countering Hallucination (Hallucination Vulnerability
Index)
The Troubling Emergence of Hallucination in Large Language Models –An Extensive
Definition, Quantification, and Prescriptive Remediations

3. Fake News/misinformation
FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W
Question-Answering

Introduction/details/links to papers (EMNLP 2023):
https://www.linkedin.com/feed/update/urn:li:activity:7117565699258011648 

I think this community won’t find this perspective alien:

Data driven only approaches can’t/won’t address these challenges well—
need to understand the duality of data and knowledge.
Knowledge (including KGs/ontologies/world model/structured semantics) and 
neuro-symbolic AI (arxiv) which use a variety of relevant knowledge (linguistic, common
sense,
domain specific, etc) will play critical role in
addressing these. The same goes for three of the most important requirements
(knowledge will play a critical role in making progress on these):
grounding, intractability, and alignment.
More to come on this from #AIISC.

Cheers,
Amit

Amit Sheth LinkedIn, Google Scholar, Quora, Blog, Twitter	
Artificial Intelligence Institute; NCR Chair	
University of South Carolina	
#AIISConWeb, #AIISConLinkedIn, #AIISConFB