Bad news for anybody who claims that larger amounts of data improve the performance of LLM-based systems. The converse is true; Smaller, specialized amounts of data produce better results for questions in the same domain.
In any case, hybrid systems that use symbolic methods for evaluating results are preferable to pure LLM-based techniques.
Some excerpts below from www.newscientist.com/article/2449427-ais-get-worse-at-answering-simple-ques… .
John
____________________
AIs get worse at answering simple questions as they get bigger
Using more training data and computational power is meant to make AIs more reliable, but tests suggest large language models actually get less reliable as they grow.
AI developers try to improve the power of LLMs in two main ways: scaling up – giving them more training data and more computational power – and shaping up, or fine-tuning them in response to human feedback.
José Hernández-Orallo at the Polytechnic University of Valencia, Spain, and his colleagues examined the performance of LLMs as they scaled up and shaped up. They looked at OpenAI’s GPT series of chatbots, Meta’s LLaMA AI models, and BLOOM, developed by a group of researchers called BigScience.
The researchers tested the AIs by posing five types of task: arithmetic problems, solving anagrams, geographical questions, scientific challenges and pulling out information from disorganised lists.
They found that scaling up and shaping up can make LLMs better at answering tricky questions, such as rearranging the anagram “yoiirtsrphaepmdhray” into “hyperparathyroidism”. But this isn’t matched by improvement on basic questions, such as “what do you get when you add together 24427 and 7120”, which the LLMs continue to get wrong.
While their performance on difficult questions got better, the likelihood that an AI system would avoid answering any one question – because it couldn’t – dropped. As a result, the likelihood of an incorrect answer rose.
The results highlight the dangers of presenting AIs as omniscient, as their creators often do, says Hernández-Orallo – and which some users are too ready to believe. “We have an overreliance on these systems,” he says. “We rely on and we trust them more than we should.”
Alex: "We need to formalize our scientific theories to use computers to their full potential."
I agree, but the formalization is ALWAYS context dependent. The engineering motto is fundamental:
ALL THEORIES ARE WRONG, BUT SOME ARE USEFUL.
That is true about formalization. It's only precise for subjects that can be expressed in finite bit strings. For 99.9% of all the information we get every second of our lives, vagueness is inescapable. We must deal with it by informal methods of approximations. Any formal statement is FALSE in general, but it may be useful when the limitations are made explicit.
In your note below, you mention computer models. But any model for a digital computer has already assumed a mapping to bit strings. But an engineering model must recognize the complexity and CONTINUITY of the world.
Natural languages are very flexible and much more expressive than any model for a digital computer. If you ignore that flexibility, you destroy their
power and your formalization is guaranteed to be FALSE .
A translation of a natural language to a formal language may SOMETIMES be necessary. But different applications will require different ways of translating the same NLs, As the engineers will agree, any formal specification can only be made in the context of and with the knowledge about the specific application.
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
John,
Let's split a formalization in two steps.
I) structural representation of knowledge. Here, instead of a sequence of words, we get a structure (aka syntactic). It can even be just nonsense like
"Гло́кая ку́здра ште́ко будлану́ла бо́кра и курдя́чит бокрёнка" see
Proposal for structural representation of English sentences see, for formal languages here.
II) structural knowledge processing. What kind of "logic" i.e. a rule of knowledge processing we use in this or that science, engineering or everyday
life?
We should ask these particular scientists, engineers or citizens.
How to formalize their rules of knowledge processing is our task here. These rules are far from Modus Ponens.
Some rules we use to solve simple tasks about ugraphs pointed out here.
It should be also mentioned that there is an initial step usually not included in formalization: formal, mathematical representation of physical bodies and processes.
We usually call them computer models. 3D-twins are the most famous.
We apply our formalized knowledge to 3D twins using a computer to gain useful insights into real things and processes.
It's a good idea to separate language and logic. In many cases, we know the language of our opponent, but we don't know her rules for processing knowledge.
So we have a first-order LANGUAGE (actually a family of languages, but let's take one) and a set of first-order logics.
We need to formalize our scientific theories to use computers to their full potential.
Alex
Alex and Chuck,
That claim is FALSE in general, and determining the error bounds is essential.
It is true that you can write a formal statement that seems to state something similar to what is stated in English or other natural languages. But that does not imply that the two statements are equivalent.
A vague statement may express a continuous range of possibilities, but the translation to a statement in logic is limited to a very precise and very limited range of possibilities. Sometimes that is an advantage, but sometimes it can be horribly false or misleading or disastrous.
Engineers know this point very well. I quoted their motto in my previous note: "All theories are false, but some are useful." This point is absolutely TRUE. And I would apply it to your claim about formalization.
The critical issue is to determine what range of values in a translation is acceptable or useful. Unless you emphasize that range of options, your formalization is an
invitation to DISASTER.
Re Leibniz: He had many good ideas, but he oversimplified issues about precision. He did not emphasize the importance of vagueness and the dangers of ignoring the error bounds.
John
----------------------------------------
From: "Chuck Woolery" <chuck(a)igc.org>
Alex. Well stated!
From: ontolog-forum(a)googlegroups.com <ontolog-forum(a)googlegroups.com> On Behalf Of Alex Shkotin
----------------------------------------
John,
Any verbal knowledge can be formalized, at least for the English language🦉 How precisely this knowledge is a topic for scientists and practitioners working in a particular area of reality. We simply formalize knowledge to use the power of a computer. But you are right, we need a reason for formalization as it's hard. In some cases, formalization can reveal some unclear areas in informal knowledge. And sometimes, in very rare cases, formalization can find errors in a mathematical text. There is a report of this
kind from the Isabelle research group.
Just to make it clear: even wrong, inaccurate, vague knowledge may be formalized. If we need to.
And after that we can run the verification algorithm and it will say that this knowledge is incorrect, inaccurate, or vague.
The first person to put forward this idea as a project was G. Leibniz, who was 25 years old. He hoped to obtain a formal language in 2-3 years🏋️
Alex
Alex,
As I have said many times, in email notes, slides, and publications, a precise formal ontology of everything cannot be done until all the unsolved problems in the foundations of every science have been solved. That includes psychology (of humans and all other living things on any planet in the universe).
But I do believe that it's possible and highly desirable to develop formal ontologies of things that are implemented on digital computers. The reason why that's possible is that anything implemented in strings of bits is discrete and finite. Therefore the implementation itself is a formal definition of what the program does. And a formal description in a more concise and readable form is possible and valuable..
I am happy to see that you are considering "the field of knowledge itself: education, the learning process, and so on is of course very interesting as a branch of Psychology."
Nothing in any of those topics can be formalized precisely because every one of them has an enormous number of unknown issues for which the best known studies are incomplete. I have a high respect for what has been done in those fields. But every research issue they solve opens up many, many more unsolved problems.
Attempts at formalization can be useful in order to show the vast realm of unknown and unknowable issues that make any formal theories of everything hopeless.
Re: " they stopped classifying objectives and moved on to classifying mental abilities themselves."
That seems to be a step toward recognizing the immense scope of the problem. But it's essential to make the distinction between the formal representation of what is computable and the unknown and poorly understood continuum of the mind and the world. The discrete can be formalized, but it's impossible to formalize the continuum in any finite notation with a discrete set of symbols.
Engineers have an excellent way of summarizing these issues: "All theories are wrong, but some are useful."
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
Hi Ali,
You still haven't sent us which version of BT you're working with. But between the first in 50s and the second in 90s versions, an important change occurred: they stopped classifying objectives and moved on to classifying mental abilities themselves. I'll take a look tomorrow, because the field of knowledge itself: education, the learning process, and so on is of course very interesting as a branch of Psychology.
Alex
Michael,
The word 'understand' is a vague word that has no formal definition of any kind. In any specific example, it's essential to replace it with a statement of the operations that had been performed.
MDB: ChatGPT ... can understand Turtle but it seems to understand natural language definitions better.
Short summary: LLMs process patterns of symbols. Their most reliable applications map strings of symbols (in natural or artificial languages) to strings of symbols in other languages. Those are not perfect, but they are the most reliable because they do the least amount of processing. But they do nothing that remotely resembles what humans do when they "understand" the source and target languages.
The more steps in the processing, the more unreliable the results. Translation is the most reliable because the source and target patterns are very closely related. LLMs can also be used to find information when a pattern in a question has a more distant relation to a
pattern in the target that is found.
That kind of search is more reliable the closer the search pattern is to the pattern that is found. But LLMs don't do any evaluation. If a search pattern has a closer match to bad data than good data, the bad data will be retrieved.
The most spectacular, but also the most unreliable applications of LLMs search for a pattern that does some kind of transformation and then apply that transformation to some source data to produce some target data. People often call these transformations "reasoning". But the kind of reasoning should be called "guessing" or "hypothesis" or "abduction".
Humans who understand what LLMs do can often find them very useful because they are sufficiently knowledgeable about the subject that (1) they recognize bad guesses and ignore them; and (2) they do further tests and checks to evaluate the answers before using them.
There is much more to say about the details. But never, ever use the words 'reasoning'
or 'understanding' for what LLMs do. However, it may be permissible to use the word 'reasoning' for a hybrid system that uses symbolic methods for deduction and evaluation of the results that LLMs find or generate.
And most of all, it's best to reject , discard, or ignore any publication that claims LLMs are approaching a human level of understanding or reasoning. I have never seen any article about artificial general intelligence (AGI) that has any insight whatsoever into human intelligence.
John
PS: Penrose wrote some excellent books on physics. When he wanders outside of his level of expertise, his ideas may be interesting -- or not.
----------------------------------------
From: "Michael DeBellis" <mdebellissf(a)gmail.com>
Sent: 9/26/24 11:41 AM
As we know there is a large and subtle discussion around Penrose thesis. I am not in it. I am sure that there should be a forum for this topic. And your question
"Why do we have to accept the "indisputable
validity" of these statements that lie outside the scope of P?"
is to RP not to me. Sorry.
Alex, no need to apologize. Although I'm starting to wonder how subtle the discussion really is. I'm starting to suspect this is a case of the emperor having no clothes (ironic given the title of one of Penrose's book). Specifically, 95% of the people who read Penrose's book don't have the capability to understand Godel at all, and of the 5% that can at least somewhat grasp Godel (most of us) even fewer (like Paolo Mancosu, the guy at Berkeley who taught a class I audited a long time ago on Godel, Turing, etc.) who are the real experts aren't interested enough to point out obvious errors.
For example D. Hilbert wrote the axiomatic theory of Euclid's geometry. Do we have formalization? No
I don't quite understand that. Isn't an "axiomatic theory" a formalization?
But our robots are waiting for them.
One of the ironies of this whole discussion IMO is that in some circumstances
(LLMs) it is now easier to communicate to software agents using natural language than formal language. I still need to do a lot more work on this but so far that is what I'm finding in my work with ChatGPT, it says it can understand Turtle but it seems to understand natural language definitions better. One of the things I plan to work on is a simple generator to generate basic NL descriptions of classes, properties, and axioms. It shouldn't be hard. If something like this exists I would appreciate a pointer
How many theories outside math are formalized?
One of the philosophers in the Vienna circle, think it was Carnap, thought that we SHOULD formalize other disciplines of science and attempted to do that for physics. I've always thought one of the reasons he failed was because to create such a complex model required some kind of tool. For a while I even thought it might be possible to do with OWL. I actually tried but it was soon apparent that 1) I don't know physics well
enough and 2) Even if I did (here I'm in violent agreement with John Sowa) OWL wasn't powerful enough for this kind of model.
Michael
On Wed, Sep 25, 2024 at 1:11 AM Alex Shkotin <alex.shkotin(a)gmail.com> wrote:
Michael,
As we know there is a large and subtle discussion around Penrose thesis. I am not in it. I am sure that there should be a forum for this topic. And your question
"Why do we have to accept the "indisputable validity" of these statements that lie outside the scope of P?"
is to RP not to me. Sorry.
And any discussion outside some theory, which should be definitely pointed to concrete theory, is just mind to mind games.
In Linguistics there are a lot of theories fighting. For example D. Everett has one theory and N. Chomsky has another.
Outside scope on any theory we just gymnastics the mind. Why not!
But the challenge is to formalize one or another existing theory 🎯
For example D. Hilbert wrote the axiomatic theory of Euclid's geometry.
Do we have formalization? No 😂
How many theories outside math are formalized? 0.
But our robots are waiting for them.
By the way topic of truth values is one of the subtle in math logic 👍
Everyone, who has created a formal ontology, formalized some theoretical knowledge. From what theory? Where is this theory expressed? How to justify his formalization by this theory?
We can always verbalize formalization and then must find in this particular theory a justification or ask experts.
Theory first, robots second 🏋️
Alex
Alex,
Please reread my note below. For a subject that cannot be formalized, the practice of formalization is not just a waste of time -- It is a BAD IDEA. Any formalization will make it say something that is guaranteed to be false. Therefore, people who understand the issues will reject whatever you produce, and people who don't understand the issues will mistakenly use your false results.
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
John,
What a nice day to read this
"Chuck's sentence is true about science and technology: "My hypothesis is that the language of science, technology, and mind is HOL." "
There are a lot of details to discuss later.
But the main direction is to formalize theories we already have in "science, technology, and mind"
Thank you,
Alex
сб, 21 сент. 2024 г. в 23:55, John F Sowa <sowa(a)bestweb.net>:
Alex and Chuck,
I strongly agree on the importance of formal logic, but I must also add that the overwhelming majority of information that we must deal with comes from perception and natural languages. There is no simple, general, dependable, and trustworthy method for mapping those sources to and from any version of logic. 'There are good approximations, but none of them are as precise and dependable as any version of logic.
Chuck's sentence is true about science and technology: "My hypothesis is that the language of science, technology, and mind is HOL."
But the human mind is far more general than anything that can be mapped to and from HOL by any known or imagined technology that can be processed by any kind of computerized system that is known or proposed or imagined.
In fact, I would add that the reasoning by your pet dog or cat is beyond what can be done with HOL. In fact everybody's favorite nematode C. elegans has only 303 neurons. Scientists have detected and mapped its complete connectome -- every connection of every neuron. But they are unable to predict or simulate its behavior,
There is much more to say about these issues, but there is one serious warning: Any formal ontology about the world is limited by the methods for mapping language and perception to any kind of logic. I sympathize with the concerns that Chuck mentions and links to.
But I am very well aware of the need for serious work on methods of detecting, correcting, and working around the limitations. Anybody who doesn't recognize and include such methods will be doing more harm than good.
For more about these issues, see https://jfsowa.com/ikl
That rather short web page has links to many important publications that discuss these issues. I strongly urge people to browse some of them.
John
Mike,
I agree that those two questions are extremely difficult questions about physics, and nobody knows how to anwer them. But that is not a question about mathematical logic, which Mihai Nadin was discussing in his note below.
For the first question, you could start by writing an equivalent in an Englishy kind of logic:
1. For every x of type "law of nature" for every time t, law x is consistent at time t.
That is a rather simple statement in logic, but nobody would know how to answer it by any possible experiment short of examining every region of the universe for all eternity.
The second has the same limitation. It's a simple statement in logic that cannot be answered without testing it in every location of the universe at all points in time.:,
2. For every x of type "law of nature" for every y of type "region of the universe", x is applicable at y.
John
----------------------------------------
From: "Mike Bergman" <mike(a)mkbergman.com>
Hi John,
OK; I'll bite, and give you two. See below.
On 9/21/2024 5:09 PM, John F Sowa wrote:
I agree with the note below. But nobody but a logician who has studied advanced issues in logic knows how to state an undecidable proposition,
I would challenge anybody to find a single undecidable proposition in any branch of science or engineering or economics or politics or any field other than advanced mathematics.
- "The laws of nature are consistent across time" (the counter was a favorite of Peirce)
- "The laws of nature are universally applicable across all regions of the universe."
Thanks, Mike
I won't deny that a professional logician might contrive an undecidable proposition about some subject matter in any of those fields. But I will predict with almost absolute certainty that no professional in that field would understand it. And even if the logician could explain that proposition to somebody in that field, I predict with almost absolute certainty that the professional in that field would consider it irrelevant or even ridiculous.
And by the way, if anybody could contrive such a proposition, I would be delighted to see it. Please send it to the list, state the proposition in English, translate it to some version of logic, prove that it is indeed undecidable, and find a professional in that field who would consider it significant.
I seriously doubt that anybody could do that, but I'd be happy to be proven wrong.
John
----------------------------------------
From: "Nadin, Mihai" <nadin(a)utdallas.edu>
Sent: 9/20/24 5:39 PM
To: "ontolog-forum(a)googlegroups.com" <ontolog-forum(a)googlegroups.com>
Subject: [ontolog-forum] goedel and more
Discussing Penrose is justified. BUT: the focus should be on Goedel. Allow me to repeat my advice: read the paper, i.e. his proof. Read also Turing, i.e. his proof for the Hilbert challenge.
Against my better judgment, I am suggesting one of my papers:
https://www.researchgate.net/publication/314424008_The_Intractable_and_the_…
(in my book https://link.springer.com/book/10.1007/978-3-031-43957-5 this is discussed in detail)
But I am not reacting to recent opinions only for the sake of pointing to my work.
The thing that concerns me regarding the discussion is expressed as
…and perhaps some day someone will somehow prove things that are unprovable according to GT/C
Let us agree that science is not only about what is possible, but also about what is not possible. Squaring the circle—anyone? Just to give an example. Within the number construct we are using, there is PI)—no need to redefine it here. As long as we stick to the numbers used currently in order to represent our measuring (quantitative aspects) of reality, the doubling of the cube and trisecting the angle will remain impossible because of how we defined . NOT even the not yet invented new forms of computation will do it.
Goedel discovered for a well-defined aspect of mathematics that consistency and completeness are not possible. Those using the outcome of his proof owe it to him to understand what the meaning of this is. Doubt is good (Dubito ergo sum), on the assumption that is grounded in knowledge. Get if from the source.
Mihai Nadin
https://www.nadin.wshttps://www.anteinstitute.org
Google Scholar
I agree with the note below. But nobody but a logician who has studied advanced issues in logic knows how to state an undecidable proposition,
I would challenge anybody to find a single undecidable proposition in any branch of science or engineering or economics or politics or any field other than advanced mathematics.
I won't deny that a professional logician might contrive an undecidable proposition about some subject matter in any of those fields. But I will predict with almost absolute certainty that no professional in that field would understand it. And even if the logician could explain that proposition to somebody in that field, I predict with almost absolute certainty that the professional in that field would consider it irrelevant or even ridiculous.
And by the way, if anybody could contrive such a proposition, I would be delighted to see it. Please send it to the list, state the proposition in English, translate it to some version of logic, prove that it is indeed undecidable, and find a professional in that field who would consider it significant.
I seriously doubt that anybody could do that, but I'd be happy to be proven wrong.
John
----------------------------------------
From: "Nadin, Mihai" <nadin(a)utdallas.edu>
Sent: 9/20/24 5:39 PM
To: "ontolog-forum(a)googlegroups.com" <ontolog-forum(a)googlegroups.com>
Subject: [ontolog-forum] goedel and more
Discussing Penrose is justified. BUT: the focus should be on Goedel. Allow me to repeat my advice: read the paper, i.e. his proof. Read also Turing, i.e. his proof for the Hilbert challenge.
Against my better judgment, I am suggesting one of my papers:
https://www.researchgate.net/publication/314424008_The_Intractable_and_the_…
(in my book https://link.springer.com/book/10.1007/978-3-031-43957-5 this is discussed in detail)
But I am not reacting to recent opinions only for the sake of pointing to my work.
The thing that concerns me regarding the discussion is expressed as
…and perhaps some day someone will somehow prove things that are unprovable according to GT/C
Let us agree that science is not only about what is possible, but also about what is not possible. Squaring the circle—anyone? Just to give an example. Within the number construct we are using, there is PI)—no need to redefine it here. As long as we stick to the numbers used currently in order to represent our measuring (quantitative aspects) of reality, the doubling of the cube and trisecting the angle will remain impossible because of how we defined . NOT even the not yet invented new forms of computation will do it.
Goedel discovered for a well-defined aspect of mathematics that consistency and completeness are not possible. Those using the outcome of his proof owe it to him to understand what the meaning of this is. Doubt is good (Dubito ergo sum), on the assumption that is grounded in knowledge. Get if from the source.
Mihai Nadin
https://www.nadin.wshttps://www.anteinstitute.org
Google Scholar
Alex and Chuck,
I strongly agree on the importance of formal logic, but I must also add that the overwhelming majority of information that we must deal with comes from perception and natural languages. There is no simple, general, dependable, and trustworthy method for mapping those sources to and from any version of logic. 'There are good approximations, but none of them are as precise and dependable as any version of logic.
Chuck's sentence is true about science and technology: "My hypothesis is that the language of science, technology, and mind is HOL."
But the human mind is far more general than anything that can be mapped to and from HOL by any known or imagined technology that can be processed by any kind of computerized system that is known or proposed or imagined.
In fact, I would add that the reasoning by your pet dog or cat is beyond what can be done with HOL. In fact everybody's favorite nematode C. elegans has only 303 neurons. Scientists have detected and mapped its complete connectome -- every connection of every neuron. But they are unable to predict or simulate its behavior,
There is much more to say about these issues, but there is one serious warning: Any formal ontology about the world is limited by the methods for mapping language and perception to any kind of logic. I sympathize with the concerns that Chuck mentions and links to.
But I am very well aware of the need for serious work on methods of detecting, correcting, and working around the limitations. Anybody who doesn't recognize and include such methods will be doing more harm than good.
For more about these issues, see https://jfsowa.com/ikl
That rather short web page has links to many important publications that discuss these issues. I strongly urge people to browse some of them.
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
Chuck, welcome on board.
The more we formalize the better for World. As this is a way to check LLMs and other algorithms in use.
And please share that "The status of this document is RFC (Request For Comments). It keeps a number of ideas to discuss. Abstract. Following the idea of R. Montague (RM for short) "English is a formal language" we will give examples of constructing operator expressions for sentences in the English language. For comparison of approaches, the examples are taken from the text [PTQ] by R. Montague. This makes it possible to compare operator bracket expressions with the author's constructions later."
Alex
сб, 21 сент. 2024 г. в 15:39, Chuck Woolery <chuck(a)igc.org>:
Alex, Thank you for this paragraph...
My hypothesis is that the language of science, technology, and mind is HOL. And when we have a framework for theories of mathematical logic, it will be the primary source for referencing any definitions, theorems, and proofs. We have the beginnings of a framework for undirected graph theory. Incidentally, we also need a framework for tasks, problems. Alex
With your permission I will be sharing it with other in my blog posts attempting to bring sanity to a world where religious, economic, and political words can mean damn near anything.
Cw
Chuck Woolery, Former Chair
United Nations Association, Council of Organizations
315 Dean Dr., Rockville, MD 20851
Cell:240-997-2209 chuck(a)igc.org