Additional excerpts from Sections 1, 2, and 3 can clarify the issues discussed in Section6. See the attached Section6.pdf. This includes the excerpts in the previous version.
John
Alex,
The issues you mention happen to be topics that are included in that article I'm writing. I'll add some excerpts from other sections to Section6.pdf and send the update to these lists.
Alex> pictures that are recorded in our memory and are more or less accessible depending on access to visual memory.
Certainly. And not just visual memory, but memory of all perceptions, internal and external, are fundamental to what Peirce called the phaneron. All those perceptions are continuous and language consists of a string of discrete words that represent concepts that people consider significant. Diagrams constructed as patterns of concepts and relations serve as the intermediate stage between continuous perceptions and discrete strings of words.
The mappings go both ways: Perceptions -> images -> mental diagrams -> languages (spoken, signed, and artificial). And languages -> mental diagrams -> images -> actions in and on the continuous world (or some local part of it). The mental diagrams (and representations in one, two, three, or more dimensions) are an essential stage in those mappings.
C. S. Peirce recognized the importance of diagrams, and he had plans to extend them to "stereoscopic moving images". At the language end, nodes of the diagrams can be mapped to and from discrete words or concepts. At the image end, the nodes of diagrams can be moved and mapped to the parts of an image, either static or dynamic, which they represent.
Alex> A broad interpretation of the term diagram is possible. This is somewhat reminiscent of systems engineering. Consider "system thinking" vs "diagrammatic thinking".
Of course. The only change I would make is to replace "vs" with "and". Systems thinking is and must be in terms of diagrams that relate one-dimensional specifications (in words or other kinds of symbols) to the three-dimensional moving systems that engineers design and build.
Alex> It is important that a mind is able to store and operate with visual images - this is cooler than diagrams.
I'm not sure about the temperature. But human memory (and probably the mental imagery of other animals) can include imagery from perception as well as imagery from imagination. (In another note, I'll tell you the story about how Yojo the cat dreamt that there were monsters under the bed.)
Alex> LLM for me is an engineering invention around which there is a lot of noise, because it unexpectedly turned out to be capable of simulating many mental activities.
Yes. I'm glad that you used the critical word "many". The crucial addition is "but not all." I believe that LLMs are valuable for what they do. But as discrete patterns that support a limited set of operations, they are limited in what they can do.
Alex> Interesting topics include visual thinking and movie thinking.
Yes. That's what Peirce wrote in 1911, when he mentioned "stereoscopic moving images." In December of that year, he introduced an extension of his existential graphs called Delta graphs. Unfortunately, he had an accident before he finished writing his MS about them. But what he did write seems to be along the lines we have been discussing.
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
John,
There is an important difference between a diagram and a visual representation (pictures that are recorded in our memory and are more or less accessible depending on access to visual memory):
you need to be able to read diagrams.
A broad interpretation of the term diagram is possible. This is somewhat reminiscent of systems engineering. Consider "system thinking" vs "diagrammatic thinking".
It is important that a mind is able to store and operate with visual images - this is cooler than diagrams.
LLM for me is an engineering invention around which there is a lot of noise, because it unexpectedly turned out to be capable of simulating many mental activities.
I'm slowly learning how LLM works. There are a lot of surprises there.
As far as I know, there are GPTs that can build diagrams and even accept them as input. After all, the first ANN layer is more likely intended for an image than for text.
Interesting topics include visual thinking and movie thinking.
Alex
вт, 26 сент. 2023 г. в 23:35, John F Sowa <sowa(a)bestweb.net>:
Alex,
The only relevant item in that reference is a publication that is cited before the paywall: https://arxiv.org/pdf/2309.06979.pdf
What they prove is that you can train a system of LLMs to simulate a Turing machine. But that proves nothing. Almost every AI system designed in the past 60 years can be "trained" to simulate a Turing machine.
Every LLM that is trained on natural language data is limited to the kind of "thinking" that is done in natural languages. As I pointed out in Section6.pdf (and many other publications), NL thinking is limited to all the ambiguities and limitations of NL speaking. In human communication, NLs must be supplemented by context, shared background knowledge, and gestures that indicate or point to non-linguistic information.
The great leap of science by the Egyptians, Stone-hengers, Babylonians, Chinese, Indians, Greeks, Mayans, etc., was to increase the precision and accuracy of their thinking by going beyond what can be stated in ordinary languages. And guess what their magic happens to be? It's DIAGRAMS!!!!
Translating thoughts from diagrams to words is a great leap in communication. But it cannot replace the precision and generality of the original thinking expressed in the original diagrams.
As I said, you cannot design the great architectures of ancient times, the complex machinery of today, or any of the great scientific innovations of the past 500 years without geometrical diagrams that are far more complex than anything you can state in humanly readable natural language.
I admit that it is possible to translate any geometrical design or any bit pattern in a digital computer into a specification that uses the words and syntax of a natural language. But what you get is an immense amount of verbiage that no human could read and understand.
That is the most important message that we can get across in the forthcoming mini-summit. LLMs trained on NL input cannot go beyond NL thinking, and they cannot do any thinking that can go beyond thoughts expressible in NLs. To test that statement, show somebody (anybody you know) a picture, have them describe it, and have somebody else draw or explain what they heard, and have a fourth person compare the original to the explanation. (By the way, my previous sentence would be much clearer if I had included a drawing.)
John
Alex,
The only relevant item in that reference is a publication that is cited before the paywall: https://arxiv.org/pdf/2309.06979.pdf
What they prove is that you can train a system of LLMs to simulate a Turing machine. But that proves nothing. Almost every AI system designed in the past 60 years can be "trained" to simulate a Turing machine.
Every LLM that is trained on natural language data is limited to the kind of "thinking" that is done in natural languages. As I pointed out in Section6.pdf (and many other publications), NL thinking is limited to all the ambiguities and limitations of NL speaking. In human communication, NLs must be supplemented by context, shared background knowledge, and gestures that indicate or point to non-linguistic information.
The great leap of science by the Egyptians, Stone-hengers, Babylonians, Chinese, Indians, Greeks, Mayans, etc., was to increase the precision and accuracy of their thinking by going beyond what can be stated in ordinary languages. And guess what their magic happens to be? It's DIAGRAMS!!!!
Translating thoughts from diagrams to words is a great leap in communication. But it cannot replace the precision and generality of the original thinking expressed in the original diagrams.
As I said, you cannot design the great architectures of ancient times, the complex machinery of today, or any of the great scientific innovations of the past 500 years without geometrical diagrams that are far more complex than anything you can state in humanly readable natural language.
I admit that it is possible to translate any geometrical design or any bit pattern in a digital computer into a specification that uses the words and syntax of a natural language. But what you get is an immense amount of verbiage that no human could read and understand.
That is the most important message that we can get across in the forthcoming mini-summit. LLMs trained on NL input cannot go beyond NL thinking, and they cannot do any thinking that can go beyond thoughts expressible in NLs. To test that statement, show somebody (anybody you know) a picture, have them describe it, and have somebody else draw or explain what they heard, and have a fourth person compare the original to the explanation. (By the way, my previous sentence would be much clearer if I had included a drawing.)
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
Hi Andrea,
The topic you touched is so hot that there should be a lot of overview done yet. For me one source of it is a medium portal. Unfortunately it's a little bit paywalled :-(
Have a look at the newest [1].
Alex
[1] https://medium.com/@paul.k.pallaghy/llms-like-gpt-do-understand-agi-implica…
Some people have claimed that generative AI based on LLMs can support a path toward Artificial General Intelligence. But the best that can be said is that LLMs are useful for machine translation (especially for Standard Average European), for NL interfaces to computer systems, and for various kinds of text processing, summarization, and generation -- provided that they are supplemented with suitable methods for error checking.
I am currently writing an article that explains why LLMs are not capable of supporting the full range of human thinking, reasoning, and language understanding. In fact, they could not replace the brain of a dog, a raven, or a horse on tasks they do best. See the attached Section6.pdf, which includes Section 6 of the article and the opening paragraphs of Section 7. At the end, it includes two references for covering some of the material in the preceding five sections of the article I'm writing.
John
I'm sorry that the 2008 version of CL is not available. The 2018 version has some complex extensions that nobody uses. The IKL extensions are simpler and more useful. Unfortunately, the 2018 ISO standard does not separate the extensions from the much simpler and more useful core.
There were some debates about the usefulness of the different versions, Unfortunately, the other one became the official ISO standard. For the IKL extension and many related publications, see https://jfsowa.com/ikl/
John
----------------------------------------
From: "James Davenport' via ontolog-forum" <ontolog-forum(a)googlegroups.com>
That page has gone, and is replaced by the 2018 version:
https://standards.iso.org/ittf/PubliclyAvailableStandards/c066249_ISO_IEC_2…
James Davenport
Hebron & Medlock Professor of Information Technology, University of Bath
National Teaching Fellow 2014; DSc (honoris causa) UVT
Former Fulbright CyberSecurity Scholar (at New York University)
Former Vice-President and Academy Chair, British Computer Society
Apparently, the operations supported by DNA can serve as a biological computer. Excerpts below.
www.newscientist.com/article/2391747-dna-based-computer-can-run-100-billion…
John
_________________
DNA-based computer can run 100 billion different programs
Mixing and matching various strands of DNA can create versatile biological computer circuits that can take the square roots of numbers or solve quadratic equations.A liquid computer can use strands of DNA to run over 100 billion different simple programs. It could eventually be used for diagnosing diseases within living cells.
Fei Wang at Shanghai Jiao Tong University in China and his colleagues set out to make circuits similar to those on a computer chip, except with DNA molecules acting as wires and instructing the wires to configure in certain ways.When you enter a command on a conventional computer, it instructs electrons to flow through a specific path on a silicon chip. These circuit configurations each correspond to different mathematical operations – adding functions to chips means adding such paths.
To replace the wiring with DNA, Wang and his team modelled how to combine short segments of DNA into larger structures that could serve as circuit components, like wires, or function to direct those wires to form different configurations. They put this into practice by filling tubes with DNA strands and a buffer fluid and letting them attach to each other, combining into larger molecules through chemical reactions. The researchers also equipped all the molecules with fluorescence markers so they could keep track of what the circuit was doing based on how its parts were glowing....
Alex,
Those things were done and published years ago. They are not research issues, and there is nothing controversial about them. They were published in an official ISO standard. The latest version was published in 2018, but it is more complex, and the subset that was defined in 2007 is the only version that has been implemented and used: ISO/IEC standard 24707 for Common Logic. Even more important, it can be downloaded for free: http://standards.iso.org/ittf/PubliclyAvailableStandards/c039175_ISO_IEC_24…
The ISO standard for Common Logic specifies the core semantics in an abstract syntax that is independent of any readable notation of any kind. Then it states that any concrete syntax (linear or diagrammatic) that has a formally defined mapping to the abstract syntax may be called a dialect of Common Logic. Then three different concrete syntaxes are specified in the Appendices: (1) Common Logic Interchange Format (CLIF), which has a LISP-like syntax: (2) Conceptual Graph Interchange Format (CGIF); and (3) an XML-based notation (XCL).
In that standard, the core semantics is formally equivalent to Peirce's existential graphs. The formal name for the notation is "core CGIF", but I use the name EGIF (Existential Graph Interchange Format) because the core can be mapped to and from the graphic notation for EGs. Anything stated in the full CLIF or CGIF or XCL dialects can be mapped to CGIF and then to the core EGIF. The mappings are defined in that standard.
For more details about the full graph notation plus extensions, see the peer-reviewed research publication in the International Journal of Applied Logics: Sowa, John F. (2018) Reasoning with diagrams and images, http://www.collegepublications.co.uk/downloads/ifcolog00025.pdf . That issue of the journal contains several articles presented at a conference in Bogota, Columbia. My article is the second one. It defines an extension to EGs that also supports mappings to and from images.
But before reading all those formal publications, I recommend the slides from the talk that I presented at the European Sematic Web Conference in 2020: https://jfsowa.com/talks/escw.pdf .
These slides present a simpler overview, which may help smooth the way toward the more detailed formalism. They also contain more links to other publications and presentations that can add useful background. See the links at the bottom of most slides, and the suggested readings in the last slide.
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
John,
For me the next steps are
-to find axiomatic theories of EG, CG in your egtut.pdf [0] or other papers.
-wait for development of [1].
-to continue with E2HOL [2] where we need algorithms: string is input, graph or diagram is output.
I am happy we align our terminology.
Alex
[0] https://jfsowa.com/pubs/egtut.pdf
[1] https://inquiryintoinquiry.com/2023/09/15/logical-graphs-formal-development…
[2] https://www.researchgate.net/publication/366216531_English_is_a_HOL_languag…
Alex,
As I said, we are in total agreement. Every dictionary definition, everything you cited in your previous note, and your most recent comment:
Alex> Diagrams are geometrical structures (special kind of pictures, figures, usually labeled) chosen by humans to help with representations, explanations and reasoning.
That is what Peirce said, I said, all the dictionaries said, and all the references you cited said. The only differences are in the amount of surrounding detail. And none of that detail denies the conclusion.
In summary. structures are or can be described by mathematical formulas that are independent of any plans, purposes, or uses that any human might intend. But diagrams are structures that people have selected or drawn for the purpose of representing, explaining, or reasoning about something.
That's my summary of the basic points in all the references any of us copied or cited. That includes everything you, I, or anybody else said in this thread. What else is there to say?
John
----------------------------------------
From: "Alex Shkotin" <alex.shkotin(a)gmail.com>
John,
Let's write down my dissenting opinion:
Diagrams are geometrical structures (special kind of pictures, figures, usually labeled) chosen by humans to help with representations, explanations and reasoning.
Let's just note this difference.
Alex
чт, 14 сент. 2023 г. в 23:13, John F Sowa <sowa(a)bestweb.net>:
Alex,
Both dictionaries I quoted (and others I did not bother to quote) make a very clear distinction between structures (as patterns that exist independently of what anybody may think or say about them) and diagrams (as patterns that people or animals or computers choose to represent for explaining or reasoning about structures).
And thank you for the definition from Wikipedia, which makes exactly the same distinction and emphasizes that it goes back to prehistoric times (over 10 thousand years ago). Euclid and other geometers (more than 2,000 years ago) adopted diagrams for the foundation of mathematics, especially geometry -- and those diagrams are also used for reasoning and for applications in science, engineering, and architecture.
And thank you for the three citations to the Stanford articles. The one on diagrams discusses the writings by C. S. Peirce, and it also cites two things by me: (1) My book on Knowledge Representation, published in 2000, and (2) an article I wrote in 2011, which was published in the journal Semiotica: https://jfsowa.com/pubs/egtut.pdf
The article about the structure of scientific theories admits that humans may have discovered and stated the theories. But it focuses on the patterns in the theory that are independent of the scientists who discovered or stated them. Their primary example is Newtonian mechanics, but it ignores anything that Newton himself thought or did. I discusses only the mathematical patterns.
Today, we know that the mathematical pattern of Newtonian mechanics is only approximately true about the universe. But the patterns implied by that theory exist (in a mathematical sense) independently of what we think about them. When considered as a mathematical formula and the collection of patterns implied by that formula, it is independent of what anyone may think about those patterns..
The article about structural realism admits that some people might consider structures as things that people built or imagined. But it also makes a case that the structures that are really real -- they exist independently of what anybody may think about them.
In summary, the American heritage definition of diagram is as good as any and better than most:
- A plan, sketch, drawing, or outline designed to demonstrate or explain how something works or to clarify the relationship between the parts of a whole.
- A graphic representation of an algebraic or geometric relationship.
- A chart or graph.
This implies that diagrams are chosen by humans for explanations and reasoning. Structures are patterns considered as existing by themselves. The processes that create the structures (human or non-human) are not relevant to their existence as structures. Unless anybody can find any better terminology for any application of ontology, I believe that this distinction is the best we have.
John
----------------------------------------
From: "alex.shkotin" <alex.shkotin(a)gmail.com>
IN ADDITION
"A diagram is a symbolic representation of information using visualization techniques. Diagrams have been used since prehistoric times on walls of caves, but became more prevalent during the Enlightenment.[1] Sometimes, the technique uses a three-dimensional visualization which is then projected onto a two-dimensional surface. The word graph is sometimes used as a synonym for diagram."
https://en.wikipedia.org/wiki/Diagram
Alex,
Both dictionaries I quoted (and others I did not bother to quote) make a very clear distinction between structures (as patterns that exist independently of what anybody may think or say about them) and diagrams (as patterns that people or animals or computers choose to represent for explaining or reasoning about structures).
And thank you for the definition from Wikipedia, which makes exactly the same distinction and emphasizes that it goes back to prehistoric times (over 10 thousand years ago). Euclid and other geometers (more than 2,000 years ago) adopted diagrams for the foundation of mathematics, especially geometry -- and those diagrams are also used for reasoning and for applications in science, engineering, and architecture.
And thank you for the three citations to the Stanford articles. The one on diagrams discusses the writings by C. S. Peirce, and it also cites two things by me: (1) My book on Knowledge Representation, published in 2000, and (2) an article I wrote in 2011, which was published in the journal Semiotica: https://jfsowa.com/pubs/egtut.pdf
The article about the structure of scientific theories admits that humans may have discovered and stated the theories. But it focuses on the patterns in the theory that are independent of the scientists who discovered or stated them. Their primary example is Newtonian mechanics, but it ignores anything that Newton himself thought or did. I discusses only the mathematical patterns.
Today, we know that the mathematical pattern of Newtonian mechanics is only approximately true about the universe. But the patterns implied by that theory exist (in a mathematical sense) independently of what we think about them. When considered as a mathematical formula and the collection of patterns implied by that formula, it is independent of what anyone may think about those patterns..
The article about structural realism admits that some people might consider structures as things that people built or imagined. But it also makes a case that the structures that are really real -- they exist independently of what anybody may think about them.
In summary, the American heritage definition of diagram is as good as any and better than most:
- A plan, sketch, drawing, or outline designed to demonstrate or explain how something works or to clarify the relationship between the parts of a whole.
- A graphic representation of an algebraic or geometric relationship.
- A chart or graph.
This implies that diagrams are chosen by humans for explanations and reasoning. Structures are patterns considered as existing by themselves. The processes that create the structures (human or non-human) are not relevant to their existence as structures. Unless anybody can find any better terminology for any application of ontology, I believe that this distinction is the best we have.
John
----------------------------------------
From: "alex.shkotin" <alex.shkotin(a)gmail.com>
IN ADDITION
"A diagram is a symbolic representation of information using visualization techniques. Diagrams have been used since prehistoric times on walls of caves, but became more prevalent during the Enlightenment.[1] Sometimes, the technique uses a three-dimensional visualization which is then projected onto a two-dimensional surface. The word graph is sometimes used as a synonym for diagram."
https://en.wikipedia.org/wiki/Diagram
четверг, 14 сентября 2023 г. в 12:26:31 UTC+3, alex.shkotin:
John,
We can dive into a fascinating exploration of how the terms diagram and structure are used in everyday life.
Meanwhile I decided to look into the Stanford Encyclopedia of Philosophy. And I was glad to find an article [1] that explained your approach to me.
Unfortunately, the term structures does not have a separate article. But there is an article “The Structure of Scientific Theories” [2]. This is what I do to the best of my ability with an emphasis on formalization. After all, before formalizing a theory, it must be structured. However, there is an interesting article “Structural Realism” [3].
Alex
[1] https://plato.stanford.edu/entries/diagrams/
[2] https://plato.stanford.edu/entries/structure-scientific-theories/
[3] https://plato.stanford.edu/entries/structural-realism/
чт, 14 сент. 2023 г. в 01:27, John F Sowa <so...(a)bestweb.net>:
Since I suggested that anybody who is trying to define anything should check the definitions in a good dictionary, I decided to take my own advice. See the attached defs.htm for definitions of the words 'diagram' and 'structure' in the American Heritage Dictionary and the Merriam Webster Dictionary. In general, I have found the American Heritage definitions and etymologies very good. They are usually clearer and more precise than the definitions in other dictionaries. But it's always useful to get a second or third opinion.
An important distinction: A structure is a pattern in an entity of some kind. A diagram is a pattern that somebody draws or imagines as a representation or explanation of a pattern that somebody observed of found in some structure.
Therefore, a diagram would be more likely to be the kind of pattern that some human or animal or computer would be likely to use to support reasoning or computation about a pattern of any kind.
John
Signspiel • 1
• https://inquiryintoinquiry.com/2019/04/17/signspiel-1/
All sorts of players have given us all sorts of spiel about speech acts
over the years but Peirce stands out from the chorus in giving us models
of semiotic processes whose generation by triadic sign relations allows
them to maintain a constant relation among signs, their active interpretants
in conduct, and their ultimate pragmata, the objects and objectives of the
whole action. Shy of that, the speilerei of Austin and Wittgenstein simply
never gets off the ground.
Regards,
Jon
cc: https://www.academia.edu/community/Vvr3EV
cc: https://mathstodon.xyz/@Inquiry/11106389953288650
