I used the abbreviation BS to avoid being flagged by things that flag stuff.  The authors are not condemning ChatGPT.  As they say, "We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Frankfurt (On Bullshit, Princeton, 2005):  the models are in an important way indifferent to the truth of their outputs" . . . 

I agree with that comment.  It emphasizes my point:  LLMs generate hypotheses (guesses) whose truth values are unknown.  Technically, they may be called abductions.  Further testing and deduction are necessary before any abduction can be trusted.  Following is the abstract of the article at https://link.springer.com/article/10.1007/s10676-024-09775-5 

John

______________________________

Michael Townsen Hicks · James Humphries · Joe Slater

Abstract:  Recently, there has been considerable interest in large language models: machine learning systems which produce humanlike text and dialogue. Applications of these systems have been plagued by persistent inaccuracies in their output; these are often called “AI hallucinations”. We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Frankfurt (On Bullshit, Princeton, 2005): the models are in an important way indifferent to the truth of their outputs. We distinguish two ways in which the models can be said to be bullshitters,  and argue that they clearly meet at least one of these definitions. We further argue that describing AI misrepresentations as bullshit is both a more useful and more accurate way of predicting and discussing the behaviour of these systems.