Close this search box.
Close this search box.

Knowledge in the age of generative AI

Can generative artificial intelligence help us to know the truth?

The first response that comes to mind is without doubt ‘no’: after all, if this AI is supposedly generative, then it generates content rather than telling us what actually exists. Indeed, these artificial intelligences have in theory no first-hand experience of the real world. How would they know what is true and what is not?

We are all well aware of the issue of hallucinations: when a large language model (LLM) invents a fact that does not exist, in an effort to produce a probable text. ‘What did Napoleon eat before the Battle of Waterloo?’ ‘An apple tart.’ It is not absurd, but it is not true. A language model struggles to tell the difference because it operates through statistical probabilities and not through precise research in a database. Of course, a model’s answer will often be right, especially if the information requested frequently appears in the data on which the model was trained (e.g. the date of the Battle of Waterloo). In short, the language model produces the plausible rather than the true and struggles to accept what it may not know. In this, it is similar to the false scholars described by Descartes: ‘If they wish to know how to speak about all things (…), they will achieve this more easily by contenting themselves with verisimilitude, which can be found without great difficulty in all sorts of matters, than by seeking the truth, which is only gradually discovered in some, and which, when it comes to speaking about others, obliges them to confess frankly that they are ignorant of them.’

What we may call the ‘peril of plausibility’ is enough to advise against the use of LLMs, in their current state, as simple search engines or sources of raw information. But this peril goes further still. Beyond inaccurate facts, there is the problem of banal ideas. A plausible text runs the risk not only of being false but of being average, normal. What could be an advantage when writing an umpteenth press release becomes a disadvantage when inventing new ideas, or even reporting on an original idea. All progress in knowledge implies a departure from what is commonly accepted: Galileo would not have been Galileo if he had submitted to the common knowledge of his time.

Some therefore imagine a scenario where, by delegating reflections to language models, humanity will end up experiencing a ‘knowledge collapse’ leading to the loss of real knowledge, drowned in increasingly distorted summaries, trivialised by machines, to the point of scientific regression.[1] Without going quite that far, there is a real risk of experiencing this phenomenon in a company: if the company relies too heavily on generative AI to process its knowledge base, it risks losing its expertise. Expertise presupposes not only knowledge of an entire field, but also knowledge of its exceptions, irregularities and extreme cases. An airline does not operate solely thanks to the relevance of its technical documentation, but, of course, thanks to the specific knowledge, anchored in experience, of its engineers and specialised technicians, who may remember unique and unlikely situations.

Nevertheless, it would be a cruel mistake to stop there. Because, in other ways, and if we overcome this peril of plausibility, generative AI can help us to make decisive progress in the world of knowledge.

First, we must realise that the potential for a text written by AI not to be ‘true’ is not a flaw in the AI itself; it is a flaw in our critical thinking. Too often we rely on arguments of authority: I am reading this text by a great author, so I trust it. But the text itself provides no certainty of truth, even when it is written by a human. Its truth depends on the internal coherence of its arguments, and it is incumbent on us as readers to spot it. The same applies to images: we have associated photography with truth, but that is an all-too-easy shortcut. The photographer Annie Leibovitz declared that AI does not worry her at all because ‘Photography itself is not really real’.[2] It can be altered, and even without alteration, it is only a transcription of photons on a plane, not the object itself.

A text written by AI must therefore be read with a critical mind… much like a text written by a human. In the words of Roland Barthes, who would have been unperturbed by ChatGPT, ‘the unity of a text lies not in its origins (…) but in its destination’.[3] So what can we do with the products of generative AI to help us advance our knowledge?

First, we need to focus on using these models as processors rather than producers of information. The language model is a universal translator, which can not only translate English text into French and vice versa, but also translate French text into a Python script, a poem into prose, or an Excel spreadsheet into a PowerPoint presentation. It can also translate a complicated text into a simple one, and make fields communicate that would otherwise never have spoken. A small study recently showed that although ChatGPT is often used by researchers to write the introductions to their papers, as revealed by a measure of ChatGPT’s language quirks in academic journals (for example, the word delve, overused by ChatGPT), researchers do not consider these introductions to be any less good than those written by hand.[4] If researchers make discoveries by themselves and share them with the world with the help of ChatGPT, the communication of knowledge is improved. Fluid exchange between researchers is one of the key factors in scientific progress, which is why printing has contributed so much to the progress of modern science since the Renaissance. Generally speaking, this transformative capacity of LLMs helps to improve communication at all levels: between technicians and non-technicians, between different specialities, and even between different personalities.

More fundamentally, the language model can act as a stimulus. We have seen that it tends to produce texts and ideas that are plausible rather than true, but this is not necessarily a fault. Innovators often need a sparring partner to evaluate and test their ideas; every Sherlock Holmes needs a Dr Watson, and an LLM is an excellent Watson. It is often by confronting common ideas that we come up with unique ones. What is more, if the LLM spontaneously tends towards the average, this tendency can be fought by throwing it off track, by asking it to imagine ideas inspired by unexpected sources. Indeed, by forcing it to interpolate between usually distinct notions and inspirations (why not, financial strategy and Vermeer’s art), the LLM will be forced to create something new. Not everything will be worthwhile, but valuable insights may emerge.

Generative AI can also complete unnoticed aspects of a thought process. You may be better than ChatGPT in your area of expertise, but it is probably better than you in every other area. You may have a new idea that only a human could have originated, but the applications of this idea in fields other than your own can no doubt be imagined better than you by an LLM.

Finally, we cannot rule out the hypothesis that future models will have much more advanced reasoning capabilities than they do today, in particular by going beyond the framework of the language model and linking it with other architectures that are less subject to the peril of plausibility. Perhaps we will see artificial intelligences proposing new theories and putting them to the test. The possibilities for discovery would then be almost infinite.

In the meantime, and provided we guard against the risks mentioned above, we can be sure that major progress will be made in the next few years, both thanks to generative AI, serving as a support for innovators, and, let’s not forget, thanks to non-generative AI, which is already enabling superhuman tasks to be carried out, such as the massive prediction of the structure of proteins[5], the discovery of new materials[6] or the reading of Roman papyri charred by the eruption of Vesuvius.[7]


[1] Peterson, Andrew. AI and the Problem of Knowledge Collapse. (2024).
[2]Photography itself is not really real… I use all the tools available’: Annie Leibovitz talks AI, at her L’Académie des Beaux-arts induction, Digital Camera World, 24 March 2024.
[3] Barthes, Roland. ‘La mort de l’auteur’ (1968). In Le Bruissement de la langue (1984).
[4] Sikander, B., Baker, J.J., Deveci, C.D., Lund, L., & Rosenberg, J. (2023). ChatGPT-4 and Human Researchers Are Equal in Writing Scientific Introduction Sections: A Blinded, Randomized, Non-inferiority Controlled Study. Cureus, 15. (2023).
[5] Jumper, J., Evans, R., Pritzel, A. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596 , 583–589 (2021).
[6] Merchant, A., Batzner, S., Schoenholz, S.S. et al. Scaling deep learning for materials discovery. Nature 624, 80–85 (2023).
[7] The Vesuvius Challenge (2023).


Raphaël Doan – Author, Si Rome n’avait pas chuté
Accuracy Talks Straight #10 – The Academic Insight