Artificial intelligence and illusions of understanding in scientific research
Lisa Messeri & M. J. Crockett
Scientists are enthusiastically imagining ways in which artifcial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusionsof understanding in which we believe we understand more about the world than
we actually do. Such illusions obscure the scientifc community’s ability to see theformation of scientifc monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risksintroducing a phase of scientifc enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancingdiscussions of responsible knowledge production in the age of AI.
Fig. 1 | Illusions of understanding in AI-driven scientific research.
a, Scientists
using AI tools for their research may experience an illusion of explanatory depth. In this example, a scientist uses an AI Quant to model a phenomenon (X) and believes they understand X with more depth than they actually do.
b, In a monoculture of knowing, scientists are vulnerable to an illusion of exploratory breadth, in which they falsely believe they are exploring a space of all testable hypotheses, whereas they are actually exploring a narrower space of hypotheses that are testable with AI tools.
c, In a monoculture of knowers, scientists are vulnerable to an illusion of objectivity, in which they falsely believe that AI tools do not have a standpoint (as desired for Oracles and Arbiters) or are able to represent all possible standpoints (as desired for Surrogates in research using human participants), whereas AI tools actually embed the standpoints of their training data and their developers.
LEGGI TUTTO https://doi.org/10.1038/s41586-024-07146-0
