Despite decades of effort, scientists have still not found a substitute for reading the paper

Science is difficult, but not for the reason you think. It’s not that conceiving a research program is difficult. It’s not that planning and performing an experiment or observational study is difficult. It’s not that getting funding to support your planned study is difficult. It’s not even that explaining the results of an experiment and how they answer your original research question is difficult. All that is challenging, sure, but it’s what most scientists signed up for. What’s challenging, on the most grinding day-to-day level, is keeping up with the reading.

In principle I can keep up with the state of my field by regularly checking in on journals in evolution and ecology—that’s baseline effort. There’s also the reading I find myself doing every time I start writing a grant proposal or paper, making sure I have the background covered and haven’t missed important precedents, and the reading I continue to do as I write and discover new information I want to cite. Then there’s reading to evaluate: student assignments and projects, papers for peer review and editorial decisions, grant proposals (if and when I volunteer to help with them), and (least frequently) CVs of job applicants. It’s a lot, and much of it occurs under some manner of deadline pressure. I need to sift through pages of text to find a critical point of fact, or to evaluate the scholarly value of the text, and I don’t have enough time to do it.

This is, I think, the fundamental reason why we’re in the midst of a boom in proposed solutions to the problem of reading scientific papers.

Many of the newest alternatives to reading are enabled by some flavor of large language model (“AI”) summarization or chatbot. Papers on the website of the megapublisher Wiley now have an “AI Companion” widget embedded with them, offering to answer questions about the paper but conspicuously not promising to answer them accurately (tip of the hat to Abeba Birhane for putting this on my timeline). A utility recently presented in Nature proposes to generate an “AI agent” from any paper, so readers can ask it questions about the work presented and query the supporting data to re-analyze it on the fly.

Screenshot of the page for a (very good!) paper by Alvestegui et al. in a recent issue of the journal Ecology, with the Wiley AI Companion widget open.

However this impulse pre-dates the widespread availability of LLMs. It’s why journals have added significance statements, visual abstracts, and even “research article summaries” embedded as a first page of the PDF file containing the paper. Journal websites may offer a reader, by my count, as many as six different glosses on a paper—an invited commentary on the paper, an editor’s summary, a “research article summary”, a significance statement, a visual abstract, and an old-fashioned abstract written by the authors—in addition to the paper itself. And that’s before a chatbot gets its chance to digest the text into a dubiously reliable improvised Q&A.

Journals, or their publishers, are plainly flailing to find a way to maximize the number of people who read and cite and recognize the “impact” of the papers they publish, and the consensus seems to be that they can do this by making it easier to read the papers without actually reading them. Readers are certainly susceptible to the temptation, given how much text we’re faced with—and the anxiety that we might miss something critical in the vast array of literature on a topic, or the tangle of less-than-familiar terminology we might find in the publications list attached to a colleague’s application for an award, or for tenure.

But the trouble with all of these additions and applications piled atop the humble research article is that they’re new solutions to a comprehensively solved problem. The vast majority of peer-reviewed articles, especially in the sciences, are presented with an abstract, a tightly length-limited summary of an article’s content; many journals even require structured abstracts that break out key findings as a labeled section. Most articles also have titles that convey their most important finding in an even shorter length; and the actual bodies of most articles are structured to enable a knowledgeable researcher to zero in on key findings: Introduction, Methods, Results, Discussion. The sequence from title to abstract to paper proper takes the reader through more and more detailed descriptions of the work presented and the authors’ interpretation of its results; the addition of supporting data and online supplements has meant that there are often still richer details available with a little independent work.

Significance statements, visual abstracts, research summaries, even conversational chatbots and “agentic” AI models—all of these are reproducing functions of the title and abstract, with varying degrees of input from the authors or even knowledgeable human editors. Some of them may be useful in specific contexts. Significance statements can conceivably convey the importance of results in less-specialized language for readers of a journal covering many topics, though inevitably at the cost of the precision that motivates specialized vocabulary; and in my experience of both reading and writing them, significance statements are remarkably like rewritings of the abstract found on the same PDF page.

It’s true that authors often give their papers titles that are more calculated to catch the eye than to accurately convey content, and they often write abstracts poorly or misleadingly. Are editor-written summaries or digests created by unsupervised chatbots inherently more reliable or informative? I don’t know any reason to think that they are. Editors bring their own quirks and biases and ulterior motives to the presentation of a paper in a journal; LLM-generated summaries flow in the grooves worn by the most common phrasings in their training corpus, to the point that they may auto-complete results that aren’t in the paper they’re supposed to be summarizing.

The problem these various summaries and addenda are trying to solve is the problem of communication between human minds. Scientific articles—however stilted and structured—are written by human beings to convey information to other human beings. That communication is inherently imperfect, because both the writers and the readers are imperfect, limited, beings. No technological intervention can change this, and no tool or editorial presentation absolves a scientist from the responsibility to use her own scholarly judgement about what is interesting and relevant and important in a paper she’s read.

At the end of the day, you still have to read the paper, and figure out what it means for yourself.