From Jargon to Clarity: A Randomized Trial of AI-Generated Cancer Research Summaries
How do you make cutting-edge cancer research understandable to the very people whose lives it affects most—patients and their caregivers—without losing accuracy or nuance? That question lies at the heart of a recent study published in JCO CCI exploring whether generative artificial intelligence can be harnessed to create lay abstracts and summaries of urologic oncology research. The researchers wondered: can AI not only produce text quickly, but also maintain the delicate balance between readability and scientific fidelity? And if so, does this actually help patients and caregivers understand complex findings better than traditional scientific abstracts?
To find out, a team from the University of Southern California began with a dataset of forty original abstracts, chosen from high-profile journals covering prostate, bladder, kidney, and testis cancers. These represent some of the most important areas of urologic oncology, where treatment decisions often hinge on understanding highly technical results. Each abstract was fed into a new available generative AI tool called Pub2Post, specifically designed to create media contents from scientific and scholar literature, which produced three separate lay summaries per abstract. Why three? To test for consistency—if the tool’s outputs varied wildly, it would raise questions about reliability. In total, this process generated six hundred sections of text, which the researchers then subjected to a series of evaluations.
One key dimension was readability. Scientific abstracts, while rigorous, often score poorly on standard readability metrics because they are dense with jargon, long sentences, and specialized terminology. In contrast, the AI-generated lay summaries were expected to produce texts closer to what an educated non-specialist might follow with ease. Indeed, the comparison was striking: while the original abstracts averaged a readability score of 25.3, the AI outputs soared to 68.9—a highly significant improvement (P < .001). In plain terms, this means the summaries shifted from material typically accessible only to advanced academic readers to something approachable by a wider audience.
But readability is only half the story. Could the AI preserve the integrity of the science? Two independent reviewers carefully assessed each section for accuracy, completeness, and clarity, while also watching for “hallucinations”—those notorious instances where AI invents details not present in the source text. Overall, the system performed impressively: quality scores ranged between 85% and 100%, and hallucinations cropped up in only about 1% of sections. The methods sections were where the AI stumbled slightly, scoring 85% accuracy compared to 100% for the original abstracts, and achieving full “trifecta” quality (accuracy, completeness, clarity) 82.5% of the time versus 100% for the originals. Yet even here, the AI’s performance was still solid, and in all other sections it maintained high quality at or above 92.5%.
The ultimate test, however, was not in metrics but in human comprehension. The researchers ran a pilot study with 277 patients and caregivers, who were randomly assigned to read either the original abstracts or the AI-generated lay summaries. Participants then completed assessments of both comprehension and perception—essentially, how well they understood the material and how they felt about it. The results were clear: those who received the lay summaries consistently outperformed those who had the original abstracts, with significantly better comprehension scores and more favorable perceptions. In fact, the presence of a lay summary was the only factor that consistently predicted improved understanding (P < .001).
"This study provides data, including a small randomized trial, showing that lay abstracts and summaries produced by a GAI tool improves the readability of medical research articles to patients and the general public, while largely maintaining accuracy" reported Ziad Bakouny, MD, MSc, JCO Clinical Cancer Informatics Associate Editor . "Such tools promise to make scienti c research more accessible, which is particularly important to help with the growing problem of scientific misinformation"
Think of it like translating a foreign language. A patient handed a scientific abstract is often like someone given a manual in technical legalese—it’s possible to parse it with training, but exhausting and prone to misunderstanding. The AI-generated lay summaries acted like a skilled interpreter: simplifying sentence structures, choosing accessible words, and preserving meaning without distortion. Importantly, this translation was achieved in under ten seconds per abstract, a speed that underscores the scalability of such an approach.
The study concludes with a balanced message. Pub2Post has clear potential to democratize access to oncology research, allowing patients and caregivers to engage more deeply with findings that affect their decisions and well-being. At the same time, the researchers caution against blind trust: human oversight remains essential, particularly in checking for subtle inaccuracies or omissions in methods.
The promise is powerful—an ecosystem where research is no longer locked behind specialist language, but shared in forms that respect both the rigor of science and the needs of its ultimate beneficiaries. The next steps will likely involve refining these tools, embedding oversight frameworks, and perhaps expanding beyond oncology to other fields where clarity can empower. In short, AI may not replace the role of human communicators, but it can become a remarkably effective ally in bridging the gap between research and real lives.
