Why you shouldn’t ask chatbots to act like an expert
A new study from Wharton’s Generative AI Labs suggests prompting chatbots to act like a subject matter expert can actually hurt accuracy.
2 min. read
By testing six large language models on graduate-level questions in science, engineering, and law, researchers from the Wharton School’s Generative AI Labs (GAIL) find that so-called “expert personas” deliver no consistent boost in accuracy, and in some cases worsen the results. The pattern holds across most models, suggesting organizations may get more value from how they use AI than from how they prompt it.
This cuts against widely-promoted guidance from leading chatbot makers—including OpenAI, Google, and Anthropic—encouraging users to assign roles, with prompts that cast the model as a “math teacher” or “tax expert.” But the evidence suggests otherwise.
The study is the fourth in Wharton’s “Prompting Science” series examining how different prompting techniques shape AI performance. It is co-authored by Savir Basil, Ina Shapiro, GAIL senior fellows Dan Shapiro and Lilach Mollick, and management professor Ethan Mollick.
The findings point to a different way of working with AI, coming after a period in which “prompt engineering” was widely seen as a way to get better results, even spawning a new role focused on crafting the right instructions.
Instead, the paper suggests organizations are likely to get more value from how tasks are framed for AI, what gets fed in, and how results are checked, rather than from layering personas on top of prompts.
On the move: Water quality and antimicrobial resistance testing goes mobile
Penn researchers have developed a mobile field laboratory allowing them to test water quality and track antimicrobial resistance in and around San Cristóbal in the Galápagos.
Penn researchers Vanessa and Alexis North are studying one of the largest collections of Egyptian blue objects ever found in the ancient Near East, shedding new light on how the world’s first synthetic pigment may have traveled far beyond Egypt.
A new role for dental providers? Rethinking where vaccines are given
Vaccinations have traditionally been administered in primary care settings or pharmacies. However, for about a decade, some states have authorized dentists to administer vaccines, and many others did so during the COVID-19 pandemic. The effectiveness of this approach has led some countries and states to expand or consider expanding the role of dental providers in delivering routine vaccines, including those against COVID-19 and the human papillomavirus (HPV), and in providing vaccine education.
Penn student develops a way for computer chips to run more efficiently
Fluid Silicon, a platform from President’s Sustainability Prize winner Nhlanhla Mavuso, allows computer chips to continuously monitor their health and self-tune as their characteristics change. The technology has the potential to reduce energy usage in data centers and improve reliability in mission-critical applications.
