Issue 5: Beware of Interesting New Research
How to Use Boring Old Research Instead
A few weeks ago, Todd Kashdan posted about his encounter with a reporter who didn’t want to report on research from 2013 because it was “too old.”
I just had a similar experience. I was talking to a journalist about my brand-spanking new research on unhelpful help (with my coauthors Julianna Pillemer and Teresa Amabile). It is so hot-off-the-presses it isn’t even off the press yet.
The journalist was asking me pretty normal questions about the article. But, then the conversation turned to when the data had been collected. The answer was mostly from 2011-2014.
“What!?” she said. “That changes everything!”
But does it? Like most researchers, I take it for granted that research is like cocktail shrimp — I should offer only the freshest to visiting journalists. I also understand the impulse to ask whether older research has a “use by” date. The world today feels radically different than a decade or two ago. A decade ago, A.I. was just a movie! Pandemics were problems for medieval peasants. Donald Trump was the host of a reality show! Surely research from these ancient times has now spoiled.
The laws of physics aren’t affected by new discoveries about it (OK, I see you Schrödinger), but the laws of humanity are. For social scientists, the changing world is a real problem. Organizational researchers like me are always shooting at a moving target — and the target transforms when you hit it. Despite a lot of hemming and hawing about whether our research makes a difference, a few rougue ideas have broken out of the ivory tower to infiltrate the real world. Major changes to how we work and collaborate have come from prospect theory, bounded rationality, work design theory, goal-setting theory, and psychological safety, for example. Those insights aren’t showing many signs of spoilage. In fact, like fine wine, they seem to improve as they age.
When considering whether “old” research still applies today, the most important question to ask is: Why wouldn’t it apply? Of course, science is constantly updating and correcting itself. But durable research insights are usually based on fundamental principles of psychology and social structure. Superficial differences between today and yesterday aren’t reasons to dismiss truisms about being human. Unless there is newer, better evidence, we should continue to assume that insights from the past are our best guess for explaining social reality.
Is interest in interestingness causing a crisis?
I wanted to write about the value of old research in part because of a bunch of flack our field has been taking. A new article by Daniel Engber in The Atlantic about cases of alleged research fraud in business school research has, rightly, garnered a lot of interest.
As Engber put it:
Business-school psychologists are scholars, but they aren’t shooting for a Nobel Prize. Their research doesn’t typically aim to solve a social problem; it won’t be curing anyone’s disease. It doesn’t even seem to have much influence on business practices, and it certainly hasn’t shaped the nation’s commerce. Still, its flashy findings come with clear rewards: consulting gigs and speakers’ fees, not to mention lavish academic incomes.
This article has generated a new wave of disparagement of “business school psychology” — you know, those money-grubbing pseudo-scientists (like me) who write HBR articles, sign lucrative deals to write books for a popular audience, and take the occassional speaking gig?
Countering Engbers’s points requires a complex, nuanced argument:
Actually, the above paragraph aside, I’m not upset by the article. I’m just looking for excuses to reference The Good Place. In truth, I agree with most of Engber’s concerns about our field. Luckily, so do many other smart researchers who came of age during the replication crisis (the name for the dawning realization that some well-cited findings in social psychology were difficult or impossible to replicate) and are now working tirelessly to improve the quality and reliability of research.
Personally, I think those smart people are doing a pretty good job - in my 20 years in the field, there’s been a ton of positive change in research practice. (Skip the rest of the paragraph if you aren’t that interested in getting into the weeds on the replication crisis - I’m not going to explain all this here, but
does a better job than I could). The many of the p-hacking techniques that literally used to be taught in grad school have become mostly a thing of the past. Replicating your own findings, pre-registration, and sharing data have become the norm. Reviewers are much more likely to call authors on fishy findings, like…smelling rotting fish leads people to be more suspicious (this was a real study). In fact, the rate at which the field has adapted is shocking because organizations and institutions suck at change. That doesn’t mean there’s not work to do yet. Obviously, we should make structural changes to encourage more reliable research and less fraud. But I don’t have much to add to what has already been said about what we, in the research community, should do.Instead, I want to pick up on Engber’s “flashy findings” as another source of problems. We will never completely stop everyone from doing click-baity research. But we can stop reading and writing about those click-bait findings.
The Problem with “Interesting” Reserach
Most people think research is boring. And for good reason - rigorous research should be a bit boring. Academic articles are and should be devoted to convincing other scholars of the bedrocks of science: the importance and validity of the work. They shouldn’t be super-interesting to non-scholars.
Yet, many academics chafe at our boorish stereotype. So we push each other and our PhD students to be more interesting. And to read Murray Davis’s classic article “That’s Interesting.”
To define the sociology of interestingness, Davis argued that people decide whether something is interesting or boring first, and then worry later about whether it was true or false. Interesting things need to affirm some of what we believe so as to be credible, but surprise us a bit by disconfirming an assumption or two.
In other words, little insights that tweak, but don’t upend your worldview, are the most interesting. Write-ups of research that promise “secret,” “hidden,” or “one weird trick” pique our interests more than, “research finds people don’t like it when their boss yells at them.” Most of us in the business of publicizing research already know that. Ironically, Davis’s ideas about interestingness aren’t that interesting anymore.
But there’s a problem in the relentless pursuit of Murray’s interestingness: Most of our assumptions are correct. To be interesting, you need to constantly refute yet another assumption — otherwise, its yesterday’s news. You can only disconfirm so many assumptions before you start questioning some valid ones.
So here’s the paradox. People won’t click, read, or watch unless something is interesting. So anything you write to reach a broad audience must be interesting. But constantly pitching things as interesting eventually strains the truth. So what can we do?
We should try to make old research more interesting
The most helpful research-based prescriptions are built on the bedrock of old research. Although I spend most of my time producing “cutting edge” research, that new research makes up only a tiny fraction of what I teach aspiring leaders. Most of the problems I encounter in the world show we need to pay more attention to old research. Most people don’t even know the basics of teamwork, creativity, or leadership discovered decades ago. Some examples of real silliness: I’ve met with an organization that still uses the MBTI to select employees for its leadership development program. I’ve talked to leaders who think that shouting at people is good leadership. I watch (and attend) pointless, poorly run meetings that should have been emails.
Research already has a lot to say about these problems. The problem is that old research, on its own, can’t find a way to make itself heard.
Sometimes, I forget to do some things I literally wrote the book on — I don’t lead every team they way I know they should be led because…its hard. Most work-based problems emerge because the realities of modern work are at odds with our psychology. So any distraction or stressor can throw us off even when we are well-informed.
Although I will continue to help raise awareness about new research among my colleagues, the prescriptions for living and working together better are always based on the new in conversation with the old.
Rather than looking for the tastiest cocktail shrimp research, we should focus on how to apply old insights to new situations. And the most important new situation is today. Today is different than yesterday, which was different than 10 years ago. When you read an old article or book, it may have outdated language and cultural references that seem to be speaking to our grandparents - they don’t feel like they are written for us to use today. That makes them easy to dismiss.
To come up with practical, research-based recommendations, I look first at studies of studies, like systematic reviews and meta-analyses. Those give us a high-level perspective on what the most solid research-based insights are.
And then, I try to come up with new examples of the same principle - to connect a timeless insight to the world today. Then, I spruce it up with language to make it more interesting. Even new studies are an excuse to re-introduce fundamental principles about its topic, not to extrapolate only from the most novel bits. The interestingness of a research-based piece comes from a new perspective on old knowledge — not fresh, flashy, but unverified research.
Unlike the journalists from the top of the post, we shouldn’t turn up our noses at “old” research like it was week-old cocktail shrimp. This isn’t just journalists - researchers themselves only promote their newest work on social media. Universities do the same. Very few try to get attention for work more than a year old. Rather than promoting research when it is published, we’d be better off applying our expertise to things that are already in the news. In fact, that’s how I ended up writing my book (more on that in another post).
Getting people to consume old-but-true ideas is like helping them to eat healthier. The challenge for us as writers, educators, and scientists is to help people to see how old wisdom fits into people’s current lives. And, in this recontextualization, old insights become new again.
“Old wine in a new bottle” shouldn’t always be a criticism. So raise a glass of old wine to old research - and let’s try to find a new audience for old ideas.
Other Stuff
It’s been a while since I sent out a newsletter because:
Hooray! This also means that I now have to channel my post-bookwriting energy somewhere. I’d love this newsletter to be it. I’m going to make some tweaks and try to send it out a bit more frequently next year - stay tuned! And please share this newsletter with people you think might be interested, post to social media, etc. Although it is cliche, it really does help a lot when you share this.
Also, let me know in the comments what you’d like to see more of next year!
I would be "interested" in an example of thebfollowing: "And then, I try to come up with new examples of the same principle - to connect a timeless insight to the world today. Then, I spruce it up with language to make it more interesting"