Testing the Evidence
|
Subscribe to the Three Star Leadership Blog |
|
|
![]() |
|
|
Contact Wally about coaching, consulting, or speaking to your group. |
Jeffrey Pfeffer and Robert Sutton had an excellent piece in the NY Times titled "Trust the Evidence, Not Your Instincts" which began this way.
"Consider this hypothetical situation: You have a serious illness. Your doctor prescribes an intrusive, painful and costly treatment. What she doesn’t say — because she hasn’t consulted the research — is that most studies find the treatment ineffective and fraught with negative side effects."
That's a provocative beginning to an article calling for bosses to "consult the evidence." I applaud the objective, but if you're a boss, you have a problem that the hypothetical physician doesn't have. You have to figure out which evidence to follow.
It ought to be easy. After all, over ten thousand business books are published every year. There are also textbooks and articles of all kinds, both popular and academic. And let's not forget blog posts and tweets and newsletters. What's a boss to do?
First find a good source, and then look for evidence. Bob Sutton is a primary source for me because I like the way he explains the research findings he shares and the way he connects them to the world I live in. A book he wrote with Pfeffer, Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management, and his solo book, Good Boss, Bad Boss are both excellent.
Read more than the headline. Headline writers are not rewarded for being accurate. They are rewarded for increasing readership. Sometimes you may have to read the research the article is based on.
Last month there were many headlines about how "mean" bosses get paid more. Jim Morgan (another favorite source) noticed one of them with the headline: "Life isn't fair: Play nice, get paid less." Jim read the article which used the terms "jerk" and "rude" to describe well-rewarded bosses.
Then he dug into the research and discovered that the actual findings were far more nuanced and less provocative than the article or headline implied. Read his full report here.
Be skeptical. Most of the findings that seem counterintuitive turn out to be wrong. Reports on research should be suspect if they're used to sell something. Even if the research is solid, the reporting may be selective and biased. You have to investigate to know.
If a finding is important to you, check out the methodology used in the research. Was it observational or survey research? Who was studied? In what circumstances? What questions were asked? One reason I read the Evidence Soup blog is for the tips on evaluating evidence.
Do your own experiments. If a research finding makes sense to you and suggests you should do things differently, try it. Note the results.
Boss's Bottom Line
It's a good idea to base your practice and decisions on the evidence, as long as the evidence is good.
Additional Resource
After the NY Times article appeared, Bob Sutton posted "Our New York Times Piece on Evidence-Based Management: The Uncut
Version" on his blog.
Wally's Working Supervisor's Support Kit is a collection of information and tools to help working supervisors do a better job. It's based on what Wally's learned in over twenty years of supervisory skills training. Click here to check it out.




Great post. A subject right up my alley.
Reply to this
Yes it is, David. One could say that you are a professional evidence tester. Thanks for stopping by.
Reply to this