Ever read some new scientific study and thought, “Can I trust this?” Well, you’re not alone. With all of the research floating around, it’s kind of tricky to know what’s legit and what’s not. Need a guide on how to single out the reliable studies? Here it is.
If It Seems Too Good to Be True, It Probably Is
Where best to begin? Well, with the obvious perhaps: if a finding seems too wild to be true, it probably is. As psychologist Christopher Chabris mentioned in a blog posting, researchers Uri Simonsohn, Leif Nelson, and Joseph Simmons showed that some effects are so ridiculous they just cannot exist. They even fudged data to indicate that listening to the Beatles’ song “When I’m Sixty-Four” would make a person grow younger! When you come across a study that has you going, “Huh?”, run it by your grandma. If she’d raise an eyebrow, the study might not be trustworthy.
Let Ideas Age a Bit
New studies can be interesting, but they may not have withstood the test of time. Research that was published more than 20 years ago may be superseded, and brand-new studies may not yet have been replicated. The sweet spot? Research that is new enough to be relevant but also old enough to have been picked over by others. File new research away in your mental file cabinet, and see whether it has staying power.
Useful Beats Clever
All too often, it seems as though novelty gets more attention in science than any practical benefit. Indeed, research derives its actual value from the utility it offers. Research might catch the clever headlines, generally those containing insights of use more sound. If the study answers real-world problems, the chances are that it is based upon solid research.
How Research Works
This will help in assessing the credibility of the study, and for that, a little information about how research studies are conducted is necessary. Steps many researchers follow are as follows:
• Identify the Problem: The researchers pick up some problems or questions.
• Survey Literature: Conduct background reading to determine what is currently known about this topic.
• Develop Hypotheses: Based on what’s learned, they develop hypotheses that will guide the research study.
• Research Design: Hence, a fine plan regarding the collection of data is required.
• Describe the Population: They decide whom or what they are going to research.
• Collection of Data: The data is collected for reliable and valid data.
• Analysis of Data: In this phase, researchers draw some conclusions by analyzing the data.
• Writing Report: The final step is they prepare a report about the findings.
Check the Methods and Data of the Study
Good research should have an open and transparent methodology. Studies that do not describe their methods clearly are suspicious. Besides this fact, consider the strength of data collection processes. Example: Was it a large enough sample size? Were the participants diverse?
Peer Review and Publication
Research published in good, peer review journals has been critiqued by other experts. While that absolutely does not mean the science is perfect it is a good indicator of the validity of a study. Avoid research in questionable or non-peer-reviewed journals.
Think about who funded the research
Who provided the funding for this research? With the proliferation of today’s findings sponsored by business organizations, any research sponsored by an organization that has something to gain or lose by its outcome should raise suspicions. For biased funding might taint results, and there’s invariably always something to look for in the way of conflict of interest.
Personal Testimonials and Skepticism
Worth mentioning is that personal stories or experiences can yield valuable insights, but should be taken with a grain of salt. Anecdotal proof does not replace well-conducted research; however, at times, it may point to areas that are really ripe for further study.
The Bottom Line
It’s often hard to separate bunk from solid research, but not impossible. These rules of testing extraordinary claims, ageing ideas, valuing utility over novelty, remaining aware of the research process, methods used and data checked, consideration of peer review, looking into funding these can render you a more critical consumer of scientific research. But importantly, as you move through the heterogenous world of scientific research, remember that the best weapon you have is precisely that which is held by research scientists themselves: a healthy dose of skepticism.
