Podcast Pearls {02}

Stuff You Should Know: 

Research Tips from SYSK

I've said it before, but SYSK is my go-to podcast. When I hear that cheesy elevator swing music, I feel like my ears have come home. I love Josh and Chuck and how they approach the wide variety of topics, and do a great job of being entertaining and looking at both sides of the coin. This podcast about how to gather valuable research as a responsible adult in today's age. Sound boring? Maybe, but it is very valid for us as educated individuals looking for truth in a swamp of half truths and strong wills. Science does not prove any religion. Science does not make you a republican or a democrat. Anyone can find someone and some article to agree with them. Educated adults are colanders who sift through the sources of their information to get the good stuff. You're not a sponge who's supposed to soak up all information until you're saturated with "well researched opinions." These research tips from my buddies Josh and Chuck help us sort through the muck to get to what science is actually saying and what we can safely deduce from it.

Here are some questions to ask yourself when evaluating the weight and value of a study or one of those clickbait articles you see as you scroll through your social media feeds.

• Who / what was studied in this experiment?  I think the most staggering nugget of information from the podcast was that most studies (80% according to a 2010 survey) used what is referred to as WEIRD test subjects. WEIRD stands for those who are Western-Educated from Industrialized, Rich and Democratic countries. What's worse is this demographic makes up only 12% of the world's population! That spells bad news bears when it comes to statistics. How many of our scientific proof is only indicating truth for 12% of the world? How broad of a stroke is the study painting from that information? It's not only misleading, it comes from a deep crevice of privilege perpetuating ignorance in those of us born lucky. 

• How large is the sample size? Depending on what you're trying to prove, but the vast majority of studies done should be chalked up as "preliminary studies" for how small their sample size is. 12 people? nope, can't use those numbers. That doesn't carry any further than information about those 12 people. 

• Is it repeatable? More importantly, has it been repeated? Often you will read about a study that shows such and such correlation. That scientist will get roses thrown at her feet and more plaques for her wall, but the scientist who retests it to confirm in a study independent of the first gets no funding and no glamour shots. With such a fast paced technology and engineering driven world, we want answers immediately. We pay extremely intelligent people to get us answers fast. Unfortunately, science is not fast. Clinical studies and lab research  take years and years to come to a conclusion that can hold water. Without repeated testing, there is a 5% chance that any study will show a correlation just due to random chance. The media has a tendency to take and run with studies, and that 5% is a margin I am not comfortable with.
• Who is funding this study? Confirmation bias is a real thing, and if you have a multi-billion dollar company signing your checks, you have a bit of pressure to give complimentary results. Scientists will through out results and call them "outliers," but science is not something you can treat like a buffet. You have to take those nasty kalamata olives of test results in your salad whether it leaves a bad taste in your mouth or not. 

I worked as a research assistant in college, and we had a course all about how to read scientific articles from academic journals. For the most part, you only read the abstract, sometimes the intro, and the conclusion to get an idea of what is in the journal. You only look further at the rest of the chicken scratch if you're trying to repeat the study or if you want to tear it apart. My professor gave me this snarky handout to help interpret the scientific muck:

And admittedly, when it came time for me to write my own academic paper on my research, I followed some of these same guidelines. I can't tell you how many drafts I re-wrote due to my supervisor telling me to spice it up with bigger words to prove my intelligence--whether I was saying anything or not. 

There are major issues with how scientific research is funded and performed today. For example, if your study is non-conclusive or if you didn't get the results you wanted, then it is really difficult to publish. This leaves hundreds of other scientists trying to scramble to do the same thing as you, but coming up with the same result. This type of competitiveness is wasting crucial, much-needed academic money and the time of some brilliant scientists around the world.

My good friend John Oliver also has the hard and fast (and a wee bit explicit) version of what Josh and Chuck have to say too.

Please please listen to this podcast! It's great. If you can eat this stuff up, SYSK has another episode about The Scientific Method that acts as a good companion to this one. 

Remember, you're a colander, not a sponge. 

Do your homework and be an educated adult.