Teaching research methods - tips and pitfalls

[Sun_Worshiper]

Dear Forumites,

I'm teaching a social science research methods course later this semester, and since this is a class that students generally don't care for (apparently it is difficult and boring), I thought I would share a few tricks that I use to liven it up, and also ask for any ideas that you all might have.

The course is about 75% quantitative methods and about 25% qualitative. We use statistical software for the quant stuff, but no software for the qual. Students are master's level.

Sun-Worshiper's tips for teaching methods:

• Show the math, to make sure that they understand it, but don't let them lose sight of the purpose of the math (which is to conduct statistical analysis)
• When talking about variables, use examples from the other classes students have taken
• Say that correlation is not causality, but also show it using both serious and funny examples
• Make students actually do some qualitative methods (e.g. an interview out of class or focus groups in class)
• Make students do quantitative research with real data
• Talk about (and show) p-hacking and similar ethical issues in research
• Show how qual and quant methods complement each other

Other tips and ideas?

[Parasaurolophus]

Don't forget HARKing!

One possible fun source of examples is the Twitter account 'justsayinmice', which does a great job of pointing up the disparity between actual results and how they get reported in the press.

It sounds like you're already doing this sort of thing, but maybe one instructive exercise for them would be to be given some published (and hopefully retracted!) papers whose methods, upon inspection, turn out to be kind of poor, and critique them. It's one thing to see the prof pick out examples for class, but it's another to have to do it yourself when you're staring at a whole article.

[research_prof]

Talk to them about how to properly cite other researchers' work (no kidding).

[Hibush]

Don't forget HARKing!

One possible fun source of examples is the Twitter account 'justsayinmice', which does a great job of pointing up the disparity between actual results and how they get reported in the press.

It sounds like you're already doing this sort of thing, but maybe one instructive exercise for them would be to be given some published (and hopefully retracted!) papers whose methods, upon inspection, turn out to be kind of poor, and critique them. It's one thing to see the prof pick out examples for class, but it's another to have to do it yourself when you're staring at a whole article.

+1. This is a fantastic exercise and so crucial for making them realize this is fundamental stuff. So at odds with many people's habit or instinct that the press stories turn the logic backwards in order for people to buy it.

Vedantham's "Hidden Brain" is good for this as well. The experiments are usually quite simple, so you can test whether the assumptions are true for the specific case, and also whether the assumptions underlying the generalization apply. Vedantham picks relatable examples that are mostly sound.

Just today at a virtual conference, I heard a speaker answer a question by saying "Well, I didn't talk about that because the results weren't really supporting our hypothesis."  Yikes!

[Golazo]

Make sure students can actually connect the dots for both quant and qual. It is one thing to be able to do the regression or the within-case comparison and another to be able to clearly explain how it fits with the rest of the student's research project, and the field more broadly. There are way too many masters students who can't really do this.

[Puget]

This has a nifty p-hacking demo: https://fivethirtyeight.com/features/science-isnt-broken/#part1 (scroll down a bit for the interactive)
I used it with our honors thesis seminar the other day and I think it really helped them grasp the idea of researcher degrees of freedom (which for some reason seems hard for them to grasp, maybe because they don't yet understand stats well enough to understand there are multiple ways of doing things and you have to make choices).

This is a good video on a lot of the basics of the replication crisis and what to do about it:
https://www.youtube.com/watch?v=42QuXLucH3Q

I also like this article:
https://journals.sagepub.com/doi/full/10.1177/2515245918754485

[polly_mer]

Make sure students can actually connect the dots for both quant and qual. It is one thing to be able to do the regression or the within-case comparison and another to be able to clearly explain how it fits with the rest of the student's research project, and the field more broadly. There are way too many masters students who can't really do this.

Definitely this.

Pattern identification is important and mere numbers aren't the same as good answers to research questions.

[marshwiggle]

This one has a lot of cool examples of statistics used to answer understandable questions.

https://onlinestatbook.com/case_studies_rvls/index.html

[mamselle]

If they're ever going to need to travel (when that becomes possible) to distant libraries for sources, have a session on etiquette in foreign archives, and in foreign places, generally.

Likewise if their fieldwork will be abroad, at least mention the courtesy of considering some language study, etc.

I realize those may still be somewhat remote at present, but research is also about having an open mind and heart to other cultures and people.

And how you get along makes a huge difference in peoples' willingness to be helpful and supportive of your work.

I've seen good and bad examples of both, and the results of each; while quant and qual analysis is definitely important, so is learning how to be a human being with other human beings.

Maybe some values learning, or case studies on situational decision-making would be useful in that regard.

M. 

[Sun_Worshiper]

Good suggestions! Keep them coming.

Here are some fun spurious correlations that students get a kick out of: https://www.tylervigen.com/spurious-correlations

[ergative]

Good suggestions! Keep them coming.

Here are some fun spurious correlations that students get a kick out of: https://www.tylervigen.com/spurious-correlations

I use that site all the time!

[Cheerful]

Good suggestions! Keep them coming.

Here are some fun spurious correlations that students get a kick out of: https://www.tylervigen.com/spurious-correlations

That first graph about suicides is insensitive, offensive.  Too bad the creator selected such an unfortunate, unpleasant, painful example.

[marshwiggle]

Good suggestions! Keep them coming.

Here are some fun spurious correlations that students get a kick out of: https://www.tylervigen.com/spurious-correlations

That first graph about suicides is insensitive, offensive.  Too bad the creator selected such an unfortunate, unpleasant, painful example.

Actually several are related to death. My guess is that it is partly intentional to illustrate the preposterousness of considering that other random factors that happen to correlate are actually responsible. It's quite possible that some people would see two "variables" of which neither was particularly familiar to them and be able to believe that the connection was causal. By picking one variable which was very easy for people to grasp and very dramatic, it's easier to show people that the correlation is coincidental.

[dismalist]

One gets a spurious correlation between any two variables that exhibit a time trend. Unless the examples are used to motivate time-series analysis, they are not helpful.

[apl68]

Good suggestions! Keep them coming.

Here are some fun spurious correlations that students get a kick out of: https://www.tylervigen.com/spurious-correlations

So...by producing too many PhDs in mathematics we're increasing the amount of nuclear waste stored at nuclear plants?  Looks like the recommended course of action is clear!