I'm teaching a stats class for master's students in a social science field. I've taught this class several times in the past and I have always dutifully gone through the assumptions tests for t-tests, anova, linear regression, etc. in my lectures and made students do so for the homeworks. However, I'm thinking of just leaving them on the cutting room floor this time.
A few reasons:
(1) These students mostly don't have a stats background and so this is all new to them. The course is compressed into seven weeks, so I'm throwing a lot of information at them in a short time. They're sticking with me so far, but every new wrinkle complicates things further for them. Assumptions testing is a major wrinkle that frustrates them and tunes some out completely.
(2) These students are, by and large, not going to be statisticians - they certainly won't have the skills to be statisticians based on this class alone. The course is more to shore up their data literacy. So the takeaway should, perhaps, not be a technical explanation of how to do every statistically technique perfectly, but an understanding for what the techniques are, how they are done, and what the results mean.
(3) Often, students don't really understand the assumptions or how to deal with them and the ones that do follow it tend to go overboard in the homework testing them (i.e. transforming all their variables).
So, what do you all think? Would I be doing a disservice by leaving aside assumptions tests or am I doing more harm than good by emphasizing them?
Given your description of the students, I would focus on understanding of core concepts and how to interpret results.
I do think understanding assumptions is important, but with the advent of robust estimators, normality assumptions are less important (though they should probably understand the basic concept of normality and when a robust estimator is needed at least).
I'd be more concerned that they understand how to look at data (i.e., don't go blindly running tests without first looking at distributions and scatter plots), and pick an appropriate test for their hypotheses and the nature of the data.
I'd also emphasize good data and analysis practices (e.g., no HARKing, p-hacking, etc).
I think I'd provide at least one example where not testing the assumptions really backfires, but then focus on why that is both a rarity, and now less needed citing the reasons above that Puget mentioned. And , as always, I'd encourage social scientists to consult with a statistician when designing a study.
i would have them calculate *some examples* to get a sense of what the calculation "looks like." A problem with the physical sciences is that students are so comfortable with calculations, that they just sort of blindly do them. I doubt very many understand the underlying concepts. This was a bigger deal in experimental physics some years ago. Maybe its my school, but I feel there's just too emphasis on "Just go measure something. Figure out the stats later, if there is a later"
Is the goal to teach them how to do the tests (in which case 7 weeks is a joke), or how to read the papers that do the tests? If the latter, then sure, leave out assumptions. If the former, then maybe you can get away with leaving them out, but hammer into their heads in a big way 'if you plan to actually use this, come talk to me first about assumptions testing.'
I think you can split the difference and say "all of these tests have assumptions and limitations, but for the purposes of this class we will assume the selected analysis for appropriate." Maybe put in one or two really obvious examples of a poorly designed analysis. If the students need to know more later, they are clever enough to learn the expansions after they have a good foundation.
I think that we teach stats to a similar type of student, though I don't attempt in 7 weeks (Eek!). I try to walk the middle path as some are suggesting here. I do discuss the assumptions and why they are important, and we walk through examples in class checking for them and discuss what to do when they are broken. On homework/exams, I ask questions about assumption concepts, but if they are running analysis I often tell them that the assumptions are met--the data sets are created to meet them. My goals are generally that for each test, students know what type of question/hypothesis would be appropriate, what the data should look like, how to run the test (we use SPSS pull down menus, not teaching syntax), do basic interpretation, and write up the main result in disciplinary style. I try to show them accessible articles in the field that used the statistic. And we also added in time to discuss stats they may be more likely to use professionally (for my students, that is things like reliable change, clinically significant change, etc).
Thanks everyone. Just to be clear, I don't mean ignoring any and all assumptions. We've already talked a lot about the purpose of random sampling for surveys and random assignment for experimental designs. But having these students use scarce time, energy, and patience understanding and identifying heteroscedasticity, for example, may be doing more harm than good, especially since all that effort doesn't seem to be amounting to a clear understanding of the concept.
Quote from: Sun_Worshiper on April 01, 2023, 11:23:34 AM
Thanks everyone. Just to be clear, I don't mean ignoring any and all assumptions. We've already talked a lot about the purpose of random sampling for surveys and random assignment for experimental designs. But having these students use scarce time, energy, and patience understanding and identifying heteroscedasticity, for example, may be doing more harm than good, especially since all that effort doesn't seem to be amounting to a clear understanding of the concept.
I always mention heteroscedasticity, but that's just because it's one my most favoritest words.
Update: I have decided to not talk much about assumptions in class, but to make a power point with assumptions for each test available to the students. This way, interested students who can absorb the extra nuance have those resources (which I'm happy to discuss with them in office hours), but the other students don't get bogged down in something the aren't really able to understand.
I have spent countless hours having performing damage control on administrators and staff wonks who know just enough statistics to screw up their use of statistics in the workplace.
Please, please, teach your students the basic assumptions. And then assess the students on their understanding of those assumptions.
Basically, Do what OneMoreYear said.
Quote from: Sun_Worshiper on April 05, 2023, 01:57:46 PM
Update: I have decided to not talk much about assumptions in class, but to make a power point with assumptions for each test available to the students. This way, interested students who can absorb the extra nuance have those resources (which I'm happy to discuss with them in office hours), but the other students don't get bogged down in something the aren't really able to understand.
It's probably also a good idea to demonstrate what can go wrong if you blindly apply a statistical test or measure when the assumptions are violated, to drive home the point that the assumptions matter to the validity of the test.
Quote from: Aster on April 07, 2023, 07:43:10 AM
I have spent countless hours having performing damage control on administrators and staff wonks who know just enough statistics to screw up their use of statistics in the workplace.
How many times have the results of some "study" been reported in the media where it's glaringly obvious to anyone who understands statistics that there's at least one possible confounding variable that could make the given interpretation
entirely meaningless if it hasn't been addressed? And how often is such a "result" used by politicians to take some sort of action which could actually be not just useless but
completely counterproductive?
I've taken two graduate level statistics courses designed for non-statisticians, but for people who needed to know basic statistics. We were taught assumptions, how to test for them, and when to exclude/ignore them. Very important to know when to use parametric vs non-parametric tests, and whether they've been used appropriately in studies that are read. Also important to understand p-values and effect sizes. Not everything that is statistically significant is actually significant in the real world.
Quote from: marshwiggle on April 08, 2023, 09:05:58 AM
Quote from: Aster on April 07, 2023, 07:43:10 AM
I have spent countless hours having performing damage control on administrators and staff wonks who know just enough statistics to screw up their use of statistics in the workplace.
How many times have the results of some "study" been reported in the media where it's glaringly obvious to anyone who understands statistics that there's at least one possible confounding variable that could make the given interpretation entirely meaningless if it hasn't been addressed? And how often is such a "result" used by politicians to take some sort of action which could actually be not just useless but completely counterproductive?
Yes. Although for more immediate workplace angst, I'd replace "politician" with Dean, Provost, College President, Staff, etc...
Quote from: mleok on April 07, 2023, 03:24:58 PM
Quote from: Sun_Worshiper on April 05, 2023, 01:57:46 PM
Update: I have decided to not talk much about assumptions in class, but to make a power point with assumptions for each test available to the students. This way, interested students who can absorb the extra nuance have those resources (which I'm happy to discuss with them in office hours), but the other students don't get bogged down in something the aren't really able to understand.
It's probably also a good idea to demonstrate what can go wrong if you blindly apply a statistical test or measure when the assumptions are violated, to drive home the point that the assumptions matter to the validity of the test.
Doing only tests for which the data sets sufficiently meet the assumptions is so important. Grad students make this mistake more than any other. I like the idea of having the students do (or see) an analysis where the data come close enough, realistically imperfect compared to a similar one where the data stray from the assumptions enough to give false inferences.
How about not teaching NHST at all? Personally, I think the assumptions are more important than the test. The tests invite binary thinking (always wrong) and divert attention from what can and cannot be learned from the data. Assumptions are part of that picture, but probably a minor part. Most problems with studies are not that the assumptions have been violated, but that the data measurements (data quality) are poor and/or the data has been improperly filtered/selected/aggregated. So, students should be aware that there are assumptions, but there are so many more important things they need to understand.
Quote from: dlehman on April 22, 2023, 05:41:16 AM
How about not teaching NHST at all? Personally, I think the assumptions are more important than the test. The tests invite binary thinking (always wrong) and divert attention from what can and cannot be learned from the data. Assumptions are part of that picture, but probably a minor part. Most problems with studies are not that the assumptions have been violated, but that the data measurements (data quality) are poor and/or the data has been improperly filtered/selected/aggregated. So, students should be aware that there are assumptions, but there are so many more important things they need to understand.
The binary thinking is a killer. Completely agree that it prevents interpretation (and even collection of) informative data.
I would include data quality and appropriateness of filtration among the assumptions one wants to be aware of. E.g. can we assume the sampling error is a lot smaller than the natural variation? Can we assume the outlier was a data-collection error? My (biology) grad students have a lot of trouble developing objective approaches to answering even such fundamental questions. I think it is well worth my effort to give them a better foundation.