Colleges righteously gaming the SAT

[smallcleanrat]

I have to laugh that the SAT was supposed to be the "great equalizer" that allowed a bright kid without social connection or wealth to compete for admission to elite institutions based on intellectual ability but is now seen as impediment to desired social outcomes when results are not even among groups. Gee, the test (and all others such as GRE, LSAT, MCAT, etc) must be biased. We refuse to admit as a society that anything can be objectively quantified, that facts don't care about feelings and that no matter how you slice it, 50% of children are below average.

Since it's behind a wall, I don't know the specifics of what this particular article is driving at, but I have read numerous articles over the last several years arguing for dropping requirements for applicants to submit SAT or GRE scores.

I don't personally recall ever seeing an argument of the form: SAT scores provide an objective measure of intellectual ability.  The higher average SAT scores of students from wealthier backgrounds is has made the superior intelligence of children raised in families with higher-than-average incomes a marked advantage when it comes to accessing higher education. Therefore, to serve the goal of attaining equality of social outcomes regardless of merit, this particular means for superior students to show their superior intellect needs to be dropped so students with middling intellects aren't excluded from higher education.

I can't prove you wrong if you make the claim that this is what many people secretly think while hiding their true intent behind less direct, more "euphemistic" language like calling the tests "culturally-biased." I'm not sure I'd even call the claim implausible. I'm saying I haven't personally encountered this explicit argument.

The arguments I've seen have all boiled down to saying there is reason to believe that standardized test scores are not as useful for identifying academic skill or aptitude as some people assume. I haven't done enough of a deep dive to feel confident in personally judging the validity of these claims. I am simply saying that these are what I understand to be the most commonly used claims related to this argument:

1) High SAT scores actually correlate rather weakly to objective measures like high college GPA or likelihood of completing the degree, and this makes the justification for how heavily they have been emphasized as part of the admissions process questionable. It goes beyond saying that SAT scores are imperfect indicators of intelligence or aptitude to saying they are *poor* indicators, and far from the best available.

2) Test-specific tutoring and cram schools can help students increase their standardized test scores without necessarily helping them hone any academic skills or intellectual talents beyond how to be good at taking standardized tests. (e.g. gaining familiarity with the structure and scoring system of the test to make strategic decisions about skipping questions; extensive drilling on questions of the format used on such tests; rote memorization of formulas to cut down on the need to think when the test clock is ticking)

I think the reason people bring this up is to encourage skepticism about using standardized tests as indicators of aptitude (as opposed to being an indicator of the acquisition of test-taking skills). If you are skeptical of the notion that innate aptitude can be substantially improved by a cram course, then it makes sense to also be skeptical of how much an SAT score can really tell you about a student's potential.

3) It *is* possible for tests to be culturally-biased, and being aware of this possibility can help to mitigate errors of interpretation (e.g. some early IQ tests including questions requiring knowledge of the rules of tennis; someone pointing out that questions like this may muddy the ability to draw accurate conclusions about the test-taker's intelligence can be doing so purely out of a desire for sound methodology). I don't even know if it's possible in most cases for any type of human evaluation to be completely unbiased. Just think it's worth pointing out that "wokeness" for its own sake isn't the *only* possible motivation for exploring whether an established system could do with some reform.

[marshwiggle]

The arguments I've seen have all boiled down to saying there is reason to believe that standardized test scores are not as useful for identifying academic skill or aptitude as some people assume. I haven't done enough of a deep dive to feel confident in personally judging the validity of these claims. I am simply saying that these are what I understand to be the most commonly used claims related to this argument:

1) High SAT scores actually correlate rather weakly to objective measures like high college GPA or likelihood of completing the degree, and this makes the justification for how heavily they have been emphasized as part of the admissions process questionable. It goes beyond saying that SAT scores are imperfect indicators of intelligence or aptitude to saying they are *poor* indicators, and far from the best available.

One thing I've heard related to this is that there's a threshhold effect. (I've seen it linked to high school averages as predictors of university performance, but the same principle would apply.) For a very selective place, only accepting students above the 95th percentile doesn't show any improvement in performance over accepting students above the 90th percentile. The reason is that student success is affected by emotional maturity, study habits, etc. which aren't reflected in that number. So above some score, (such as the 90th percentile), anyone will have the academic capability to succeed, but how they actually perform will depend on a lot of other factors that the tests don't (and can't) measure.

In that case, the test is still valuable as a filter to eliminate students who are highly unlikely to succeed, even if they can't guarantee which students will succeed.

[Hibush]

One thing I've heard related to this is that there's a threshhold effect. (I've seen it linked to high school averages as predictors of university performance, but the same principle would apply.) For a very selective place, only accepting students above the 95th percentile doesn't show any improvement in performance over accepting students above the 90th percentile. The reason is that student success is affected by emotional maturity, study habits, etc. which aren't reflected in that number. So above some score, (such as the 90th percentile), anyone will have the academic capability to succeed, but how they actually perform will depend on a lot of other factors that the tests don't (and can't) measure.

In that case, the test is still valuable as a filter to eliminate students who are highly unlikely to succeed, even if they can't guarantee which students will succeed.

In this selective-school scenario, the test does serve to identify students from (expensive private) high schools that give out a "Gentleman's  A", which makes the GPA a poor predictor.

In order to experimentally test the utility of the SAT for this situation, selective colleges would have to admit students who have good grades but terrible SAT scores, and then see how they do. I'm not holding my breath!

Colleges currently admitting mostly the top 95th %ile also can't game their rankings by going test optional. They already have all the points available for that measure. They can still make the (empty) claim that they are lowering the barrier to entry for underrepresented groups.

[dismalist]

The latest from the College Board

https://collegereadiness.collegeboard.org/pdf/national-sat-validity-study.pdf

Correlation between SAT and first year GPA is ca. 0.5. That is not whelming.

[Hibush]

The latest from the College Board

https://collegereadiness.collegeboard.org/pdf/national-sat-validity-study.pdf

Correlation between SAT and first year GPA is ca. 0.5. That is not whelming.

Check put figure 3 in that report. Some students with a High school GPA (>3.7) don't do well in college (GPA<2.5). The SAT does a good job identifying them. That's the situation described in the post above.

And that is after the association is weakened by the fact that student with the high HSGPA/low SAT are likely to be admitted to colleges with easier grading.

[marshwiggle]

The latest from the College Board

https://collegereadiness.collegeboard.org/pdf/national-sat-validity-study.pdf

Correlation between SAT and first year GPA is ca. 0.5. That is not whelming.

Well, if it was close to 1 it would mean all of us would be out of work, since we'd be entirely irrelevant.

[dismalist]

Well, if it was close to 1 it would mean all of us would be out of work, since we'd be entirely irrelevant.

Nah, only if college were irrelevant! :-)

[kaysixteen]

I get that there are now and have always been big problems with the SAT, but they are necessary nowadays, because HS GPAs cannot really be used as a legitimate way for college admissions professionals to differentiate amongst applicants.   There is so much grade inflation, nowadays even course title inflation, across hss nationwide, AND many hss are, ahem, well... not so good.

[financeguy]

Pointing out that there is a flaw in the SAT does not invalidate my argument. Like all human endeavors, this is possible and even easy to do. Show me something better than standardized tests in terms of uniform application among a broad pool of candidates without otherwise easy to compare backgrounds that does so in a way that it has greater predictive validity.

[spork]

Zip code. Or household income.

[marshwiggle]

Zip code. Or household income.

At that rate, why not cut to the chase and just mail degrees to the correct addresses and not waste several years and thousands of dollars? The correlations are probably similar. As I said above, if the correlation between any metric of incoming ability and performance were very high, it would mean institution, program, instructor quality, etc. are basically irrelevant.

There are negative filters and positive filters, and both are useful. Being able to eliminate people who are likely to be unsuccessful is very useful, even if it's harder to guarantee who will be successful. Perhaps a "grit" assessment of people with high SAT scores would be a worthwhile addition for that purpose.

[financeguy]

We already do (by proxy) admit based on income in two ways, legacy admission and self selection based on lack of ability to afford tuition. I actually don't have a problem with private schools using legacy so long as they are not accepting federal dollars. Of course these restrictions will never be imposed so it's totally theoretical.

[Hibush]

We already do (by proxy) admit based on income in two ways, legacy admission and self selection based on lack of ability to afford tuition. I actually don't have a problem with private schools using legacy so long as they are not accepting federal dollars. Of course these restrictions will never be imposed so it's totally theoretical.

What is the cultural context of legacies at your college?

This is something that varies a great deal among schools, so I'm curious about some of the nuances.

Is is it a family tradition in some groups to attend? Does that mean they prime coming generations to want to attend? If so, that is good for the school in that the yield is higher from legacies. Admitting students who are more likely to enroll strikes me as a legitimate selection criterion. How much weight should that get?

Is it a family tradition to attend, and to pay full tuition?
Is it a family tradition to attend, and to donate generously to the annual fund after graduating?
Is it a family tradition to attend, and to have your grandchildren learn from the XXX Family Endowed Professor, or to learn in the XXX Family Lecture Hall (made possible by seven or eight figure donations)?

These expectations of the legacy effect must influence how a college thinks about how to estimate the financial impact of making and admission offer.

[dismalist]

There are many colleges. Each college can decide what kinds of students it wishes to admit, and whether to use the SAT or not.

If the college in question is public, voters must indeed have the opportunity to determine what admissions criteria are used.

My original citation of the College Board study, about which I could say a lot more, was merely intended to illustrate that there is one hell of a lot of chance left in any of these decisions, so that we should not obsess about them.

[mamselle]

Is it a family tradition to send all the kids to good prep schools where they are both possibly socialized and perhaps educated to the norms, and maybe even the scholastic standards of the target schools?

It starts in kindergarten.

M.