Colleges righteously gaming the SAT

[Hibush]

Is it a family tradition to send all the kids to good prep schools where they are both possibly socialized and perhaps educated to the norms, and maybe even the scholastic standards of the target schools?

It starts in kindergarten.

M.

That background certainly help check the  boxes for getting into any elite college, whether one is a legacy or not.

Two colleges with very high legacy enrollment, far higher than the Ivys, are Notre Dame and Baylor. Both are fine schools, with lots of tradition. No prep school is needed to to meet the academic requirements for admission. What role do legacies play in their strategy of engagement, yield, finances and gaming rankings?  [They are in the range where those could be gamed, just to touch base with the original topic.]  Does the preference serve to exclude people from the positions of influence in the Catholic and Southern Baptist power structures that are accessible to alumni of these schools?

Then in a different context altogether, College of the Ozarks, committed to serving a regional population and the local identity weighs legacies in admission decisions. It is more selective than Notre Dame. As far as I know, concerns have not been raised about their practices. What would those concerns be.

[financeguy]

My own institution is not allowed to preference legacy due to state law as a public institution. I highly recommend that everyone check out The Price of Admission which goes over this phenomenon and names names/institutions. I don't see how anyone defends the practice at institutions receiving money from the general tax base. Want to be a club? Go for it, but on your own dime.

[dismalist]

From SAT's to legacies.

Let the colleges do whatever they please, on their own dime, but voters, not us, must be able to sanction State supported schools' admissions standards.

[marshwiggle]

Is it a family tradition to send all the kids to good prep schools where they are both possibly socialized and perhaps educated to the norms, and maybe even the scholastic standards of the target schools?

It starts in kindergarten.

M.

That background certainly help check the  boxes for getting into any elite college, whether one is a legacy or not.

Two colleges with very high legacy enrollment, far higher than the Ivys, are Notre Dame and Baylor. Both are fine schools, with lots of tradition. No prep school is needed to to meet the academic requirements for admission. What role do legacies play in their strategy of engagement, yield, finances and gaming rankings?  [They are in the range where those could be gamed, just to touch base with the original topic.]  Does the preference serve to exclude people from the positions of influence in the Catholic and Southern Baptist power structures that are accessible to alumni of these schools?

Then in a different context altogether, College of the Ozarks, committed to serving a regional population and the local identity weighs legacies in admission decisions. It is more selective than Notre Dame. As far as I know, concerns have not been raised about their practices. What would those concerns be.

For a country that was founded on the theory that hereditary leadership is bad, the US seems pretty enamored with the idea in practice.

[mamselle]

The idea of an hereditary monarch might never have been on the table, but the discussions between Hamilton and Jefferson, and Adams, and Franklin, et al., suggest that a certain level of capable, wider-seeing, leadership among the supporting ranks--especially those not in need of money, so that their opinions could be formed without reference to graft--was always a consideration.

Obviously, some of those ideals have been sorely tested of late, but the theories behind them are not without merit.

M.

[apl68]

Is it a family tradition to send all the kids to good prep schools where they are both possibly socialized and perhaps educated to the norms, and maybe even the scholastic standards of the target schools?

It starts in kindergarten.

M.

That background certainly help check the  boxes for getting into any elite college, whether one is a legacy or not.

Two colleges with very high legacy enrollment, far higher than the Ivys, are Notre Dame and Baylor. Both are fine schools, with lots of tradition. No prep school is needed to to meet the academic requirements for admission. What role do legacies play in their strategy of engagement, yield, finances and gaming rankings?  [They are in the range where those could be gamed, just to touch base with the original topic.]  Does the preference serve to exclude people from the positions of influence in the Catholic and Southern Baptist power structures that are accessible to alumni of these schools?

Then in a different context altogether, College of the Ozarks, committed to serving a regional population and the local identity weighs legacies in admission decisions. It is more selective than Notre Dame. As far as I know, concerns have not been raised about their practices. What would those concerns be.

For a country that was founded on the theory that hereditary leadership is bad, the US seems pretty enamored with the idea in practice.

Societies in general have an awfully strong tendency to create elites who control resources.  Their progeny inevitably have something of an advantage in the next generation.  And of course in democratic politics, where publicity and recognition are so important, having a famous name can be an advantage--hence the occasional election of a leader who has some kind of relationship to a previous leader.

[pigou]

We all know that many students game the SAT by taking cram classes that result in a score that is higher than what their general education would support.

This is not gaming the SAT, that's type-revealing. The students who care enough to prepare for the SAT are also the students who will end up doing better in college. Lots of students take the SAT without ever doing a practice test or looking at any of the free books available in libraries everywhere... those students will do worse, holding all else equal, not just on the test, but in college (and probably life) more generally. 90% of everything in life is about preparation.

IIRC, the association between SAT score and first-year college academic success as measured by GPA is pretty weak. Meanwhile a testing industry that is supposedly non-profit pays its executives really high salaries, and you have the test prep companies like Kaplan that enable those who can afford it to boost their scores, belying the College Board's claims of predictive validity for the SAT. This is why I said it's less expensive all around and just as predictive to use zip code.

I might have seen the OLS regression figures on SAT score and college GPA in The Big Test, but it's been a long time since reading that book and the book itself was published many years ago. I assume there is more recent research on this topic.

You can't do any kind of statistical analysis of the effect on SAT score on college GPA when the SAT score itself is used as an admissions decision. That's known as "conditioning on a collider" and effectively is a form of select bias: you observe the really bright students who got accepted despite a weak SAT score and who will do well, but you don't observe the college performance of students who had a weak SAT score and weren't exceptional in other ways (because they didn't get admitted).

[smallcleanrat]

We all know that many students game the SAT by taking cram classes that result in a score that is higher than what their general education would support.

This is not gaming the SAT, that's type-revealing. The students who care enough to prepare for the SAT are also the students who will end up doing better in college. Lots of students take the SAT without ever doing a practice test or looking at any of the free books available in libraries everywhere... those students will do worse, holding all else equal, not just on the test, but in college (and probably life) more generally. 90% of everything in life is about preparation.

I'm not sure about this if it is in reference specifically to test tutoring and cram classes. A kid can be forced by ambitious parents to go to these; it doesn't necessarily show drive and initiative on the kid's part.

Also, grabbing a library test prep book (which I remember being slightly complicated by frequent alterations in standardized test structure making older prep books less helpful) for practice and self-study does show the type of behavior that would carry over to success in college. But cram schools can set the study pace, present a set curriculum, assign practice tests and drills, give feedback...these student are not required to do nearly as much as the self-study student in the way of self-organization and self-motivation because of the level of structure provided externally.

[marshwiggle]

IIRC, the association between SAT score and first-year college academic success as measured by GPA is pretty weak. Meanwhile a testing industry that is supposedly non-profit pays its executives really high salaries, and you have the test prep companies like Kaplan that enable those who can afford it to boost their scores, belying the College Board's claims of predictive validity for the SAT. This is why I said it's less expensive all around and just as predictive to use zip code.

I might have seen the OLS regression figures on SAT score and college GPA in The Big Test, but it's been a long time since reading that book and the book itself was published many years ago. I assume there is more recent research on this topic.

You can't do any kind of statistical analysis of the effect on SAT score on college GPA when the SAT score itself is used as an admissions decision. That's known as "conditioning on a collider" and effectively is a form of select bias: you observe the really bright students who got accepted despite a weak SAT score and who will do well, but you don't observe the college performance of students who had a weak SAT score and weren't exceptional in other ways (because they didn't get admitted).

As I indicated earlier, I'm not sure what the "ideal" correlation would be between SAT score and 1st year GPA. It certainly wouldn't be close to 1, since then it would mean choice of major, work habits, instructor skill, etc. essentially count for nothing.  Has anyone seen any kind of argument for what sort of correlation would be "good" and why?

[apl68]

We all know that many students game the SAT by taking cram classes that result in a score that is higher than what their general education would support.

This is not gaming the SAT, that's type-revealing. The students who care enough to prepare for the SAT are also the students who will end up doing better in college. Lots of students take the SAT without ever doing a practice test or looking at any of the free books available in libraries everywhere... those students will do worse, holding all else equal, not just on the test, but in college (and probably life) more generally. 90% of everything in life is about preparation.

I'm not sure about this if it is in reference specifically to test tutoring and cram classes. A kid can be forced by ambitious parents to go to these; it doesn't necessarily show drive and initiative on the kid's part.

Also, grabbing a library test prep book (which I remember being slightly complicated by frequent alterations in standardized test structure making older prep books less helpful) for practice and self-study does show the type of behavior that would carry over to success in college. But cram schools can set the study pace, present a set curriculum, assign practice tests and drills, give feedback...these student are not required to do nearly as much as the self-study student in the way of self-organization and self-motivation because of the level of structure provided externally.

At any rate, to the extent that cram courses do help students, it is a potential advantage for those who can afford them.  Students with more resources are going to have certain advantages.  It's not an easy issue to fix.  The standardized tests were supposed to BE the fix, for both the advantages of wealth and of caste.  But they didn't fix the problem, at least not completely.

[smallcleanrat]

Pointing out that there is a flaw in the SAT does not invalidate my argument.

Your first post seemed to be making the point the exam is not actually flawed. So pointing out a flaw would invalidate that.

Show me something better than standardized tests in terms of uniform application among a broad pool of candidates without otherwise easy to compare backgrounds that does so in a way that it has greater predictive validity.

Not my field of expertise, so I don't have an informed opinion on which metrics are better or worse.

I'm seeing claims that GPA is a stronger predictor of college success than standardized test scores, despite the issues about how non-standardized high school grading systems are: https://www.forbes.com/sites/nickmorrison/2020/01/29/its-gpas-not-standardized-tests-that-predict-college-success/?sh=49ee75cb32bd

And I haven't got the article(s) in front of me, but I've read the SAT subject tests have better predictive validity than the general SAT but subject test scores are not asked for as much as scores on the general. But I can't remember ever reading an argument to de-emphasize the general SAT in favor of the subject SATs.

[dismalist]

I've read the SAT subject tests have better predictive validity than the general SAT but subject test scores are not asked for as much as scores on the general.

Yeah, I once read, but it was one hell of a long time ago, that any predictive value for success in all college fields, not just quantitative fields, came from the Math portion of the test. The Verbal section merely added noise.

Whatever, there is sufficient looseness about all these things that we should let colleges and universities admit who they please. For State universities, let the voters of each State sign off on admissions standards.

[pigou]

As I indicated earlier, I'm not sure what the "ideal" correlation would be between SAT score and 1st year GPA. It certainly wouldn't be close to 1, since then it would mean choice of major, work habits, instructor skill, etc. essentially count for nothing.  Has anyone seen any kind of argument for what sort of correlation would be "good" and why?

You'd have to control for choice of major (e.g. within a specific major, do students with high SAT score get better grades?) to say anything. Students with low SAT scores will be more likely to end up in majors with easier grading (i.e. not STEM), which is another reason why just correlation isn't very insightful. But even so, there's no "good" correlation, because you cannot interpret the value that comes out of a correlation analysis. This particular phenomenon is known as Berkson's paradox: https://brilliant.org/wiki/berksons-paradox/

The first figure illustrates how, when there's truly a positive correlation between GPA and SAT score, you could see a negative relationship between GPA and SAT among admitted students.

[kaysixteen]

Well, the 'A' in 'SAT' originally stood for 'aptitude', though I believe the Federal Trade Commission forced the College Board to change it to 'achievement' some years back.   But an aptitude test is supposed to test, ahem, aptitude, which is not supposed to be preparable by test prep courses, books, generic studying, etc.

[financeguy]

I have never seen an objective metric of performance such as a standardized test that wasn't critiqued.

In a sales organization someone else got the better leads or better seat for walk in traffic. Actual deals closed didn't tell the whole story.

Numbers is sports were the result of who got the best spot or the other side cheating or the contribution of team mates not "reflected in the numbers."

That requirement in black and white for a job? That makes no sense. I knew a guy once who didn't have that qualification that did really well in that field!

Standardized tests are just one more objective metric that people on both sides of the equation wish to critique. Those subjected to the tests wish to perceive their abilities (based on no evidence) greater than what is reflected in their score. Those on the other side overemphasize the value of their own subjective decision-making over objective data of any kind, often fueled by tangentially related objectives such as race, gender or other group preferences.

The bottom line is that standardized metrics of success are everywhere in the world, including licensing tests for one of every three fields of employment. Early in life is the time to recognize that these tests a) determine opportunities you will have regardless of your opinion of them b) will not have their outcomes changed no matter how many "flaws" you may think they have and c) are always right no matter what for these reasons. This is one area where you (or mom) can't ask to speak to the manager.

Of course a college admissions committee is not (usually, or at least that they would admit) using standardized tests as a blanked yes/no criteria for admission.  If you get a 1000, you know you're much less likely than someone with a 1450 to go to a top 10 institution, but nothing is set in stone. If you think the test "doesn't reflect your true abilities as much as this intangible thingy" then demonstrate whatever that might be. My assumption is that given the option of admitting a student with a 200 point lower SAT score than their average, an institution will be more likely to do so if they come to the table with a patent, published article, Olympic medal, or any other significant accomplishment to support the idea that this is the case.