What do you folks think about the adversity score being added to the SAT? The commentariat at IHE pokes a lot of holes in the overall notion:
https://www.insidehighered.com/admissions/article/2019/05/17/college-board-will-add-adversity-score-everyone-taking-sat
I'll be interested in reading what you all think, especially since so many of you are from/working in the US. To my mind, the whole test seems like for-profit bunkum with minimal real value, and this latest move sounds like a desperate stab to make it seem relevant. But maybe I'm wrong, and the new score does actually help to ensure "fairer" outcomes?
Also, I think it's telling that they're keeping the adversity scores secret. Surely that's the right business decision, but it doesn't inspire much confidence.
Actually, The Atlantic just had an interesting article on that as well.
It was an online article, in which the author, raised in adverse conditions, suggested that it did no service to those who were defined as more oppressed. (Can't link from this phone, sorry).
I can follow that idea, that such an effort might not not habilitate so much as humiliate students in such a situation.
But it's also true that many potentially capable individuals are indeed handicapped by elements in their own background not of their own choosing or cause.
Could students possibly "opt in" or "opt out," choosing to have the scale applied to them, or not?
Or would that be open to abuse by the more cynical?
Addressing inherent inequalities in the current system is a step forward, but it may want further consideration, or more trialling, to discover alternate solutions with better potential outcomes.
M.
My take? This is an attempt by the College Board to regain some of the business it has lost from universities making the SAT optional. The process of dropping SAT scores as an application requirement seems to have started in large part with the publication of The Big Test: The Secret History of the American Meritocracy, by Nicholas Lemann, in 1999 -- which has, in my opinion, a fascinating analysis of the role standardized testing has played in U.S. higher education.
I once attended a presentation about the biases reflected by the SAT. The speaker said college applicants' zip codes were a good proxy for SAT scores and probably had a stronger correlation with college academic performance than SAT scores did.
I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,
Quote from: eigen on May 19, 2019, 05:31:56 AM
I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,
So what do we use instead? I've spent a lot of time at nearly open enrollment institutions and have thus encountered a fair number of students who had transcripts indicating excellent performance in high school, but were not ready for college.
A colleague called it the zip code effect where some of these high schools were doing so little education that:
- A C meant "Handed in some work at some point and didn't contribute to an assault on a teacher that resulted in hospitalization".
A B meant "Handed in at least half the work and attended class regularly, probably without assaulting the teacher at all".
An A meant "Did everything that was asked and was moderately respectful of the teacher".
I dealt with tears every term as I had to explain to the diligent A students who came from terrible high schools that they were so underprepared that the hardest they've ever worked will be at best a C in the class. I handed over the tissue box regularly as I explained that the local performance indicated needing remedial work in math, despite that A in high school calculus that seemed based more on attendance and willingness to try than demonstrated ability to solve problems.
The standardized tests generally aren't all that useful in discerning the difference between good, great, and excellent for those who score pretty well, but a poor score can indeed indicate areas that need improvement even when the transcript indicates the subject matter has been mastered.
Quote from: mamselle on May 18, 2019, 04:17:25 PM
Actually, The Atlantic just had an interesting article on that as well.
It was an online article, in which the author, raised in adverse conditions, suggested that it did no service to those who were defined as more oppressed. (Can't link from this phone, sorry).
One recent
The Atlantic article is https://www.theatlantic.com/education/archive/2019/05/explaining-the-college-boards-new-adversity-scores/589708/, but that's a pretty positive spin from the CEO of the company that creates the SAT.
More likely, Mamselle refers to https://www.theatlantic.com/technology/archive/2019/05/college-board-sat-adversity-score/589681/, which contains the paragraph
Quote
Galton would die tied to his beliefs, but Brigham grew to regret inventing the SAT, writing in 1930 that SAT test scores don't measure innate ability passed through genes, but are instead "a composite including schooling, family background, familiarity with English and everything else, relevant and irrelevant." That sounds shockingly similar to the stance in favor of the adversity index: that exam scores are inseparable from the external contexts bearing down or lifting up students as they receive their education and take the test.
Quote from: polly_mer on May 19, 2019, 09:22:19 AM
Quote from: eigen on May 19, 2019, 05:31:56 AM
I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,
So what do we use instead? I've spent a lot of time at nearly open enrollment institutions and have thus encountered a fair number of students who had transcripts indicating excellent performance in high school, but were not ready for college.
A colleague called it the zip code effect where some of these high schools were doing so little education that:
- A C meant "Handed in some work at some point and didn't contribute to an assault on a teacher that resulted in hospitalization".
A B meant "Handed in at least half the work and attended class regularly, probably without assaulting the teacher at all".
An A meant "Did everything that was asked and was moderately respectful of the teacher".
I dealt with tears every term as I had to explain to the diligent A students who came from terrible high schools that they were so underprepared that the hardest they've ever worked will be at best a C in the class. I handed over the tissue box regularly as I explained that the local performance indicated needing remedial work in math, despite that A in high school calculus that seemed based more on attendance and willingness to try than demonstrated ability to solve problems.
The standardized tests generally aren't all that useful in discerning the difference between good, great, and excellent for those who score pretty well, but a poor score can indeed indicate areas that need improvement even when the transcript indicates the subject matter has been mastered.
I think what we move to depends on the type of institution we're talking about.
For open enrollment institutions, I think replacing standardized scores with entrance exams for placement would be much better, and would allow departments to assess whether the students were ready for going into their starting classes, or not.
If we're talking about selective institutions, I don't think the scores really do help all that much. I still see lots of students with high scores (and high GPAs) that aren't ready for college material, especially as more and more high schools focus on SAT and ACT prep in courses. That said, I'm not sure what to replace them with. Essays help, quite a bit, and a lot of it depends on what the institution is prepared to handle in terms of background and remediation.
As I understand it, the new score is not factored into the regular SAT scores — those remain the same. As I understand it, the new score is intended to reflect the student's socio-economic circumstances, and goes alongside the main scores. So institutions can ignore the new score if desired. Has anyone actually seen the new scores, who can confirm?
My institution has recently decided to go test-optional. I have a lot of mixed thoughts about it. Like most public institutions, we're situated in a state that has large, urban schools that are well-funded, and small rural schools gasping for every breath. And their test scores are a pretty accurate reflection of this fact.
In my experience, a good test score almost always indicates a good student, even if the GPA is mediocre or poor (I call this the slacker profile). A bad score, on the other hand, might not necessarily indicate a bad student, but it does raise a red flag. Since schools and GPAs vary so wildly across my state, the test has become the great equalizer. For all its flaws, it still gives me useful information about this student's likelihood of succeeding in college. I think caution should be exercised before placing too much stress on the test score, but it cannot be ruled out entirely. Unfortunately, at the end of the day, succeeding in college is an awful lot about taking tests and doing well on them.
I can buy the argument that the tests aren't great - they're biased in many ways. But we don't seem to have viable alternatives at this point. Essays take a lot of time to assess, and they can be gamed even more easily than standardized tests. Interviews can give a lot of information, but they're even more time-consuming than reading an essay - an admissions office would have to double or triple its staff to handle that amount of work.
It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better. Because if we could, we would. Wouldn't we?
Quote from: wareagle on May 20, 2019, 09:57:05 AM
It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better. Because if we could, we would. Wouldn't we?
Well. Other countries (including other Anglophone countries) don't use standardized tests for university admissions.
Quote from: Parasaurolophus on May 20, 2019, 10:06:12 AM
Quote from: wareagle on May 20, 2019, 09:57:05 AM
It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better. Because if we could, we would. Wouldn't we?
Well. Other countries (including other Anglophone countries) don't use standardized tests for university admissions.
One very well known Canadian university applies a "correction factor" to high school grades for applicants to highly competitive programs. So if you went to a poorly-ranked high school, even if you performed spectacularly, you're out of luck. Is that an improvement?
"Other countries (including other Anglophone countries) don't use standardized tests for university admissions." They don't? They certainly do in Britain. Do A-levels not count as standardized tests?
I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees. The more sources of information, the better, seems to me.
Quote from: Hegemony on May 21, 2019, 12:51:54 PM
"Other countries (including other Anglophone countries) don't use standardized tests for university admissions." They don't? They certainly do in Britain. Do A-levels not count as standardized tests?
True. I forgot about A-levels.
Well, then, I'll rephrase: not all other countries--not even all Anglophone countries--use standardized tests for university admissions.
Quote from: Parasaurolophus on May 21, 2019, 02:48:24 PM
Quote from: Hegemony on May 21, 2019, 12:51:54 PM
"Other countries (including other Anglophone countries) don't use standardized tests for university admissions." They don't? They certainly do in Britain. Do A-levels not count as standardized tests?
True. I forgot about A-levels.
Well, then, I'll rephrase: not all other countries--not even all Anglophone countries--use standardized tests for university admissions.
Quote from: Hegemony on May 21, 2019, 12:51:54 PM
I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees. The more sources of information, the better, seems to me.
Would you think differently if you were operating in a country where educational achievement at the secondary level was very even across the board, including between students of different class and ethnic backgrounds? (I guess I'm wondering to what extent you think the need for these extra measures is driven by problems particular to/particularly acute for education in the US.)
The problem is that the SAT/ACT are just capstones on top of a vastly unequal system of primary and secondary education in the US. Wealthy, invested parents will work any system developed. Even without the new adversity score, keep in mind how magnet schools often work -- they're placed in poorly performing districts and attract wealthy white high performing students by in effect setting up a school within a school, where the enrichment programs segregate a small group who then reap the advantages of both the extra educational investment and the cachet of applying from a low-performing district.
My offspring attend a high school with a 15.4% free and reduced school lunch rate. Because they are in honors/AP, they are in classes that are almost all white and Asian (we have a tech business base that has drawn many wealthy Asian immigrants over the last decade). Their classmates take the SAT and ACT multiple times. Some have consultants to "help" them with their essays and determine which schools should receive their early decision applications. And this is nowhere close to peak wealthy college admissions gaming behavior.
I suppose on some level something like a regents' exam system can at least determine which schools are handing out As for showing up, but no matter what metric is chosen, the privileged will leverage it.
Quote from: Parasaurolophus on May 21, 2019, 02:50:47 PM
Quote from: Hegemony on May 21, 2019, 12:51:54 PM
I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees. The more sources of information, the better, seems to me.
Would you think differently if you were operating in a country where educational achievement at the secondary level was very even across the board, including between students of different class and ethnic backgrounds? (I guess I'm wondering to what extent you think the need for these extra measures is driven by problems particular to/particularly acute for education in the US.)
Presumably countries who can do that must have some may of ensuring consistent standards
in high school, which would mean they've managed it earlier in the pipeline. (Examples???)
Or, by one means or another, they channel students more directly into vocational vs. liberal arts tracks earlier on.
Friends in France had to really push to get into the theology courses they wanted to take, because theological studies weren't in the profile of courses included in their program.
M.
Quote from: mamselle on May 22, 2019, 08:08:12 AM
Or, by one means or another, they channel students more directly into vocational vs. liberal arts tracks earlier on.
So is that a good thing or a bad thing?
Quote
Friends in France had to really push to get into the theology courses they wanted to take, because theological studies weren't in the profile of courses included in their program.
M.
Is it better to try and funnel students into programs where they are more likely to be successful, or to let them try whatever they like and then try to remediate like mad when they struggle?
Quote from: polly_mer on May 19, 2019, 09:22:19 AM
Quote from: eigen on May 19, 2019, 05:31:56 AM
I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,
So what do we use instead? I've spent a lot of time at nearly open enrollment institutions and have thus encountered a fair number of students who had transcripts indicating excellent performance in high school, but were not ready for college.
A colleague called it the zip code effect where some of these high schools were doing so little education that:
- A C meant "Handed in some work at some point and didn't contribute to an assault on a teacher that resulted in hospitalization".
A B meant "Handed in at least half the work and attended class regularly, probably without assaulting the teacher at all".
An A meant "Did everything that was asked and was moderately respectful of the teacher".
I dealt with tears every term as I had to explain to the diligent A students who came from terrible high schools that they were so underprepared that the hardest they've ever worked will be at best a C in the class. I handed over the tissue box regularly as I explained that the local performance indicated needing remedial work in math, despite that A in high school calculus that seemed based more on attendance and willingness to try than demonstrated ability to solve problems.
The standardized tests generally aren't all that useful in discerning the difference between good, great, and excellent for those who score pretty well, but a poor score can indeed indicate areas that need improvement even when the transcript indicates the subject matter has been mastered.
True story about just how hard it is to predict student achievement.
My brother and I grew up in the same household, were taught the same values, ate the same foods, learned most of the same skills in our summer job days, etc. We attended the same public school--a small-town school that wasn't great, but was hardly a sink school either. Our grades were pretty similar, and we made similar standardized test scores. Both of us won full scholarships to the same nearby SLAC during our respective years of graduation. According to any conceivable set of measurable metrics, our academic potential should have been nearly identical. If the metrics took into account extracurricular activities, he might have been rated a bit stronger.
I did well at our SLAC and graduated as one of the top students in our class, aced the GREs, got a fellowship for a PhD program at an R1 university, and went on to demonstrate that great GRE scores do NOT reliably predict success in grad school. That's why I'm a librarian now, not a career academic.
My brother blew off his classes in his freshman year, switched his major to something that seemed less demanding, kept blowing off his classes, and finally had to drop out. He eventually went into the military, and became a successful career NCO. After retiring from the service he went back to school and finally completed his education.
What set of metrics or algorithms could possibly have predicted such an outcome? Individuals are just too different to predict their futures through testing.
Quote from: marshwiggle on May 22, 2019, 05:29:41 AM
Quote from: Parasaurolophus on May 21, 2019, 02:50:47 PM
Quote from: Hegemony on May 21, 2019, 12:51:54 PM
I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees. The more sources of information, the better, seems to me.
Would you think differently if you were operating in a country where educational achievement at the secondary level was very even across the board, including between students of different class and ethnic backgrounds? (I guess I'm wondering to what extent you think the need for these extra measures is driven by problems particular to/particularly acute for education in the US.)
Presumably countries who can do that must have some may of ensuring consistent standards in high school, which would mean they've managed it earlier in the pipeline. (Examples???)
Well, at the risk of sounding like a broken record: Canada (https://www.bbc.com/news/business-40708421).
Like democracy, the standardized college exams are the worst solution, except for all the others. Polly is sadly absolutely correct to note that all high schools are not created equal, and college admissions staff need to know what a kid's real level of academic achievement is. Someone mentioned the value of college admissions essays, further, but these are often
, ahem, well let's just say one can buy success on these as well.
Quote from: Parasaurolophus on May 22, 2019, 01:03:27 PM
Quote from: marshwiggle on May 22, 2019, 05:29:41 AM
Presumably countries who can do that must have some may of ensuring consistent standards in high school, which would mean they've managed it earlier in the pipeline. (Examples???)
Well, at the risk of sounding like a broken record: Canada (https://www.bbc.com/news/business-40708421).
Since I've lived in 3 different provinces, and have some knowledge of the educations systems here, I'd say one of the main differences between here and the US is related to the teacher salary point; namely, that in Canada people expect government services to cost money, whereas in the US there seems to be almost a paranoia about the cost of government services and an obsession about eliminating "waste" in spending. (Also a dread of governemnt oversight, so things like national standards tend to raise alarm in the US, whereas in Canada having similar expectations across the country seems to be kind of "duh".)
Having said all that, in Ontario, and no doubt in other provinces, there is constant concern about whether and to what degree performance is declining over time.
Answer: 1776 vs. 1982.
;--}
M.
Quote from: mamselle on May 24, 2019, 03:28:35 AM
Answer: 1776 vs. 1982.
;--}
M.
Also, revolution vs. negotiation. :)
Yeah, that was what I was thinking.
Also, that over the two hundred years' difference, more or less, between the two, different scholastic practices evolved in the colonies-that-became-the-USA and the programs-used-in-the-UK-that-Canada-more-or-less-adopted before breaking off the bromance between London and the northern provinces.
Bailyn's study of the former is helpful in pointing out how some of those differences started, but they didn't have to move in the directions they have taken now.
The structures in Canada seem to have held back some of the stupider relativisms that have boiled up here (Alexander Hamilton is probably doing rotations in his grave, saying, "I told you so..") but the general trend of humankind to embrace stupidity and laziness wherever possible (the real deadly sin, in my book) seems to be putting things almost equal (maybe, by distant observation and the input of a couple of friends).
M.