IHE article: New SAT score

[polly_mer]

What do you folks think about the adversity score being added to the SAT?  The commentariat at IHE pokes a lot of holes in the overall notion:
https://www.insidehighered.com/admissions/article/2019/05/17/college-board-will-add-adversity-score-everyone-taking-sat

[Parasaurolophus]

I'll be interested in reading what you all think, especially since so many of you are from/working in the US. To my mind, the whole test seems like for-profit bunkum with minimal real value, and this latest move sounds like a desperate stab to make it seem relevant. But maybe I'm wrong, and the new score does actually help to ensure "fairer" outcomes?

Also, I think it's telling that they're keeping the adversity scores secret. Surely that's the right business decision, but it doesn't inspire much confidence.

[mamselle]

Actually, The Atlantic just had an interesting article on that as well.

It was an online article, in which the author, raised in adverse conditions, suggested that it did no service to those who were defined as more oppressed. (Can't link from this phone, sorry).

I can follow that idea, that such an effort might not not habilitate so much as humiliate students in such a situation.

But it's also true that many potentially capable individuals are indeed handicapped by elements in their own background not of their own choosing or cause.

Could students possibly "opt in" or "opt out," choosing to have the scale applied to them, or not?

Or would that be open to abuse by the more cynical?

Addressing inherent inequalities in the current system is a step forward, but it may want further consideration, or more trialling, to discover alternate solutions with better potential outcomes.

M.

[spork]

My take? This is an attempt by the College Board to regain some of the business it has lost from universities making the SAT optional. The process of dropping SAT scores as an application requirement seems to have started in large part with the publication of The Big Test: The Secret History of the American Meritocracy, by Nicholas Lemann, in 1999 -- which has, in my opinion, a fascinating analysis of the role standardized testing has played in U.S. higher education.

I once attended a presentation about the biases reflected by the SAT. The speaker said college applicants' zip codes were a good proxy for SAT scores and probably had a stronger correlation with college academic performance than SAT scores did.

[eigen]

I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,

[polly_mer]

I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,

So what do we use instead?  I've spent a lot of time at nearly open enrollment institutions and have thus encountered a fair number of students who had transcripts indicating excellent performance in high school, but were not ready for college.

A colleague called it the zip code effect where some of these high schools were doing so little education that:

• A C meant "Handed in some work at some point and didn't contribute to an assault on a teacher that resulted in hospitalization".
  
  


• 
  A B meant "Handed in at least half the work and attended class regularly, probably without assaulting the teacher at all".
  
  
• 
  An A meant "Did everything that was asked and was moderately respectful of the teacher".
  
  

I dealt with tears every term as I had to explain to the diligent A students who came from terrible high schools that they were so underprepared that the hardest they've ever worked will be at best a C in the class.  I handed over the tissue box regularly as I explained that the local performance indicated needing remedial work in math, despite that A in high school calculus that seemed based more on attendance and willingness to try than demonstrated ability to solve problems.

The standardized tests generally aren't all that useful in discerning the difference between good, great, and excellent for those who score pretty well, but a poor score can indeed indicate areas that need improvement even when the transcript indicates the subject matter has been mastered.

[polly_mer]

Actually, The Atlantic just had an interesting article on that as well.

It was an online article, in which the author, raised in adverse conditions, suggested that it did no service to those who were defined as more oppressed. (Can't link from this phone, sorry).

One recent The Atlantic article is https://www.theatlantic.com/education/archive/2019/05/explaining-the-college-boards-new-adversity-scores/589708/, but that's a pretty positive spin from the CEO of the company that creates the SAT.

More likely, Mamselle refers to https://www.theatlantic.com/technology/archive/2019/05/college-board-sat-adversity-score/589681/, which contains the paragraph

Galton would die tied to his beliefs, but Brigham grew to regret inventing the SAT, writing in 1930 that SAT test scores don't measure innate ability passed through genes, but are instead "a composite including schooling, family background, familiarity with English and everything else, relevant and irrelevant." That sounds shockingly similar to the stance in favor of the adversity index: that exam scores are inseparable from the external contexts bearing down or lifting up students as they receive their education and take the test.

[eigen]

I think moving away from standardized tests (SAT, ACT, GRE) entirely is the better approach, personally. I don't find them substantively reflective of students abilities,

So what do we use instead?  I've spent a lot of time at nearly open enrollment institutions and have thus encountered a fair number of students who had transcripts indicating excellent performance in high school, but were not ready for college.

A colleague called it the zip code effect where some of these high schools were doing so little education that:

• A C meant "Handed in some work at some point and didn't contribute to an assault on a teacher that resulted in hospitalization".
  
  


• 
  A B meant "Handed in at least half the work and attended class regularly, probably without assaulting the teacher at all".
  
  
• 
  An A meant "Did everything that was asked and was moderately respectful of the teacher".
  
  

I dealt with tears every term as I had to explain to the diligent A students who came from terrible high schools that they were so underprepared that the hardest they've ever worked will be at best a C in the class.  I handed over the tissue box regularly as I explained that the local performance indicated needing remedial work in math, despite that A in high school calculus that seemed based more on attendance and willingness to try than demonstrated ability to solve problems.

The standardized tests generally aren't all that useful in discerning the difference between good, great, and excellent for those who score pretty well, but a poor score can indeed indicate areas that need improvement even when the transcript indicates the subject matter has been mastered.

I think what we move to depends on the type of institution we're talking about.

For open enrollment institutions, I think replacing standardized scores with entrance exams for placement would be much better, and would allow departments to assess whether the students were ready for going into their starting classes, or not.

If we're talking about selective institutions, I don't think the scores really do help all that much. I still see lots of students with high scores (and high GPAs) that aren't ready for college material, especially as more and more high schools focus on SAT and ACT prep in courses. That said, I'm not sure what to replace them with. Essays help, quite a bit, and a lot of it depends on what the institution is prepared to handle in terms of background and remediation.

[Hegemony]

As I understand it, the new score is not factored into the regular SAT scores — those remain the same.  As I understand it, the new score is intended to reflect the student's socio-economic circumstances, and goes alongside the main scores.  So institutions can ignore the new score if desired. Has anyone actually seen the new scores, who can confirm?

[wareagle]

My institution has recently decided to go test-optional.  I have a lot of mixed thoughts about it.  Like most public institutions, we're situated in a state that has large, urban schools that are well-funded, and small rural schools gasping for every breath.  And their test scores are a pretty accurate reflection of this fact.

In my experience, a good test score almost always indicates a good student, even if the GPA is mediocre or poor (I call this the slacker profile).  A bad score, on the other hand, might not necessarily indicate a bad student, but it does raise a red flag.  Since schools and GPAs vary so wildly across my state, the test has become the great equalizer.  For all its flaws, it still gives me useful information about this student's likelihood of succeeding in college.  I think caution should be exercised before placing too much stress on the test score, but it cannot be ruled out entirely.  Unfortunately, at the end of the day, succeeding in college is an awful lot about taking tests and doing well on them.

I can buy the argument that the tests aren't great - they're biased in many ways.  But we don't seem to have viable alternatives at this point.  Essays take a lot of time to assess, and they can be gamed even more easily than standardized tests.  Interviews can give a lot of information, but they're even more time-consuming than reading an essay - an admissions office would have to double or triple its staff to handle that amount of work.

It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better.  Because if we could, we would.  Wouldn't we?

[Parasaurolophus]

It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better.  Because if we could, we would.  Wouldn't we?

Well. Other countries (including other Anglophone countries) don't use standardized tests for university admissions.

[marshwiggle]

It is interesting that we in higher ed, for all our intelligence and experience in these matters, cannot seem to come up with something better.  Because if we could, we would.  Wouldn't we?

Well. Other countries (including other Anglophone countries) don't use standardized tests for university admissions.

One very well known Canadian university applies a "correction factor" to high school grades for applicants to highly competitive programs. So if you went to a poorly-ranked high school, even if you performed spectacularly, you're out of luck. Is that an improvement?

[Hegemony]

"Other countries (including other Anglophone countries) don't use standardized tests for university admissions."  They don't?  They certainly do in Britain.  Do A-levels not count as standardized tests? 

I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees.  The more sources of information, the better, seems to me.

[Parasaurolophus]

"Other countries (including other Anglophone countries) don't use standardized tests for university admissions."  They don't?  They certainly do in Britain.  Do A-levels not count as standardized tests? 

True. I forgot about A-levels.

Well, then, I'll rephrase: not all other countries--not even all Anglophone countries--use standardized tests for university admissions.

[Parasaurolophus]

"Other countries (including other Anglophone countries) don't use standardized tests for university admissions."  They don't?  They certainly do in Britain.  Do A-levels not count as standardized tests? 

True. I forgot about A-levels.

Well, then, I'll rephrase: not all other countries--not even all Anglophone countries--use standardized tests for university admissions.

I do agree with not simply using school grades for admission — these can be so heavily weighted by factors invisible to admissions committees.  The more sources of information, the better, seems to me.

Would you think differently if you were operating in a country where educational achievement at the secondary level was very even across the board, including between students of different class and ethnic backgrounds? (I guess I'm wondering to what extent you think the need for these extra measures is driven by problems particular to/particularly acute for education in the US.)