SAT Drops Essays and Subject Tests

[marshwiggle]

With the pandemic, a lot of schools dropped the SAT requirement because of the logistical challenges with taking it. Others did so because it is trendy.

In a roundup of admissions stats today, the NY Times notes that it vastly increased applications to selective schools. The gist is that students who thought their SAT score might disqualify them now applied.

Ms. Rickard of the Common App said, "Without a test score, maybe they aren't sure exactly where to aim, or they think this is their opportunity to try to get into a more selective institution."

"We saw people that thought 'I would never get into Cornell' thinking, 'Oh, if they're not looking at a test score, maybe I've actually got a chance,'" said Jonathan Burdick, Cornell's vice provost for enrollment.

"The elimination of that barrier really did drive application increases," said Emily D. Engelschall, admissions at the University of California, Riverside.

The cynic in me says that this policy has not increased access for underrepresented students. For schools like Harvard and Cornell, which are featured in the article, the admissions rate is already below 10%. There might be a unicorn among the SAT-averse new applicants, but I suspect the 43% additional applicants at Harvard and the 17,000 at Cornell are mostly fooled into thinking that they have a chance and are actually among the 90% that won't get in.

In other words, the Dunning-Kruger effect.

[writingprof]

Over at NYT, they're reporting that some Democrats want President Biden to "drop the SAT requirement" for his first Supreme Court pick.

(All right, their phrase is "forego the usual elite credentials," but the idea is the same.)

If I were "B"lack, I would absolutely love the constant message that existing standards must be lowered so that I can meet them!

https://www.nytimes.com/2021/02/21/us/politics/biden-supreme-court-black-woman.html?action=click&module=Top%20Stories&pgtype=Homepage

[mamselle]

Over at NYT, they're reporting that some Democrats want President Biden to "drop the SAT requirement" for his first Supreme Court pick.

(All right, their phrase is "forego the usual elite credentials," but the idea is the same.)

If I were "B"lack, I would absolutely love the constant message that existing standards must be lowered so that I can meet them!

https://www.nytimes.com/2021/02/21/us/politics/biden-supreme-court-black-woman.html?action=click&module=Top%20Stories&pgtype=Homepage

Your last sentence is based on a horrifically false, I judicious, racist assumption.

Reported.

M.

[writingprof]

Over at NYT, they're reporting that some Democrats want President Biden to "drop the SAT requirement" for his first Supreme Court pick.

(All right, their phrase is "forego the usual elite credentials," but the idea is the same.)

If I were "B"lack, I would absolutely love the constant message that existing standards must be lowered so that I can meet them!

https://www.nytimes.com/2021/02/21/us/politics/biden-supreme-court-black-woman.html?action=click&module=Top%20Stories&pgtype=Homepage

Your last sentence is based on a horrifically false, I judicious, racist assumption.

Reported.

M.

Let us hope that your "report" is proofread more carefully.

[financeguy]

And yet this is exactly the type of response you are guaranteed to receive when making an announcement beforehand that you will staff a role based on race and gender.

If we really are looking to get rid of those pesky standards and appoint a black female to the Supreme Court, I recommend OJ Juror Carrie Bess. Though not an attorney, she was in a courtroom for nearly a year which should be good enough. Also, having admitted that herself and "90 percent" of those who served with her voted to acquit an obvious double murderer as payback for Rodney King, she has exactly the experience Biden seeks of injecting race in a court preceding.

Fortunately these type of announcements regarding identity preference beforehand (like the one for heels up Harris beforehand) are bringing us all together as desired!

[Cheerful]

nevermind  off-topic deleted

[Puget]

To get us back on topic. . .

There is a cost/benefit to going test optional. I think too many people assume it will automatically make the process more inclusive, but that really depends on what you replace the tests with when it comes to making admissions decisions. Are you just weighing GPA more heavily, or are you trying to incorporate other factors, and if so, what are they and how do you evaluate them? It is easy to accidentally introduce biases into the process, so this takes careful thought and planing.

I along with two colleagues (one other psychologist and someone from education) have been helping undergraduate admissions with reforming their process, and specifically how they evaluate non-academic factors. This has been my favorite piece of service I've done because the admissions team genuinely wants to do things better, has engaged in really interesting and productive discussions with us, and is implementing changes with plans to follow up with data on how well it works. I've also learned a lot about the admissions process and practical constraints involved.

We spent a lot of time on trying to make sure both the criteria, and process, for non-academic factor evaluation would not bias the process toward students from certain backgrounds. This is harder than it sounds. For example, it may seem great to rate students who have engaged in extensive community service more highly, but that penalizes students who may have had to work or babysit younger siblings instead of volunteering, so you have to make sure the rating criteria also include service to one's family as of equal value. Likewise, we value "critical thinking", but we have to be careful to define that in a way that doesn't disadvantage students from cultures and school systems where arguing with adults is not OK (e.g., China), and think carefully about how evidence of critical thinking may show up differently in the applications of these students.

One of the biggest shockers for the admissions staff was that we all immediately raised concerns about their using "fit" as a criteria (which is what they had been doing)-- we pointed out that subjective fit ratings were a great way to perpetuate selecting the same type of students we already have (and those that most remind raters of themselves), but a terrible way to achieve the stated goal of further diversifying the study body. This seemed self evident to us from our professional backgrounds, but is apparently not something widely discussed in admissions, which was rather shocking to us. To their great credit, they were very receptive to hearing about the research on this though and fully embraced moving away from subjective fit as an admission criteria, toward a set of evidence-based "indicators of success" with specific documented evidence from the applications for each rating.

How well will this work? Well, we'll find out as they track students through the admission process this year and then through college for those that enroll. No doubt will go through multiple iterations of this process, but it is better than what they had already.

TLDR version: If you are going to go test-optional, it isn't enough to just drop the test, you have to have a careful process, with good consultation, to replace it with something better, or you can end up just replicating the same biases or introducing new ones.

[mamselle]

That sounds like a project worthy of publication in a serious education/administration-type journal.

I think Octoprof was doing something like that with accounting admissions, student continuity, completion, etc. I remember being awed by the number of data variations and comparisons their team found to work with and present/publish on(always described in very unidentifiable terms, but just vaguely recognizable, as I recall) from the old forum (I was actually trying to follow up the teamwork subdivisions, reading through those posts, when it was taken down!!)

It would also seem useful, if documented judiciously, in school evaluations and accreditation cycles. The ATS (Association of Theological Schools) did a similar study on seminary admissions, time to completion, failure rates, etc., that they published on about 10 years ago. (One surprising finding was that many people only took a few courses, then were often hired to a pulpit by a church that didn't care if they'd finished because the supply of pastors was so sparse in their denominations. This was making it harder for the schools to prepare those who did stay for their general ordination exams because they couldn't always fill and run upper-level classes once the early-school-leavers had departed.)

That was used by the seminaries studied in their accreditation bids as well, from what I heard.

Anyway, sounds like a cool, worthwhile project, and it has the potential to approach issues of rigor that don't require testing to address.

M.

[Hibush]

To get us back on topic. . .

There is a cost/benefit to going test optional. I think too many people assume it will automatically make the process more inclusive, but that really depends on what you replace the tests with when it comes to making admissions decisions. Are you just weighing GPA more heavily, or are you trying to incorporate other factors, and if so, what are they and how do you evaluate them? It is easy to accidentally introduce biases into the process, so this takes careful thought and planning.

This really gets to the problems with finding what to use. Every metric of academic preparations and potential for success has the very same bias as the standardized tests. You somehow have to disentangle the actual preparation and promise from societal ones, recognizing that there is a huge interaction. (The interaction being that wealth provides both the means and the culture to support better preparation.) 

The are two general groups of criteria. One are the other metrics, with the problems above. The other, as you mention, is fit.

If one uses "fit," it is absolutely critical to clearly define what the candidates are supposed to fit. What are the larger goals of your program going forward. I'm finding that step to be the toughest. Even getting people to express something coherent is hard, getting consensus as a department is not something I ever expect to see.

If your goal is to serve a more diverse base than before, defining fit as "conforms with our experience and prejudice", is counterproductive.

The department might do better defining fit as "will best use the education we offer to improve society."  Evaluating applications is going to be a subjective, but at least you understand what you are looking for. An applicant who comes from a group underserved by the benefits of your discipline would get some credit for potentially bringing that benefit to where it has more value.


If a department is on the chopping block, it might be enough for them to fit into the empty slot in your roster.

A healthy department should be able to describe some good dimensions of fit. If the loss of an SAT causes them to do that, it could be a net positive. If they do nothing,  just having all the other difficult metrics increase in weight, it will be a loss.

[wareagle]

To get us back on topic. . .

There is a cost/benefit to going test optional. I think too many people assume it will automatically make the process more inclusive, but that really depends on what you replace the tests with when it comes to making admissions decisions. Are you just weighing GPA more heavily, or are you trying to incorporate other factors, and if so, what are they and how do you evaluate them? It is easy to accidentally introduce biases into the process, so this takes careful thought and planning.

This really gets to the problems with finding what to use. Every metric of academic preparations and potential for success has the very same bias as the standardized tests. You somehow have to disentangle the actual preparation and promise from societal ones, recognizing that there is a huge interaction. (The interaction being that wealth provides both the means and the culture to support better preparation.) 

The are two general groups of criteria. One are the other metrics, with the problems above. The other, as you mention, is fit.

If one uses "fit," it is absolutely critical to clearly define what the candidates are supposed to fit. What are the larger goals of your program going forward. I'm finding that step to be the toughest. Even getting people to express something coherent is hard, getting consensus as a department is not something I ever expect to see.

If your goal is to serve a more diverse base than before, defining fit as "conforms with our experience and prejudice", is counterproductive.

The department might do better defining fit as "will best use the education we offer to improve society."  Evaluating applications is going to be a subjective, but at least you understand what you are looking for. An applicant who comes from a group underserved by the benefits of your discipline would get some credit for potentially bringing that benefit to where it has more value.


If a department is on the chopping block, it might be enough for them to fit into the empty slot in your roster.

A healthy department should be able to describe some good dimensions of fit. If the loss of an SAT causes them to do that, it could be a net positive. If they do nothing,  just having all the other difficult metrics increase in weight, it will be a loss.

Getting them is one thing.  Keeping them is another, and this (unfortunately) is where "fit" comes in.  Every campus has a distinct culture, and it is difficult and time-consuming to change.  My own campus, for example, is probably 25 years behind in terms of diversity and inclusion.  Diversifying the campus through different admissions practices might be feasible in just a few years, but making that campus welcoming to diverse students is a completely different matter. 

I applaud puget and colleagues for their work with admissions; it sounds complicated, messy, and very satisfying.  I have always been of the opinion that the biggest factor in retention is who comes in the door in the first place, and there are many conflicting values at play here.  Yes, we want diversity.  But we also need some fit.  And we want to provide opportunity to underrepresented groups.  But budget cuts have played hell with the support services needed by that population.

And the cynic in me would point out that the tests (SAT and ACT) generally correlate with a huge factor in student success - money.  You can say what you want about the lack of political correctness in all this, but the data will tell you the same thing every time.  Higher income correlates positively with student success in college.  So do higher SATs and ACTs.  If the latter are a proxy for the former, it does not change the outcome. 

[Puget]

To get us back on topic. . .

There is a cost/benefit to going test optional. I think too many people assume it will automatically make the process more inclusive, but that really depends on what you replace the tests with when it comes to making admissions decisions. Are you just weighing GPA more heavily, or are you trying to incorporate other factors, and if so, what are they and how do you evaluate them? It is easy to accidentally introduce biases into the process, so this takes careful thought and planning.

This really gets to the problems with finding what to use. Every metric of academic preparations and potential for success has the very same bias as the standardized tests. You somehow have to disentangle the actual preparation and promise from societal ones, recognizing that there is a huge interaction. (The interaction being that wealth provides both the means and the culture to support better preparation.) 

The are two general groups of criteria. One are the other metrics, with the problems above. The other, as you mention, is fit.

If one uses "fit," it is absolutely critical to clearly define what the candidates are supposed to fit. What are the larger goals of your program going forward. I'm finding that step to be the toughest. Even getting people to express something coherent is hard, getting consensus as a department is not something I ever expect to see.

If your goal is to serve a more diverse base than before, defining fit as "conforms with our experience and prejudice", is counterproductive.

The department might do better defining fit as "will best use the education we offer to improve society."  Evaluating applications is going to be a subjective, but at least you understand what you are looking for. An applicant who comes from a group underserved by the benefits of your discipline would get some credit for potentially bringing that benefit to where it has more value.


If a department is on the chopping block, it might be enough for them to fit into the empty slot in your roster.

A healthy department should be able to describe some good dimensions of fit. If the loss of an SAT causes them to do that, it could be a net positive. If they do nothing,  just having all the other difficult metrics increase in weight, it will be a loss.

I agree with much, but not all, of this. Although it is true that no metric is completely free of bias, some are better than others, and in combination they can give a more balanced picture of the applicant. There is actually pretty good research on what predicts student success (at least when it comes to GPA and graduating on time-- arguably those aren't the only metrics we should care about but they are the easiest the measure), not all of which are correlated with SES. For example, one of the things we included in our "non-academic" criteria was evidence of perseverance and resilience to set-backs (sometimes called "grit" though I don't happen to like that term).

Certainly, you have to first identify the goal before you can come up with a system for building a class that better meets that goal. In this case we were advising undergraduate admissions for the whole university, not for a particular program (I know some places have separate competitive admissions into majors, but we don't do it that way). So we were able to refer to core values statements and strategic goals defined by various high level university committees, as well as research on what predicts student success, and use both to define what we wanted students to be evaluated on within the world of the available data from applications. There is other information we would love to have, but don't, at least until next year when we may be able to add some optional additional essays, and rating scales for recommenders.

These things are the art of the possible-- you can't let the perfect be the enemy of the good.

[Hibush]

It sounds like you are setting up a strong admissions program that will result in successful students who like being there. I'm taking note!

Resilience and perseverance are attractive features that do seem to have a lot of empirical support. I've been struggling with identifying evidence for them in applications. They are pretty easy to game for those so inclined.

Now that I have a group of others' students to be responsible for, I'm selfishly valuing  perseverance and resilience even more. The ones without it take the most time and are not very rewarding to help.

[Puget]

It sounds like you are setting up a strong admissions program that will result in successful students who like being there. I'm taking note!

Resilience and perseverance are attractive features that do seem to have a lot of empirical support. I've been struggling with identifying evidence for them in applications. They are pretty easy to game for those so inclined.

Now that I have a group of others' students to be responsible for, I'm selfishly valuing  perseverance and resilience even more. The ones without it take the most time and are not very rewarding to help.

Thanks, we hope so!

Yep, we spent a lot of time talking about how to make it less easy to game the system. You can't completely prevent it, but you can reduce it by looking for clear behavioral evidence rather than just self-description, and by also looking in letters of rec rather than just their essays.

One of the things we'd love to add in the future are teaching ratings and specific comments on the dimensions we selected, which should give better info than the often generic letters. But we also recognize the inequalities of access to teacher time and effort to fill these things out between e.g., a large under-resources public school and a private prep school. Like I said, no perfect system but we're trying. . .

[Aster]

I enjoyed reading this article. Much of what was said made sense and seemed appropriate.

I certainly do not recall ever been awarded college credit for how well I did on the SAT. I mean, isn't that what AP Tests are for? The article directly pins this conundrum down.

SAT's are used to mostly *get into* college. If you want college credits earned, that's what AP Tests (and local-scale testing) are for. The SAT is not supposed to proxy for AP Testing anymore than AP Tests should be proxying for SAT's. They are mostly supposed to be doing different things for different uses.

Honestly, I don't even recall that there even was a subject test for the SAT... I guess that I never took one. The article says that they've been optional since the 1960's. I don't really see what point they would have now that we've had AP tests in widespread use for the last few decades. I would guess that many high schools have come to a similar conclusion for at least that long. So maybe the SAT board's dropping of subject tests is really just terminating a program that has functionally been dead for a long time already.

[mamselle]

 It's been a veeery long time ago, but I think I recall that the ACTs, which do have subject tests (in the afternoon--you took the main test in the AM and sat the topical tests after lunch, i think) might have been used in some way for placement out of certain requirements.

I know I placed out of the English I comp requirement and into Calculus, and I didn't take any AP courses or tests (I'm not even sure they were very widespread then--the bigger high school in the wealthier town up the road probably had them, but we didn't.) I always thought it was based on my ACT's.

The conservatory across town taught Saturday Music Theory prep classes and private lessons that could lead to placement out of various requirements in the music schools, but again, those didn't have credit attached, you just brought it to the attention of the music school in question, etc. (Or--as one student famously did, you went to Switzerland, learned to play the lute, and went around conducting early music programs for the rest of your life.)

In fact, acing the math part made things difficult: our small high school hadn't offered pre-calc or analytic geometry, just solid and trig. The advanced Calc I placed into presumed you'd already had them. I had to do some quick catch-up, never pulled better than a B in the class, and decided to drop after my second quarter. I loved learning it but dance and music practice didn't leave me time to do the problem sets, and I finally began to see where my deeper interests really were.

But there, again, it was much less formal, and there was more discussion around specific student qualifications--and you could still end up in classes you might not really be qualified for. 

And the university campus then only had 40,000 people and a football team that had just started to be pretty good: scholastics still had a chance.

M.