Colleges righteously gaming the SAT

[smallcleanrat]

I have never seen an objective metric of performance such as a standardized test that wasn't critiqued.

Is the implication here that if objective metrics are critiqued a lot, not a single critique has any merit? Aren't subjective metrics of performance frequently critiqued as well? Couldn't someone also say "I have never seen a subjective metric of performance that wasn't critiqued?"

I'm not saying subjective metrics are better, I just don't get the point of this statement.

That requirement in black and white for a job? That makes no sense. I knew a guy once who didn't have that qualification that did really well in that field!

What if it's not one guy? What if it's a significant percent of people who don't seem to need that qualification to do well? What if, after adding that requirement, the company finds it isn't getting significantly better performance than they did before they added it? Or if a competitor is doing better and doesn't have that requirement? I'm not defending getting rid of standardized tests here, I'm trying to understand why you seem to be critiquing critiques in general instead of specific critiques.

Come to think of it, I've never seen a critique of an objective metric that wasn't critiqued.

If there is evidence that the use of an objective measure isn't serving as intended, is it never worth re-evaluating? Is every objective measure perfect as is as soon as it is implemented? There is no point trying to improve evaluation methods or rethink how much weight to give a specific objective measure compared to other measures used in decision-making?

People who devise and implement objective measures are fallible humans, so why shouldn't critiques be expected? Evidence can prove the critics wrong.

Those subjected to the tests wish to perceive their abilities (based on no evidence) greater than what is reflected in their score.

I'm not sure what you're referring to here. Are you talking about people who say their perfect SAT score means they have genius-level intelligence (something the SAT wasn't even designed to measure)? Or people who say their lower SAT score is not a true reflection of their aptitude (someone who argues SATs are not a good way to determine ability)?

Early in life is the time to recognize that these tests a) determine opportunities you will have regardless of your opinion of them b) will not have their outcomes changed no matter how many "flaws" you may think they have and c) are always right no matter what for these reasons.

So...no test has ever been restructured or revised or improved in any way? Why put flaws in quotes? Are flaws imaginary?

If your point is that you have to meet requirements whether you think they make sense or not simply because that's the way things are set up...are you talking about the individual job seeker/college applicant? Or beyond that?

Of course a college admissions committee is not (usually, or at least that they would admit) using standardized tests as a blanked yes/no criteria for admission.

Doesn't this leave a lot of room for decision-making when it comes to how much weight to grant SAT scores in the context of the entire application?

marshwiggle brought up the possibility of a threshold effect with score differences in the higher range being less useful for predicting success. So, it might be a mistake to judge a 200-point difference between 1400 and 1600 to be as informative as a 200-point difference between 1000 and 1200.

[marshwiggle]

I have never seen an objective metric of performance such as a standardized test that wasn't critiqued.

Is the implication here that if objective metrics are critiqued a lot, not a single critique has any merit? Aren't subjective metrics of performance frequently critiqued as well? Couldn't someone also say "I have never seen a subjective metric of performance that wasn't critiqued?"

I'm not saying subjective metrics are better, I just don't get the point of this statement.

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.) So the fact that
"SATS DON'T PERFECTLY PREDICT FIRST YEAR SUCCESS!!!!!" is not remotely remarkable to anyone who knows anything about statistics, but will get eyeballs. I think fianceguy's comment is a reflection of the frustration that people who actually understand statistics have in trying to calm down people who don't know much about statistics and see that sort of headline and freak out.

Twitter is one of the worst examples of how we now live in a comic book world where every statement has to end in an exclamation point! Because every statement has to be short and punchy! And memorable! So it will get retweeted!

Unfortunately, the truth is often fairly mundane, and therefore no-one can get a lot of attention by stating it.

[Hibush]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

[fourhats]

In a surprise turn of events, putting most of its content behind a paywall, and first hiding the fora, and then eliminating them outright, seems to have driven former free-content-providers like us elsewhere.

I'm late to the conversation about this, but I wanted to point out that the CHE is easily accessible for free if you use VPN to go into your institution's library, and read the journal through there. Or if you're on a campus server, you can probably read it from the office or home. I read it all the time and have never paid.

[smallcleanrat]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

Sometimes the "outrage" is the sensationalized part.

[marshwiggle]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

Sometimes the "outrage" is the sensationalized part.

The outrage shows how infantilized students have become. ( A fault of parenting over the past couple of decades.) Outrage is the typical response of a child who wants something and expects the parent to supply it. In modern politics, it reflects the idea that any problem has a well-known solution, and the people with the power to solve it simply aren't willing to do so. There's no possibility that the issue is complex enough that there is no perfect solution.
For example:

Problem: There are too many people incarcerated.
Police incarcerate people.
Solution: Defund the police!!! Less police means less incarceration. Easy-peasy

[smallcleanrat]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

Sometimes the "outrage" is the sensationalized part.

The outrage shows how infantilized students have become. ( A fault of parenting over the past couple of decades.) Outrage is the typical response of a child who wants something and expects the parent to supply it. In modern politics, it reflects the idea that any problem has a well-known solution, and the people with the power to solve it simply aren't willing to do so. There's no possibility that the issue is complex enough that there is no perfect solution.
For example:

Problem: There are too many people incarcerated.
Police incarcerate people.
Solution: Defund the police!!! Less police means less incarceration. Easy-peasy

I wasn't disputing that there are people who overreact or take extreme positions.

I was talking about another type of clickbait headline: "Outrage over..." It's making the level of contention sound more extreme than it really is because words like 'disagreement' or 'criticism' don't grab much attention.

[apl68]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

What sorts of things have they been complaining about?

[marshwiggle]

I wasn't disputing that there are people who overreact or take extreme positions.

I was talking about another type of clickbait headline: "Outrage over..." It's making the level of contention sound more extreme than it really is because words like 'disagreement' or 'criticism' don't grab much attention.

Fair enough. And "Students outraged" can technically mean 2. Most may be totally oblivious and/or apathetic.

[financeguy]

My stances are fairly clear. You can argue over the best way to design a game, but once the game has been designed and the rules are understood by both parties, they are the rules and both sides are expected to play by (or performing to) those rules, not the imaginary rules that they prefer. This is the case with subjective or objective features. I, for example, believe that nearly any objective metric has greater predictive validity than the subjective opinions of an interviewer, yet I understand that no matter how much better a CV I present than candidate 2, I can lose the job just because someone "thinks I'm a jerk." The useful response is not to point out that this guy doesn't have my qualifications, couldn't even understand the premise of my latest article if he tried, and isn't capable of evaluating me in any meaningful way. It's to not be a jerk and do everything I can to come off in a positive light to this person whose opinion has already been validated as "right" before I walk in the door.

Regarding companies with job requirements, you can certainly critique any requirement at all, but it's a moot point because the person with that mindset is never going to be in a position to hire others or start a business anyway. They will perpetually be looking at the success of others with disdain and wonder because they spent their time critiquing requirements that are actually present in the world rather than fulfilling them.

The SAT is may or may not have been designed with imperfections, but once it is there and is a requirement, it is right and as infallible as the pope simply because it's a rule of the game we all know going in which means the "non loser" mindset is best changed from worrying about the rules of the game to most effectively playing it.

Regarding how much weight an institution places on the SAT or other standardized tests, I suppose that could be the result of a number of things. How competitive in general are they? Do they teach primarily STEM subjects that must bow to objective reality or primarily fluff fields? At the end of the day, any particular institution may decide the numbers are less useful to them. If in doing so, they don't like the "hit" their ranking takes by virtue of having a lower average for entering students, they are left with a similar choice of engaging in the practical task of maintaining/increasing their ranking or complaining that the rankings are invalid.

[Hibush]

I think the media (which would include online sources) in the quest for clicks, have to sensationalize everything. (Old joke: What's the definition of a conflagration? A reporter's first fire.)

In our student newspaper, stories about students finding something suboptimal in the college experience have the headline "Students outraged at..."  Which makes me suspect they have lived a comfortable life to that point. (How they sustain all that outrage is beyond me. Exhausing!)

What sorts of things have they been complaining about?

It is really across the board. Not enough food choices in the dining hall, unresponsive advising staff, their student activity's funding being cut, vigorous parking enforcement by city police, widespread abuse and murder of US citizens. 

I reserve my outrage for the worst ones.

[Caracal]

The bottom line is that standardized metrics of success are everywhere in the world, including licensing tests for one of every three fields of employment. Early in life is the time to recognize that these tests a) determine opportunities you will have regardless of your opinion of them b) will not have their outcomes changed no matter how many "flaws" you may think they have and c) are always right no matter what for these reasons. This is one area where you (or mom) can't ask to speak to the manager.

Hmm, fields of employment? I don't really know how you define a field of employment, but that seems like a misleading statistic. I'm pretty sure that the number of people who have to take a standardized licensing test to get a job is much, much lower than that. Also, the difficulty of these tests is probably a more important metric than whether some professional license requires them. There are lots of standardized tests required for licensure that are basically the equivalent of the written portion of the driver's license exam-they are just designed to ensure a basic level of knowledge of rules rather than actually weed anyone out.

[financeguy]

Not an exaggeration at all. I have eight occupational licenses personally, which is down from ten after I decided to remove two areas of business from my "day job" outside of academia since the cost didn't justify keeping them. You need a license to cut hair, to be an interior designer in some areas, licenses to sell caskets, licenses to be a florist. It never ends. My "one in three" number (up from one in twenty in the 50s) comes from an Institute for Justice Study influenced by an Obama admin study:

https://ij.org/issues/economic-liberty/occupational-licensing/

Video at the bottom of the page is a good short (five minute) summary.

Given the option, I'd remove all occupational licenses, but I don't live in a world that has been designed to my preferences and thus I have received many of them.

Regarding difficulty, there are many licenses that are "difficult" for reasons totally unrelated to the field. In his article on occupational licenses in Capitalism and Freedom, Milton Friedman gives examples of some of the absurd questions that are present in many exams. If I recall correctly one was related to communism on an exam for something farm related. They're protection rackets and extortion scams, but again, it's not my decision. I have to recognize it's right no matter what, pass the test, and collect my money from what I'm using them for.

To make a broader observation, I believe one of the greatest indicators of success in any field or life in general, second only to the capacity to defer gratification is the willingness to meet the requirements actually present in the world, ignoring our own hypothetical preference.

[Caracal]

Not an exaggeration at all. I have eight occupational licenses personally, which is down from ten after I decided to remove two areas of business from my "day job" outside of academia since the cost didn't justify keeping them. You need a license to cut hair, to be an interior designer in some areas, licenses to sell caskets, licenses to be a florist. It never ends. My "one in three" number (up from one in twenty in the 50s) comes from an Institute for Justice Study influenced by an Obama admin study:

Right, but lots of these exams aren't difficult at all. For example 85 percent of people pass the nursing exam on their first attempt. Of those who fail, about half pass it the next time they take it. And the point I was making was that far fewer than one in three people have jobs where they have to take some sort of exam.

Sure, an ability to take standardized tests is important in some fields and might be necessary, but most people don't encounter them in their work. I'd also suggest that there's an important difference between standardized tests where all that matters is that you pass and ones in which you are being ranked against others. The basic principle of a licensing exam is that people in a field should have a basic, acceptable level of knowledge. Lawyers don't have a better chance of getting a job if they get a higher score on the bar exam.

[mamselle]

Hmmm....I always thought--from reading all the John Grisham books--that lawyers were scouted like football stars based on their bar exam results...

;--》

M.