New tool to look at college viability released

[polly_mer]

In fall 2019, Edmit was prevented from publishing a list of projected number of years colleges/universities in the US could remain open on their current paths.

Edmit has modified their report to be a searchable database that includes COVID-19-related metrics and have rated survival as low, medium, high, along with other relevant information for prospective students and their parents.

Of possible interest to faculty, staff, administrators, and others on how to do a more in-depth review of the financial health of their own institution, go to https://www.edmit.me/en/resources/how-to-assess-the-financial-health-of-a-college-or-university  and scroll to the section title that includes the phrase "If You're A Regular Person".

Edmit also has a section on the how-to-assess page with the phrase "Additional Sources of Information and Data on College Financial Health" that links to more databases/tables that are ratings based on different methods.


The folks at IHE have an article regarding this tool and background

[Aster]

Oh this is a fun tool. Nice layout too. So easy to search universities.

[TreadingLife]

This recently hit the wire
https://www.newstribune.com/news/local/story/2020/may/08/lu-declares-financial-exigency/826858/

You can't even find Lincoln University of Missouri in the database. It would have been interesting to see their ratings prior to this announcement. My college has a "high" rating. Sure, Jan.

[polly_mer]

Lincoln University couldn't have been in good shape:

https://www.newstribune.com/news/local/story/2018/nov/29/lu-enrollment-down-again/754481/

https://www.newstribune.com/news/local/story/2020/apr/16/lincoln-university-facing-another-employment-discrimination-lawsuit/824472/?utm_source=google_nl&utm_medium=entity_landing&track=entity_landing

[Puget]

Interesting tool-- poking into the methodology a bit, I'd like to see a lot more information on how they did their modeling-- I'm sure they are treating this as proprietary, but it's impossible to evaluate something where the methodology section just says the it

"uses a common statistical technique known as regression by looking at historical trends of revenue and expense, and how changes in those trends affect net asset reserves of the institution."

OK, what variables go into the regression exactly? How exactly are the effects of the new covid variables included, given that there is no historical data to put into a regression for those? They appear to be using fixed projections for these across the board for all institutions, for changes in tuition revenue, investment returns and salary-- this seems very unlikely to be the case (e.g., tuition may go up at some publics and down at privates, investment strategies differ across institutions). Sure, there isn't that much data to go on here, but performance in the great recession would at least provide a better estimate than these flat assumptions.

In addition, they are basing reliance on international students only on first-year undergraduates. I think this way undersells reliance on international students for a lot of universities that have much higher percentages of international students in graduate programs. Just to pick an example from a university I'm familiar with, it claims CU Boulder has low reliance on international students-- and that is certainly true at the undergraduate level-- but some masters programs like engineering are very strongly reliant on international students. This is also true of the business school in my university, and I'm guessing we aren't alone in that.

So, interesting, but modeling well is hard, and I'm not convinced based on what they are publicly reporting in their method section that they are doing it well.

[marshwiggle]

So, interesting, but modeling well is hard, and I'm not convinced based on what they are publicly reporting in their method section that they are doing it well.

Fair point, but is it likely that some place they identify as "high" risk is likely low? Any institution is going to have some idiosynchratic strengths and weakness the model won't catch, but I'd guess the extreme cases aren't likely to be far off from the status predicted.

[polly_mer]

If they were reporting, "College X will run out of money within 2 weeks of 7 September 2021 ", then I agree on wanting a better look at methodology that goes into the model.

I am much less concerned about ratings of equivalently "probably fine", "yellow flag", and "red flag".

It's not good for colleges to have a red flag explicitly shown to prospective students and their parents.  However, that's what those folks should be inferring from, say, a tuition sticker price of $20k with an acceptance rate of >70%, an enrollment under 1500, an endowment about the size of the annual budget, and a six-year graduation rate under 60%.

[Puget]

So, interesting, but modeling well is hard, and I'm not convinced based on what they are publicly reporting in their method section that they are doing it well.

Fair point, but is it likely that some place they identify as "high" risk is likely low? Any institution is going to have some idiosynchratic strengths and weakness the model won't catch, but I'd guess the extreme cases aren't likely to be far off from the status predicted.

I'd be more concerned that they are underestimating risk for some places--

If the business school and/or school of engineering many places are heavily dependent on international graduate students, and that isn't in the model at all, that seems like a big problem.

And for publics, it's not clear that drops in state funding are in the model at all (the only covid-related adjustments they list are tuition, investment returns, and salary costs), which also seems like a big oversight.

They are estimating tuition revenue decreases of 10% this year and 20% next year (not sure where those numbers come from, they just say "we believe these estimates are quite reasonable")--for small already struggling schools that seems like it may be a big underestimate (whereas it is likely too high for others).

The original model was probably OK, but this seems like a quick and dirty attempt at making covid adjustments that wasn't properly thought through. Is it better than nothing? I don't know, maybe. But models can also provide a false sense of certainty if they aren't reported transparently with confidence intervals. Categorizing based on high, medium and low (and they don't say where those lines are drawn) also hides variance within categories-- is an institution just over the line into "medium" or almost at "high"? You can't tell.

The right way to do this would be to run simulations under a variety of possible scenarios (e.g., plausible ranges of changes in revenue, drawn from a distribution), run many thousands of these simulations and report out the means with confidence intervals. For public consumption you could report out "best" "worst" and "intermediate" case scenarios. Heck, you can even make it interactive so people can play with the assumptions themselves and see what happens to the results. This is the way actual good forecasting models of uncertain outcomes work (see e.g., good covid models (not the Excel function models the White House tweeted out!, 538 election models, etc.).

[polly_mer]

I still worry much less about the probably good enough than the 'holy cow, just close gracefully now! You probably shouldn't have taken students last fall'.

The point in my mind is to identify the schools already off the cliff for even a good case instead of correctly classifying every institution under realistic distributions of scenarios.  One thing we worry about a lot in the big picture for physical simulations we do is wasting a lot of effort on investigating scenarios that are already clearly go or no go when those resources should go into making the models better for the grey areas or the unknown unknowns.

Around here, we are pretty insulting to the academics who want to spend limited resources on having the absolute best models far from the cliff instead of establishing exactly where the cliffs are and acquiring the additional information necessary to have more confidence that we've identified all the major factors related to the cliffs.

[spork]

If they were reporting, "College X will run out of money within 2 weeks of 7 September 2021 ", then I agree on wanting a better look at methodology that goes into the model.

I am much less concerned about ratings of equivalently "probably fine", "yellow flag", and "red flag".

It's not good for colleges to have a red flag explicitly shown to prospective students and their parents.  However, that's what those folks should be inferring from, say, a tuition sticker price of $20k with an acceptance rate of >70%, an enrollment under 1500, an endowment about the size of the annual budget, and a six-year graduation rate under 60%.

Yes, I agree. If I was a parent of a college-bound child, I'd be paying a lot more attention to a college's retention and graduation rates, the size of its endowment, and its enrollment than to whether Jimmy thought the free cafeteria lunch on admissions open house day was yummy.

[Caracal]

So, interesting, but modeling well is hard, and I'm not convinced based on what they are publicly reporting in their method section that they are doing it well.

Fair point, but is it likely that some place they identify as "high" risk is likely low? Any institution is going to have some idiosynchratic strengths and weakness the model won't catch, but I'd guess the extreme cases aren't likely to be far off from the status predicted.

I'd be more concerned that they are underestimating risk for some places--



And for publics, it's not clear that drops in state funding are in the model at all (the only covid-related adjustments they list are tuition, investment returns, and salary costs), which also seems like a big oversight.

Well, part of the problem is that you would then be trying to model legislative decisions making. What sort of cuts happen to hire education budgets are dependent on how much federal aid states get, and how they try to balance their budgets. The states themselves are working under limitations, but the federal government can print its own money. The decisions made by legislators with that kind of power during an unprecedented economic crisis seem pretty hard to model.

[marshwiggle]

Well, part of the problem is that you would then be trying to model legislative decisions making. What sort of cuts happen to hire education budgets are dependent on how much federal aid states get, and how they try to balance their budgets. The states themselves are working under limitations, but the federal government can print its own money. The decisions made by legislators with that kind of power during an unprecedented economic crisis seem pretty hard to model.

That's like burning your furniture to heat your house. Technically possible, but unsustainable and with very unpleasant long term consequences. (e.g. Zimbabwe, Nazi Germany)

[Puget]

So, interesting, but modeling well is hard, and I'm not convinced based on what they are publicly reporting in their method section that they are doing it well.

Fair point, but is it likely that some place they identify as "high" risk is likely low? Any institution is going to have some idiosynchratic strengths and weakness the model won't catch, but I'd guess the extreme cases aren't likely to be far off from the status predicted.

I'd be more concerned that they are underestimating risk for some places--



And for publics, it's not clear that drops in state funding are in the model at all (the only covid-related adjustments they list are tuition, investment returns, and salary costs), which also seems like a big oversight.

Well, part of the problem is that you would then be trying to model legislative decisions making. What sort of cuts happen to hire education budgets are dependent on how much federal aid states get, and how they try to balance their budgets. The states themselves are working under limitations, but the federal government can print its own money. The decisions made by legislators with that kind of power during an unprecedented economic crisis seem pretty hard to model.

Hard to model certainly, but you could still model a plausible distribution of cuts and see what happens under different scenarios, same as for other variables. That would be a lot less wrong than not including state funding cuts in your model at all, which is the same as modeling them as zero. And if you don't think you can plausibly model major factors, I don't think you should be putting your model out there for public use at all.

[Caracal]

Well, part of the problem is that you would then be trying to model legislative decisions making. What sort of cuts happen to hire education budgets are dependent on how much federal aid states get, and how they try to balance their budgets. The states themselves are working under limitations, but the federal government can print its own money. The decisions made by legislators with that kind of power during an unprecedented economic crisis seem pretty hard to model.

That's like burning your furniture to heat your house. Technically possible, but unsustainable and with very unpleasant long term consequences. (e.g. Zimbabwe, Nazi Germany)

The government is printing massive amounts of money right now to finance its debt. When congress passed the CARES act, they borrowed all of that money. Where's the money coming from? In a direct sense, it is being borrowed, but the FED restarted the quantitative easing program, which basically means that the FED buys all these treasury bonds from banks. So the treasury issues the bonds, they pay the FED and then the money goes back into...the treasury. They are creating money out of nothing.

They can do this, because inflation isn't a worry. The price of oil went negative a few weeks ago. Deflation is a much bigger concern.

[apl68]

Small Alma Mater has a favorable rating.  Parents' tiny AM is rated as headed for trouble.  Both probably about right.