Methodology

The top reason students attend college is to get a better-paying job and achieve a financially secure future. But political efforts to hold institutions of higher education to standards of economic accountability have so far failed.

The college rankings market has not done much better. Ranking systems are not providing prospective students and parents with useful, actionable guidance as to which colleges and programs provide economic value and which do not.

We fill this gap – detailing which schools and programs of study are most likely to lead students to greater financial security.

We rank undergraduate colleges and universities based on how long it takes students to pay back their educational investment after attending. We call this payback.

We then adjust the payback by relative earning potential – the degree premium – between students at different schools and programs in the same state. We call the output of these 2 data points the economic score.

Payback

Michael Itzkowitz, a senior education fellow at Third Way and an external consultant on this project, developed the basic payback methodology we use in our ranking system. The idea is that, on average, any worthy postsecondary education will provide additional income above what peers without a college education earn.

The payback measures the amount time it takes to pay back a student’s educational investment with that additional income.

The quicker a student can recoup the total cost of their education, the sooner he or she begins to see a return on their educational investment.

The payback rate slaps a glaring red flag on Ashford University, and students may conclude to steer clear of this school. However, Cal State Fullerton outperforms UCLA based on payback rate despite a considerable difference in earnings. Determining which of these 2 schools to attend based on payback rate alone might lead to the wrong choice.

To account for earning differences, we developed a measurement called the degree premium.

Degree premium

The degree premium calculates how much more or less students from a particular school or program earn when compared to the state weighted average.

To calculate the degree premium, we simply deduct the state weighted average earnings from the school’s median earnings.

The degree premium illustrates the relative earning potential of schools in the same state.

It is also used to calculate our economic score, the basis for how we rank colleges and degree programs.

Schools that offer earnings of less than 100% of the state average see an economic score higher than their payback rate. Schools that earn more than 100% of the state average will see an economic score lower than their payback rate. The lower the economic score, the better.

How program-level analysis is different

While school-level earnings are based on all students that enrolled at a particular institution of higher education 10 years ago, program-level data looks only at the students in a particular degree program, 3 years after they have graduated.

• Institutional-level earnings data includes those that have graduated and those that have not graduated.
• The 10-year, post-enrollment cohort has likely been working longer than the 3-years-post-graduation students, meaning they would be, on average, earning more.

It’s important to note that different programs at the same school can offer vastly different earning projections. A school might have excellent school-level earning figures but underperform on certain degrees (and vice versa).

Thus, it is very important for prospective students, particularly those that already know what they want to major in, to research our program-level economic value data.

Graduate program valuation

When ranking graduate programs, we start by looking at the debt-to-earnings ratio, which expresses what percentage of a student’s earnings are eaten up by total debt.

We then adjust the debt-to-earning ratio by our own graduate degree premium, which uses undergraduate-level earnings in the same program and state as a comparative benchmark. We again call the output of these 2 data points the economic score.

Debt-to-earnings ratio

Graduate tuition costs and expenses are not reported by government sources. Instead of using cost to determine a payback, we look at debt as the expression of a graduate student’s investment in their education. This is compared to the student’s earnings by way of a debt-to-earnings ratio.

A debt-to-earnings ratio of 1 means the annual debt is the same as annual earnings. Anything higher than 1 means that that debt is higher than annual earnings; anything lower than 1 means earnings are higher than debt. Obviously, the lower the ratio, the better.

Of course, there are situations in which much higher earnings might excuse a higher debt-to-earnings ratio. To account for earning differences when ranking graduate programs, we have created the graduate degree premium.

Graduate degree premium

To calculate the graduate degree premium, we use a weighted average earnings benchmark of the same program, in the same state, at the undergraduate level. This allows students to:

1. Compare the annual earnings of graduate programs against one another.
2. Determine whether the earnings from a particular graduate degree make it worth getting from an economic perspective.

Note that we deduct the average annual debt repayment when calculating the graduate degree premium.

Graduate program economic score

We then divide the debt to earnings ratio by the graduate premium percentage to get a value score.

Graduate programs offering an economic advantage over undergraduate programs will see an economic score lower than their debt-to-earnings ratio. Schools in which the new earnings + repayment are actually less than undergraduate earnings will have an economic score that is higher than their debt-to-earning ratio. Remember that a lower economic score is better.

Conclusion

Our methodology is meant to provide prospective students the tools needed to pick the most economically promising schools and programs.

While we are not the first in the market to take this approach, we do offer some unique contributions, namely:

1. Adjusting our payback by the degree premium to more appropriately weight comparative earnings.
2. Offering a user-friendly product so students are better able to locate, understand, and action the information.

Some models make assumptions to predict future performance and create long-term ROI projections. We deliberately avoid doing that, limiting our model to using the best available real-life earnings data.

We will continue to build on this. College Scorecard recently released 3-year post-graduate earnings data, which was immediately reflected in our models. Next year, they will likely release 4-year data, and we will use to update our data sets accordingly.

Appendix: Further explanation of data

We use weighted averages to create benchmarks used in comparing multiple schools and programs against one another.

What are weighted average earnings?

Weighted average earnings adjust the importance of each school’s median earnings in a dataset based on the percentage of students the school accounts for in the dataset.

We have calculated hundreds of different weighted averages to standardize comparison across different category types – by state, by program, by degree level, by school type, and by different combinations of state, degree, school type. We demonstrate below how weighted averages are calculated.

How it works

1. The column earnings count  represents the number of students the National Student Loan Data System has used to calculate a school’s median earnings.
2. The earning count is then multiplied by the school’s median earnings, which outputs total earnings.
3. This process is repeated for each school in the comparison category.
4. The total earnings for all measured schools are then divided by the total earning count, the outcome equaling the weighted average earnings for the category.

In the above example, earnings from 28,309 students are measured and result in total earnings of $1,425,057,262. This comes out to a weighted average earning of $50,339. When we extend this to include all schools in California, the weighted average earnings of all California bachelor students 10 years after enrollment comes to $50,770.

Institutional-level undergraduate data

Cost

1. We use the average net price, which is the amount that the average student pays out of pocket to earn their credential. This is the total cost of attendance including tuition and fees, books and supplies, and living expenses, minus the average grant/scholarship aid. This includes only students who have received a grant or loan from the federal government.
2. To determine total cost, we estimate the average number of years it takes students from a particular school to graduate.[i]
   1. If a school makes available the time it takes for students to reach program completion, we use the average time and multiply that by the average annual net price. Therefore, if it takes students, on average, 3 years to complete their degree—and it costs $10,000 per year, their total net price to earn their credential would be $30,000 at that institution.
   2. If a school does not submit completion statistics, we use data from the National Student Clearinghouse. This data indicates that students typically take 5.1 years to earn their bachelor’s degree and 3.3 years for an associate degree. In addition, we use 1 year for certificate.

Earnings

1. We use the median earnings of students 10 years after their enrollment.
   1. This includes both, students who’ve completed their degree and those who did not.
   2. The earnings include only students that received some form of federal financial aid.
   3. Earnings are compared to high school earning figures from the Census Bureau’s American Community Surveys, using median earnings from years 24-29.

Program level data

Cost

1. We use the median annual amount of tuition, room and board, books, and fees paid by students that have earned at least $1 in federal loans or grants.
2. To determine total cost, we attempt to arrive at the average number of total years it takes students from a particular school to graduate.
   1. If a school submits length of graduation KPIs, we take the average length to calculate total cost.
   2. If a school does not submit graduation KPIs, we use a 2016 NCES study which indicates average graduation lengths as 5.1 for bachelor’s level, 3.3 years for associate, and we use 1 year for credential level.
3. Note that cost data is available only at the institutional level. Certain majors or specialist programs might cost more, or less, than the standard undergraduate program. When this information is made available, we will integrate it into our model.

Earnings

4. We use the median earnings of students 3 years after they graduate.
   1. The earnings include only students that received some form of federal financial aid.
   2. Earnings are compared to high school earning figures from the Census Bureau’s American Community Surveys, using median earnings from the ages of 24-29.

Graduate studies

Cost: The cost of attending graduate school is not reported by the Department of Education. Instead, debt is used as the metric for measuring relative program costs. Therefore, costing metrics do not take into account the out-of-pocket expenses students may accrue as a result of their advanced studies.

Total debt amount shows the median Stafford and grad PLUS loan amounts a student is left with after graduation.

Annual debt amount shows the median annual debt payment.

1. 1. If a school submits length of graduation KPIs, we take the average length in order to calculate total cost.
   2. If a school does not submit graduation KPIs, we use a 2016 NCES study which indicates average graduation lengths as 5.1 for bachelor’s level, 3.3 years for associate, and we use 1 year for credential level.

Earnings

3. We use the median earnings of the students 3 years after graduating.
   1. The earnings include only students that received some form of federal financial aid.

[i] Bachelor-level years to completion rates are based on the 2014 entering class. We measure the percentage that graduated in 4 years, 5 years, and 6 years. We assume the remaining enrolled students will graduate in 8 years. (4 year grad rate / total grad rate *4)+(5 year grad rate / total grad rate *5)+(6 year grad rate / total grad rate *6)+(8 year grad rate/total grad rate*8)

Associate-level years to completion rates are based on the 2017 entering class. We measure the percentage of students that graduated in 2 and 3 years. We assume the remaining enrolled students will graduate in 4 years. (2 year grad rate / total grad rate *2)+(3 year grad rate / total grad rate *3)+(4 year grad rate / total grad rate *4)