Methodology
The top reason students attend college is to get a betterpaying job and achieve a financially secure future. But political efforts to hold institutions of higher education to standards of economic accountability have so far failed.
The college rankings market has not done much better. Ranking systems are not providing prospective students and parents with useful, actionable guidance as to which colleges and programs provide economic value and which do not.
We fill this gap – detailing which schools and programs of study are most likely to lead students to greater financial security.
A word on sources
We use only official government data, namely the U.S. Department of Education’s College Scorecard and the integrated postsecondary education data system (IPEDS). We do this to ensure that the data is factual, unbiased, and based on current realworld earning information. Additionally, the recent relaunch of institutionallevel earnings data on College Scorecard after 7year hiatus signals renewed efforts to hold schools accountable for poor performance. As such, frameworks are established, and the metrics used to set and measure performance may be found in the College Scorecard data sets. It makes sense for us to align our data strategy accordingly.
Undergraduate rankings
We rank undergraduate colleges and universities based on how long it takes students to pay back their educational investment after attending. We call this payback.
Did you know?
Our economic score equation is set so that a 1% advantage in earningsplus will offset a 1% disadvantage in payback.
We then adjust the payback by relative earning potential – the earnings plus – between students at different schools and programs in the same state. We call the output of these 2 data points the economic score.
Payback
Michael Itzkowitz, a senior education fellow at Third Way and an external consultant on this project, developed the basic payback methodology we use in our ranking system. The idea is that, on average, any worthy postsecondary education will provide additional income above what peers without a college education earn.
The payback measures the amount time it takes to pay back a student’s educational investment with that additional income.
Payback
The number of years it takes a student to pay back their educational investment with new earnings.
The quicker a student can recoup the total cost of their education, the sooner he or she begins to see a return on their educational investment.
Earnings beyond California HS graduate  Total cost to earn credential  Payback  
California State University – Fullerton  $30,306  $40,754  1.37 
UCLA  $48,924  $72,822  1.49 
Ashford University  $12,450  $104,640  8.4 
The above table illustrates how payback is calculated at a few Californiabased universities.
The payback rate slaps a glaring red flag on Ashford University, and students may conclude to steer clear of this school. However, Cal State Fullerton outperforms UCLA based on payback rate despite a considerable difference in earnings. Determining which of these 2 schools to attend based on payback rate alone might lead to the wrong choice.
To account for earning differences, we developed a measurement called earningsplus.
Earningsplus
The earningsplus calculates how much more or less students from a particular school or program earn when compared to the state weighted average.
Weighted average earnings
Rather than giving equal weight to different schools, a weighted average weighs the earnings of each school by their number of relevant students. You can read more about how this is calculated in the appendix.
To calculate the earningsplus, we simply deduct the state weighted average earnings from the school’s median earnings.
Median earnings  California weighted average  School premium  
California State University – Fullerton  $54,586  $50,770  $3,816 
University of California – Los Angeles  $73,744  $50,770  $22,974 
Ashford University  $37,270  $50,770  $13,500 
The chart above illustrates how earningsplus is calculated.
The earningsplus illustrates the relative earning potential of schools in the same state.
It is also used to calculate our economic score, the basis for how we rank colleges and degree programs.
Payback  Earningsplus  Earningsplus percentage  Economic score  
California State University – Fullerton  1.37  $3,816  107.5%  1.27 
University of California – Los Angeles  1.49  $22,974  145.3%  1.02 
Ashford University  8.40  $13,500  73.4%  11.44 
The above chart calculates the economic scores of Ashford, UCLA, and Cal State Fullerton.
Schools that offer earnings of less than 100% of the state average see an economic score higher than their payback rate. Schools that earn more than 100% of the state average will see an economic score lower than their payback rate. The lower the economic score, the better.
KEY TAKEAWAY
Our economic score considers both a school’s payback rate and earningsplus. Remember that the lower the economic score, the better.
How programlevel analysis is different
While schoollevel earnings are based on all students that enrolled at a particular institution of higher education 10 years ago, programlevel data looks only at the students in a particular degree program, 3 years after they have graduated.
 Institutionallevel earnings data includes those that have graduated and those that have not graduated.
 The 10year, postenrollment cohort has likely been working longer than the 3yearspostgraduation students, meaning they would be, on average, earning more.
It’s important to note that different programs at the same school can offer vastly different earning projections. A school might have excellent schoollevel earning figures but underperform on certain degrees (and vice versa).
Thus, it is very important for prospective students, particularly those that already know what they want to major in, to research our programlevel economic value data.
Graduate program valuation
When ranking graduate programs, we start by looking at the debttoearnings ratio, which expresses what percentage of a student’s earnings are eaten up by total debt.
Rule of thumb
Don’t take on more debt than you can reasonably expect to earn in annual salary shortly after graduating.
We then adjust the debttoearning ratio by our own graduate earningsplus, which uses undergraduatelevel earnings in the same program and state as a comparative benchmark. We again call the output of these 2 data points the economic score.
Debttoearnings ratio
Graduate tuition costs and expenses are not reported by government sources. Instead of using cost to determine a payback, we look at debt as the expression of a graduate student’s investment in their education. This is compared to the student’s earnings by way of a debttoearnings ratio.
Program  Debt  Earnings  Debt to earnings  
UCLA  Education  $32,500  $60,890  0.53 
UCLA  Film/photographic arts  $86,524  $41,491  2.09 
UCLA  MBA  $93,407  $150,028  0.62 
University of Wisconsin, Madison  Social work  $36,449  $50,494  0.72 
University of Wisconsin, Madison  Curriculum and instruction  $26,549  $47,128  0.56 
University of Wisconsin, Madison  Marketing  $34,068  $113,083  0.30 
The chart above compares the debt and earnings of different programs at the same schools.
A debttoearnings ratio of 1 means the annual debt is the same as annual earnings. Anything higher than 1 means that that debt is higher than annual earnings; anything lower than 1 means earnings are higher than debt. Obviously, the lower the ratio, the better.
Of course, there are situations in which much higher earnings might excuse a higher debttoearnings ratio. To account for earning differences when ranking graduate programs, we have created the graduate earningsplus.
Graduate earningsplus
To calculate the graduate earningsplus, we use a weighted average earnings benchmark of the same program, in the same state, at the undergraduate level. This allows students to:
 Compare the annual earnings of graduate programs against one another.
 Determine whether the earnings from a particular graduate degree make it worth getting from an economic perspective.
Premium  Earnings beyond bachelor’s  Annual debt payment  Graduate premium  
UCLA  Education  $30,352  $4,380  $25,972 
UCLA  Film/photographic arts  $8,182  $11,676  $3,494 
UCLA  MBA  $99,067  $12,600  $86,467 
University of Wisconsin, Madison  Social work  $11,458  $4,920  $6,538 
University of Wisconsin, Madison  Curriculum and instruction  $6,744  $3,576  $3,168 
University of Wisconsin, Madison  Marketing  $58,760  $4,596  $54,164 
The above chart shows how the graduate earningsplus is calculated for a few different programs at the same schools.
Note that we deduct the average annual debt repayment when calculating the grad earningsplus
Graduate program economic score
We then divide the debttoearnings ratio by the graduate premium percentage to get a value score.
Program  Debt to earnings  Grad earningsplus percentage  Value score  
UCLA  Education  0.53  185%  0.288 
UCLA  Film/photographic arts  2.09  90%  2.330 
UCLA  MBA  0.62  270%  0.231 
University of Wisconsin, Madison  Social work  0.72  117%  0.618 
University of Wisconsin, Madison  Curriculum and instruction  0.56  108%  0.522 
University of Wisconsin, Madison  Marketing  0.30  200%  0.151 
Graduate programs offering an economic advantage over undergraduate programs will see an economic score lower than their debttoearnings ratio. Schools in which the new earnings + repayment are actually less than undergraduate earnings will have an economic score that is higher than their debttoearning ratio. Remember that a lower economic score is better.
Conclusion
Our methodology is meant to provide prospective students the tools needed to pick the most economically promising schools and programs.
While we are not the first in the market to take this approach, we do offer some unique contributions, namely:
 Adjusting our payback by earningsplus to more appropriately weight comparative earnings.
 Offering a userfriendly product so students are better able to locate, understand, and action the information.
Some models make assumptions to predict future performance and create longterm ROI projections. We deliberately avoid doing that, limiting our model to using the best available reallife earnings data.
We will continue to build on this. College Scorecard recently released 3year postgraduate earnings data, which was immediately reflected in our models. Next year, they will likely release 4year data, and we will use to update our data sets accordingly.
FAQ
How do you balance earningsplus and payback?
Our value score equation is set so that a 1% advantage in earningsplus will even out a 1% disadvantage in payback.
Is there a bias towards payback over earningsplus??
Despite what looks like an equal balance, there is some bias towards payback in the model.
Earnings are forever, while costs are often paid back in a relatively short time frame. Earnings then “last longer”, and so a longer term 1% advantage would seem to more than offset a shorter term 1% disadvantage.
Students and parents may determine for themselves if they would like to provide additional weight to earning premium amounts between closely ranked schools. From our perspective, cost is currently the most pressing issue. Many parents open education funds at a child’s birth to finance future education. Earnings are subject to change. Much of a student’s future earnings depend on job performance and other immeasurable or unknown factors. While not weighing more in favor of earnings might lead to a few questionable ranking choices, overall we believe it is the right decision.
Why don’t you include graduation rates in your ranking methodology?
The number one indicator of whether a student will graduate or not is income level, and we don’t believe that a school should be penalized for admitting more lowerincome demographics. We also don’t think a school should be rewarded for limiting admittance to lower income students.
Do you indicate poor graduation rate performance?
We do indicate whether a school falls under or over the state average graduation rate, with red indicating below average and green indicating above average. Additionally, if a school falls into the bottom quarter of the national graduation rate, they are pushed to the end of the rankings list.
Why do school and degreelevel earnings measure different time periods?
The College Scorecard first released institutionallevel earnings data in 20078 and continued releasing the data until 20156. At that point, earnings data started being measured. The following year, the first of the 1year degreelevel earnings data was released. The 2year degree level earnings were released the year after that.
The College Scorecard has stated that they intend to release longer time periods as the data becomes available.
Appendix: Further explanation of data
We use weighted averages to create benchmarks used in comparing multiple schools and programs against one another.
What are weighted average earnings?
Weighted average earnings adjust the importance of each school’s median earnings in a dataset based on the percentage of students the school accounts for in the dataset.
We have calculated hundreds of different weighted averages to standardize comparison across different category types – by state, by program, by degree level, by school type, and by different combinations of state, degree, school type. We demonstrate below how weighted averages are calculated.
Earnings count  Median earnings  Total earnings  
California State University – Fullerton  7,999  $54,586  $436,633,414 
University of California – Los Angeles  6,327  $73,744  $466,578,288 
Ashford University  13,983  $37,320  $521,845,560 
TOTAL  28,309  $50,339  $1,425,057,262 
How it works
 The column earnings count represents the number of students the National Student Loan Data System has used to calculate a school’s median earnings.
 The earning count is then multiplied by the school’s median earnings, which outputs total earnings.
 This process is repeated for each school in the comparison category.
 The total earnings for all measured schools are then divided by the total earning count, the outcome equaling the weighted average earnings for the category.
In the above example, earnings from 28,309 students are measured and result in total earnings of $1,425,057,262. This comes out to a weighted average earning of $50,339. When we extend this to include all schools in California, the weighted average earnings of all California bachelor students 10 years after enrollment comes to $50,770.
Institutionallevel undergraduate data
Cost
 We use the average net price, which is the amount that the average student pays out of pocket to earn their credential. This is the total cost of attendance including tuition and fees, books and supplies, and living expenses, minus the average grant/scholarship aid. This includes only students who have received a grant or loan from the federal government.
 To determine total cost, we estimate the average number of years it takes students from a particular school to graduate.[i]
 If a school makes available the time it takes for students to reach program completion, we use the average time and multiply that by the average annual net price. Therefore, if it takes students, on average, 3 years to complete their degree—and it costs $10,000 per year, their total net price to earn their credential would be $30,000 at that institution.
 If a school does not submit completion statistics, we use data from the National Student Clearinghouse. This data indicates that students typically take 5.1 years to earn their bachelor’s degree and 3.3 years for an associate degree. In addition, we use 1 year for certificate.
Earnings
 We use the median earnings of students 10 years after their enrollment.
 This includes both, students who’ve completed their degree and those who did not.
 The earnings include only students that received some form of federal financial aid.
 Earnings are compared to high school earning figures from the Census Bureau’s American Community Surveys, using median earnings from years 2429.
Program level data
Cost
 We use the median annual amount of tuition, room and board, books, and fees paid by students that have earned at least $1 in federal loans or grants.
 To determine total cost, we attempt to arrive at the average number of total years it takes students from a particular school to graduate.
 If a school submits length of graduation KPIs, we take the average length to calculate total cost.
 If a school does not submit graduation KPIs, we use a 2016 NCES study which indicates average graduation lengths as 5.1 for bachelor’s level, 3.3 years for associate, and we use 1 year for credential level.
 Note that cost data is available only at the institutional level. Certain majors or specialist programs might cost more, or less, than the standard undergraduate program. When this information is made available, we will integrate it into our model.
Earnings
 We use the median earnings of students 3 years after they graduate.
 The earnings include only students that received some form of federal financial aid.
 Earnings are compared to high school earning figures from the Census Bureau’s American Community Surveys, using median earnings from the ages of 2429.
Graduate studies
Cost: The cost of attending graduate school is not reported by the Department of Education. Instead, debt is used as the metric for measuring relative program costs. Therefore, costing metrics do not take into account the outofpocket expenses students may accrue as a result of their advanced studies.
Total debt amount shows the median Stafford and grad PLUS loan amounts a student is left with after graduation.
Annual debt amount shows the median annual debt payment.

 If a school submits length of graduation KPIs, we take the average length in order to calculate total cost.
 If a school does not submit graduation KPIs, we use a 2016 NCES study which indicates average graduation lengths as 5.1 for bachelor’s level, 3.3 years for associate, and we use 1 year for credential level.
Earnings
 We use the median earnings of the students 3 years after graduating.
 The earnings include only students that received some form of federal financial aid.
[i] Bachelorlevel years to completion rates are based on the 2014 entering class. We measure the percentage that graduated in 4 years, 5 years, and 6 years. We assume the remaining enrolled students will graduate in 8 years. (4 year grad rate / total grad rate *4)+(5 year grad rate / total grad rate *5)+(6 year grad rate / total grad rate *6)+(8 year grad rate/total grad rate*8)
Associatelevel years to completion rates are based on the 2017 entering class. We measure the percentage of students that graduated in 2 and 3 years. We assume the remaining enrolled students will graduate in 4 years. (2 year grad rate / total grad rate *2)+(3 year grad rate / total grad rate *3)+(4 year grad rate / total grad rate *4)