Web Policy  |  Intranet  |  Contact SCHEV    
SCHEV Research Data Blog

The official blog of @SCHEVResearch at the State Council of Higher Education for Virginia. Discussions about our work, national higher education data policy, and highlights about the data we publish.


Recent Posts Blog Archives Subscribe Feed All Blogs

Notes From a Recent Presentation on Student Success Measurement

by Tod Massa 15. September 2015 23:24


I have been asked to spend a few brief minutes today talking about student success and how we measure it. “Brief” because with the wealth of data we publish on our website, it is easy for me to spend hours talking about student success in terms of degree completion and persistence towards completion.  First, I will talk about the traditional graduation rate, commonly known as the federal or “IPEDS” graduation rate. Following that, I will introduce the Student Success Index, a relatively new measure that we developed about two or three years ago.

The basic graduation rate that you are probably most familiar with is the federal rate. It is based on calculating the percentage of students who graduate within 150% of normal time to completion (six years for bachelor’s degree students, three years for associate’s degrees) who started at an institution as first-time in college students enrolled full-time. To be counted as a completer on this measure, the student must have started and finished at the same institution.


Virginia public four-year colleges and universities, as a group,  have the second highest combined graduation rate in the nation (70% and tied with Iowa), lagging behind only Delaware (73%) using the IPEDS Graduation Rate Survey results of the 2007 entering cohort of first-time, full-time students. Delaware only has two public four-year institutions (Delaware State University and University of Delaware) representing an adjusted cohort of 4,381 students compared to 28,608 in Virginia. Iowa also started with many fewer students, 9,873, from only three institutions. North Carolina is at 60%, Maryland at 61%, West Virginia 45%, and the District of Columbia 15%. 


Nationally, the National Student Clearinghouse has calculated that 63.4% of students beginning at a public four-year institution in 2007 completed a degree within six years at any institution. Limiting the measure to students completing their degree at the institution where they began, only 50.6% completed within six years. Fifteen percent of students enrolling in college for the first-time in 2007 were still enrolled in year six without having yet completed. Here again, Virginia barely trails behind Iowa in overall completion rate with 77.8% to Iowa’s 79.8. When it comes to graduating from the institution where the student initially enrolled, Virginia is on top at 67.3% with Iowa at 65.6%. A portion of Iowa’s students are bit more likely to complete a degree at a two-year institution than Virginia’s students.




For the next two slides I am going to compare the most recent graduation rates to those 10 years earlier. First. We’ll look at the graduation rates of students starting at public four-years in 1998 and students starting at the VCCS and Richard Bland College in 2001. 

The overall rate for four-years was 64% with UVa at 92% and Norfolk State at 27% representing both ends of the range. Richard Bland had 33% graduation rate, the overall VCCS rates was 11%.  




Moving forward 10 years, the overall rate for the public four-years has increased from 64% to 70%. UVa is still at the top of the range at 94% and NSU is still at the other end of the range at 32%.  We will look in a little more detail in the next slide at the individual changes, but nearly every four-year institution has improved. Richard Bland drops from 33% to 29%, but the VCCS increases from 11% to 18%.



In this chart, it is easy to see that CNU had the biggest change in graduation rate over the 10 years between 1998 and 2008 with a nearly 26 point increase from 42% to 67%. Christopher Newport was followed by Virginia Commonwealth with a 20 point increase, George Mason with 15 points, and Virginia Tech with a 10 point increase. The University of Mary Washington was the only institution to lose ground, but that is only noticeable through rounding as the actual decrease was less than a point.

Because these charts focus on the six-year rate, they obscure the fact that the number of students completing within the fifth has shrunk significantly and more students completed within four years.



The major failings of the federal graduation rate are two-fold. First, it only captures the success (defined as completion) of students who begin at the institution as first-time in college (ever) with a full-time enrollment. In other words, it ignores transfer students and part-time. For example, a student beginning at Radford but transferring to George Mason in their second year, even if they complete a degree, they count as a loss at Radford and are invisible in Mason’s counts (other than as a degree completer). This is repeated to an even greater degree in measuring community college graduation rates. A student enrolling at Northern Virginia Community College and later transferring to George Mason, as part of the largest documented transfer pipeline in the nation, would not be counted as success once they graduated from Mason. If this same student had started at NoVA as a part-time student, he or she would be invisible as far as the federal graduation rate is concerned.

This short-coming is reflected again in the recently released federal College Scorecard. 

To combat such short-comings, and because Virginia has much better data than the US Education Department, SCHEV developed the Student Success Index or SSI. The SSI captures all four groups of enrolling at an institution for the first time. These groups are First-time, Full-time; First-time, Part-time; New Transfer, Full-time; and New Transfer, Part-time. We track student enrollment and completion over a number of years specific to each group, and count completions, regardless of where the degree is earned. Our goal is to measure the success of students and not the necessarily the success of institutions. We simply use the institution as a starting point of analysis.

As you can see in this slide, SSI results in a somewhat higher score than the traditional graduation rate. This is due to both counting degrees earned at any institution, and to also counting students who continue to persist in their enrollment in the final years of the model.  It is a simple truth that some students take a much longer time to finish than most.  


The Student Success Index provides a substantially different view of student outcomes than the graduation rate.  It still understates student success as further research has shown that there are significant gains in the completion of bachelor degrees if we look out to eight years after their first enrollment. Currently this measure goes out only six years. 




In this next slide we can see some the variance by institution in the Student Success Index for Extended-time to Completion. This variance is explained primarily by the differences in the student body at each institution, with additional influences from institutional effect and resources. 




At SCHEV, we make an effort to look at different measures of success and efficiency. For example, we also consider Average Time-to-Degree. On average how long does it take to a complete a four-year degree at a Virginia public institution? For students that start as full-time, just about 4.6 years compared to just under 4.4 years at private colleges. On average, it takes about four years for the initially full-time new student at our public two-years to complete an associate degree. This is yet another reason why the traditional graduation rate is a failure for many two-year institutions.




Returning to the Student Success Index, this slide shows the variance in success, again for extended-time to completion, by various demographic groups. There are probably no surprises here. Women do slightly better than men; students with Pell grants have lower rates than the overall population; younger students at entry (and thus closer to high school completion) do better than older students; Asian students and white, non-Hispanic students out perform all other identified groups.



Virginia is perhaps the only state that documents graduation rates, out to 10 years, to the level of detail that we do.  For example, here we show the trends in six-year graduation rates for first-time, full-time students (but entering at any time during the year, not just Fall as in the federal rate) broken out by family income ranges.  These ranges are based on multiples of the federal poverty-level as adopted by Council, upon recommendation of the Higher Education Advisory Committee. While the income data are limited to those students applying for federal financial aid through the FAFSA, the patterns are fairly clear. The wealthiest students tend to have the highest completion rates, poorest have the lowest rates.



We also look at characteristics such as the unmet financial need of students. We start with each student’s budget as provided by the institution and subtract all gift aid and family contribution for the first year and group students by the ranges of need that will have to be met through loans or work. Here we see that students with unmet need greater than $5,000 have the lowest graduation rates by far.



Finally, in this chart we see the role that student preparation and effort plays in the completing college. By the way, Mr. Chairman, I should mention that all these data pieces I‘ve shown, and more, are available on the SCHEV website. 

•79% of students who earn 24 credits or more in dual enrollment graduate within six-years at their original four-year institution of enrollment. 

•84% of students who successfully complete 24 credits or greater within their first year of enrollment graduate within six years.

•96% of students who successfully complete 60 credits or greater within the first two years of enrollment graduate within six years.

•83% of transfer students successfully completing 24 credits or greater within the first two years graduate within four years of transfer.  


Categories: General, Student Success

blog comments powered by Disqus
Follow SCHEV Research on Twitter