Overview of SCHEV’s Reports of Institutional Effectiveness (ROIE)

Purpose of the 2004 Reports of Institutional Effectiveness

Welcome to the second edition of the Reports of Institutional Effectiveness (ROIE) — annual reports intended to provide meaningful information on the academic quality and operational efficiency of Virginia’s public institutions of higher education.

SCHEV initiated and led in creating this new accountability tool. In 1999, SCHEV presented the concept of measures of institutional effectiveness. In 2000, the Governor and the General Assembly agreed that this SCHEV initiative was a good idea and mandated by law ROIE's creation through the 2000-2002 Appropriation Act (Item 162 #11c).

First published on July 2, 2001, the ROIE were developed to provide evidence of institutional effectiveness — the extent to which institutions accomplish their missions and students achieve their educational goals. In highlighting the institutions’ accomplishments and demonstrating progress towards improvement, the reports give students, parents, policy makers, faculty and staff, employers, and the general public access to important information about each of Virginia’s public four-year colleges and universities, Richard Bland College, and the Virginia Community College System.

This edition of the ROIE introduces the core competency assessments for Written Communication and Technology/Information Literacy. These assessments serve as a basis to begin to validate the degrees awared by these institutions. They are not designed to certify individual students - they are guides for the institutions, policy-makers and the public to an understanding of what constitutes competency in these areas and whether or not the institutions are  finding themselves to be effective. While many of the assessments in this report are actually pilot studies, in toto they create a report that is clearly one of the first in the nation on the abilities of college students statewide.

While system-wide measures appear on all institutional reports, caution should be exercised in making comparisons between institutions with different missions. Where appropriate, however, institutions’ performance has been compared to benchmarks, commonly accepted standards, or peer institutions – those with similar missions, comparable size, student demographics, and program mix. Where available, information also is provided in the context of historical trends. These comparators provide a broad context for interpreting the data.

Though not formally linked to any future funding model for higher education, the reports are intended to provide evidence of the value added by our colleges and universities to the Commonwealth of Virginia and its educational, economic, civic and cultural vitality.

Structure of the Reports of Institutional Effectiveness

Each institutional report is organized into fivesections: institutional mission, college profile measures, system-wide measures, and institution-specific measures, and the core competencies. system-wide measures for two- and four-year institutions may differ slightly based on the unique characteristics of the institutions.

Institutional mission describes the institution’s core values and priorities. The mission statement sets a vision for the institution and defines how it will get there. For example, a large urban university might describe its mission in terms of partnering with business and industry, pursuing a national research agenda, or meeting the needs of a highly diverse, non-traditional student body. A small liberal arts college with a more homogeneous student body might focus on undergraduate education, a broadly based curriculum, and the residential experience. The mission of two-year community colleges includes preparing students for transfer to a four-year college or university, providing technology training for initial entry into the workforce, and upgrading workers’ skills. The mission statement of each of the colleges provides the overall context in which the measures of effectiveness can be understood.

Institutional profile in-depth views of enrollment and projections of future enrollment. Also included are five years of general fund appropriations per in-state student FTE and the complete degree inventory for each institution.

System-wide measures include 14 performance measures focused on operational efficiency and factors associated with academic quality: freshman to sophomore retention rate, number of transfer students enrolled from two-year colleges, class size (below 20 and over 50), percentage of lower-division courses taught by full-time faculty, six-year graduation rate, average time-to-degree, percentage of living alumni who donate annually, classroom and laboratory space utilization, percentage of state Education & General funding spent on instruction and academic support, percentage of state management standards met, percentage of professionally accredited programs, debt service-to-expenditure ratio, research and public service expenditures per full time faculty, and total student credit hours taught per full time equivalent faculty. This section of the report contains both current and historical information, and, where available, information on performance of peer institutions.

Institution-specific measures vary by institution and represent unique aspects of the mission that the college or university chose to highlight. Institution-specific measures also provide a richer context for understanding performance on the system-wide measures. For example, some doctoral research institutions have chosen to report national rankings for faculty research awards and honors. Comprehensive and liberal arts colleges with a primary focus on undergraduate education have reported results of student outcomes assessment or special opportunities for study abroad, service learning, or internship experiences. Several institutions have chosen to report on student access to technology.

Core competency reports that explore student general education assessments in written communication, technology/information literacy, quantitative reasoning, and scientific reasoning.

For a more comprehensive look at Virginia’s public colleges and universities, direct links are provided in the reports to the individual web sites for each institution.

Since the 2002 ROIE…

In-state undergraduate enrollment at the public four-years has increased by over 4,000 students (Fall 2001 – Fall 2003)

Enrollment at the VCCS and RBC increased by over 7,600 students

Over 50,000 baccalaureate degrees and 20,000 associate degrees have been awarded

Total (general funds and non-general funds) increased by 0.2% (despite enrollment increases).

Appropriations for in-state, undergraduates fell from $6,352 to $4,949 (a 22% decrease).

A Brief look at Some Findings…

Overall, retention rates of first-year students are stable, against past trends and peer institutions. The average is virtually unchanged since 2002 (82.3%, peers – 80.4%).

A number of institutions have continued to reduce the numbers of transfer students from VA public two-years. Conversely, others have continued to increase the number resulting in an overall increase from 4,994 to 5,077.

Despite resource and enrollment issues, many institutions have held the distribution of class sizes to 2002 levels. Not all have been so successful – there are fewer sections with fewer than 20 students.

Institutions that had been increasing the percentage of lower-division enrollments taught by full-time faculty lost ground in the last year especially.

Six-year graduation rates continue to improve across the institutions or at least hold constant. Overall, the average rate has increased to 57.5% against 58.1% for the peers.

Despite budget cuts and increased enrollments, average time-to-degree has remained very consistent at 4.6 years overall.

While some institutions have had increases, the percentage of alumni giving has fallen noticeably at a number of institutions.

Virginia institutions continue to spend significantly more than their peers as a percentage of E&G spending that goes to Instruction and Academic Support – 65% on average compared to 58%.

All institutions have met 100% of the Commonwealth’s management standards.

CNU is the lone institution to exceed the Commonwealth’s guideline of 7% debt service to expenditures ratio. All other institutions are below 6%.

12 of 15 of the public four-years still lag dramatically behind their peers in spending on research and public service per full-time faculty member $38,752 compared to $53,833 (up from $30, 621 and $45,642).

Due to enrollment growth and budget cuts, there is modest growth in the number of student credit hours taught by FTE faculty – up from 237 two years ago to 243.

Despite enrollment and funding pressures, VCCS and RBC remain very stable across most metrics.

However, the number of student credit hours taught by full-time faculty has continued to increase from 295 to 305 for VCCS; from 323 to 329 for RBC.

Core Competencies

Scientific Reasoning and Quantitative Analysis have been added to Writing and IT Literacy – near 100% compliance with submissions.

Institutional approaches to the development of the competencies and their assessment continue to vary tremendously.

Locally developed, general education assessment

Internal SCHEV committee to provide individual feedback

All but one institution met its definition of competent in the two areas

Conclusions

While it may be early in the undergraduate enrollment cycle, we are not seeing huge breakdowns in quality indicators reported in ROIE from the budget cuts.

We are seeing successes and improvements in some places, but we have lost ground in other areas. These areas may be those most sensitive to more immediate environmental changes.

Progress is being made, but at an uneven rate. This is most likely because of funding issues – the continued lack of funding at levels of base adequacy.

Looking to 2006

With ROIE in its third edition, it may be time to consider revising the system-wide measures to have a more conscious mix of short-term and long-term indicators.

Some measures can be dropped or treated as secondary indicators (i.e. Percent of Management Standards met).

Addition of a “System Report” to provide higher-level look at the effectiveness of the system.

The Reports of Institutional Effectiveness Prepared by SCHEV Research		Please take a moment and sign our guestbook and/or take our user survey.
State   Council   of   Higher   Education   for   Virginia 101 North Fourteenth Street, James Monroe Bldg. • Richmond, VA 23219 Tel: (804)225-2600  •  Fax: (804)225-2604  •  www.schev.edu