Summary of Conclusions and Findings

The Working Group on Using Assessment for Institutional Effectiveness (WGIE) found that the fundamental elements of Standard 7 are being met and the assessment processes in place are both effective and sustainable. The working group also found substantial evidence that assessment is now part of the culture of the University of Pittsburgh.

The WGIE report states that the University’s Planning and Budgeting System has clearly identified goals and processes that are broadly communicated. The system itself has been formally assessed and improved over time. The annual planning process is transparent, promoting a dialogue among the central administration; the individual responsibility centers; and the broader faculty, staff, and student communities. Through feedback and assessment, the annual planning process has been adapted over time to better serve both the University and the individual units.

Benchmarking at the University level is conducted in a systematic fashion, and schools and departments have increasingly incorporated internal and external benchmarking into their planning processes. Planning and benchmarking activities yield data that are meaningful and useful and have clearly impacted decisions and resource allocation. Specific planning, budgeting, and benchmarking activities have been designed to allow responsibility centers some flexibility in goals and processes to reflect their individual needs while at the same time providing a framework to ensure that unit activities align with overall University goals. A culture of assessment is clearly evident within the planning, budgeting, and benchmarking activities of the University of Pittsburgh.

The working group also found evidence of effective assessment in institution-wide infrastructure investment, as documented in the areas of information technology, facilities, the University Library System, international activities, and budget and finance. The University has articulated a low-cost, realtime, systematic culture of assessment within its regular information technology operations. The University also has effectively used assessment as a tool in facility planning for a number of years, as can be seen in the number of formal facility planning documents. An explicit commitment to assessment at every level, as well as a high level of sophistication in planning, has been demonstrated by the University Library System. In addition, the working group found that assessment in budget and finance is clearly useful, cost-effective, truthful, reasonably accurate, planned, ongoing, organized, and sustained. Throughout the WGIE report, specific suggestions or areas of improvement were noted along with a few broad suggestions:

• The annual planning process, while effective, can be resource-intensive for units to prepare. A well-designed online system could facilitate the task, although the diversity of relevant data across the many different units of the University makes it challenging to develop a single standardized reporting system.
• External benchmarking data can be quite valuable in terms of providing the information necessary for setting objectives and assessing progress, but there is some unevenness across the University in terms of the quality of the available data and the ease of gathering the data. Thus, it may be useful to examine benchmarking practices across the University to determine whether there are opportunities for improving the effectiveness of benchmarking.
• The University should continue to explore ways to assess faculty interest and involvement in research and other partnerships outside the United States, as called for in the international plan framework.
• The University should continue on its path of developing a robust financial data warehouse and using advanced analytical tools that ultimately will provide additional efficiency and speed for the administration as well as the unit levels.

The Working Group on Using Assessment to Improve the Student Experience (WGSE) reviewed the University’s processes for assessing student learning in undergraduate, graduate, and professional programs and for undergraduate general education (Standard 14); the group also reviewed the processes for assessing other aspects of the undergraduate student experience, including the effectiveness of assessment in the recruitment, retention, and graduation of undergraduate students as well as student satisfaction with the undergraduate experience (portions of Standards 8, 9, 11, and 12). The working group found evidence that assessment is now part of the culture of the University. Assessment has been integrated into the planning processes for all activities related to the student experience and into the Planning and Budgeting System. It offers as an example the Kenneth P. Dietrich School of Arts and Sciences, which uses governance mechanisms established in its bylaws to incorporate assessment. WGSE found that the assessment process used to assess student learning is consistent with the guidelines established by the Council of Deans and the standards established by the Middle States Commission on Higher Education. It concluded that assessment of student learning is appropriately done at the departmental or program level, where faculty with both the knowledge of the subject matter and regular contact with students are in the best position to explore assessment concerns; equally important, the faculty are in the best position to use the information gathered through assessment to improve the University’s educational offerings. Overall, WGSE found that the assessment of student learning processes is sound and effective.

The working group found that the admissions offices on all campuses use a number of assessment results to improve the targeted marketing of high-ability students and to increase the geographic diversity of competitive applicants, including the use of increased involvement of Pitt alumni, parents, and faculty. Finally, the working group considered the various  assessment initiatives and strategies used to improve other aspects of the student experience and concluded that the initiatives and strategies contributed to improvements in retention rates, graduation rates, and student satisfaction. The University’s ongoing assessment efforts also have allowed it to better match the attributes of its students to its mission and goals. WGSE summarizes its findings by saying that the “assessment of student learning and of the undergraduate student experience is well thought out, it’s effective, we use it when we plan changes, and it permeates the University. It is one integrated system in which everyone participates and in which responsibilities are charged where the programs are delivered.” All campuses have fairly well-developed processes in place to measure progress toward recruitment goals and to assess the effectiveness of recruitment strategies. The good practices have been instrumental in the success that the campuses have had in maintaining and expanding enrollments at a time when other institutions in Western Pennsylvania are experiencing declining enrollments. At the same time, these good assessment practices have assisted the campuses in better matching the qualifications of the students to the strengths of the campuses. The University could further build on these successes by pursuing the following suggestions:

• The University has developed many best practices in both assessment and programs; enhancing its process of sharing these across units and campuses could further develop the culture of assessment throughout the institution.
• Benchmarking data for units and programs allow them to better gauge progress against peers, so the University should continue efforts to identify and develop such data. The recent initiative to join the SERU consortium and Academic Analytics is a good step that should be supported.
• A data liaison from Computing Services and Systems Development could provide additional insight and support regarding the  University’s computing capabilities and help to make information even more accessible to deans and department chairs. Effective in fall 2011, the Provost appointed an executive data assessment liaison to become a member of the Enrollment Management Committee.
• Each campus is different, and what works for one campus might not work for others. Therefore recruitment efforts on the individual campuses may be enhanced by establishing a forum for sharing results of assessment of recruitment strategies.
• The University could increase centralized data collection efforts to provide useful information to all campuses. For example, when requesting information from the National Student Clearinghouse on the final enrollment of students who applied to the Pittsburgh campus, similar information could be collected for the regional campuses.