OREGON STATE UNIVERSITY

APAA - Academic Programs

Welcome! APAA facilities and coordinates the preparation, oversight, and implementation of academic programs and courses.

Undergraduate Academic Program Review (UAPR)

APAA coordinates faculty/peer review of undergraduate academic programs.  The in-depth reviews occur on a cycle of once every ten years, with the action plan follow-up in year three. The review process is performed in partnership with the Faculty Senate Curriculum Council.

Graduate program reviews follow a similar schedule. Reviews are coordinated by the Graduate School and conducted in partnership with the OSU Graduate Council.

Policies and Procedures

Appendix I: Guidelines for Selection of External Reviewers

Appendix II: Guidelines for the Self-Study Report

Appendix III: Guidelines for Review Committee Report

Appendix IV: Sequence of Review Procedures

Appendix V: Sample Site Visit Agenda


Overview

OSU faculty has responsibility for the quality instructional programs at OSU. Academic Program Reviews provide the opportunity to review, evaluate, and plan in a deliberate and collegial setting and can serve as a productive mechanism for constructive change.

The objectives of these reviews are to evaluate the following areas for the ultimate purpose of improving the quality of undergraduate programs:

  • The focus of the academic program and its fit with the institutional mission and strategic direction;
  • The extent to which the program is evolving along national trends;
  • The adequacy of resources to support the program;
  • The learning environment and the extent to which learning outcomes are achieved;
  • The areas where OSU can further develop its strengths; and
  • Potential areas for collaboration and interdisciplinary projects/programs.

The process of these reviews are

  1. APAA initiates a schedule of reviews, notifying the chair/school director and college dean of programs proposed for review well in advance of the review schedule. Interdisciplinary baccalaureate programs fall within the purview of this policy; where these guidelines refer to "dean" or "department," or use other terms that do not fully apply to interdisciplinary programs, appropriate adjustments will be made by the Curriculum Council to assure a meaningful and efficient review.
  2. The APAA representative works with the department chair/school director and college dean and the chair of the Curriculum Council to establish the composition of the Review Committee, the timing of the review, and the content appropriate for the self-study report.
  3. The department chair/school director is responsible for preparing the departmental/school self-study and other materials appropriate for the review.
  4. Following review of the self-study, the Review Committee conducts a site visit.
  5. Based on the site visit and the materials presented in the self-study, the Review Committee submits an Evaluation Report to the Office of Academic Programs, Assessment and Accreditation.
  6. The Evaluation Report is then reviewed by the leadership of APAA, the college dean, and the department chair/school director. After factual information has been confirmed, the report is formally reviewed by the Review Committee, accepted by the Curriculum Council, and forwarded to the Provost.
  7. Subsequently, the Provost, APAA leadership, chair of the Curriculum Council, chair of the Review Committee, the college dean, and the department chair/school director meet to agree upon an appropriate plan of implementation. In preparation for this meeting, the chair/school director will prepare an Action Plan based on the recommendations contained in the Evaluation Report.  The Action Plan will have been reviewed and agreed to by the dean before the meeting with the Provost.
  8. An academic program review follow-up meeting will be scheduled by APAA three academic years following the meeting with the Provost.  The purpose of this meeting, comprised of the same individuals who met with the Provost previously, will be to evaluate how successful the academic unit has been in implementing the recommendations contained in the Evaluation Report and subsequently incorporated into the academic unit's Action Plan.

| Back to Table of Contents |

Professionally Accredited Undergraduate Instructional Programs

Undergraduate academic programs that are subject to professional accreditation are not required to participate OSU academic program review (this includes Pharmacy and Veterinary Medicine). The professional accreditation site visit, self study report, and evaluation report will take the place of the external review component called for in this policy of the Curriculum Council. Self-studies will be made available to APAA and the Curriculum Council at the same time that they are submitted to the accrediting body. Reports should be submitted to the APAA SharePoint website (link).

Questions about access to the SharePoint website should be directed to: Beverly Dirks or Stefani Dawn.

| Back to Table of Contents |

Timing of Reviews

Undergraduate academic program reviews must be scheduled carefully and well in advance. Advance notice is provided to the chair/director of the academic program and the responsible college dean that a review is being scheduled. Coordination among these individuals is essential to assure scheduling of all Review Committee members, especially the external members. Tentative time lines should be agreed upon, with consideration of the impact of summer activities on scheduling. It is the intention of APAA and the Curriculum Council, working with the Graduate Council, to maintain, at all times, a tentative five-year schedule that lists the review of undergraduate programs. A five year calendar is available at (link)

| Back to Table of Contents |

Coordination of Graduate and Undergraduate In-Depth Reviews

To the extent consistent with the objectives and policies of the Curriculum Council and the Graduate Council, the in-depth reviews of undergraduate and graduate programs will be coordinated. The Dean of the Graduate School and the APAA leadership will meet periodically to discuss opportunities for concurrent review of programs with undergraduate and graduate components. Such integration fosters recognition and analysis of interdependencies between the graduate and the undergraduate program(s) within an academic unit. When a concurrent (joint) review is agreed upon, units should prepare a single, integrated self-study, which fully responds to the questions, informational requests, and critical analyses specified in the Undergraduate Program Reviews Guidelines (link) and in the  "Policies and Procedures for the Review of Graduate Programs."

The complexities and the degree of integration of the undergraduate and the graduate programs in a concurrent review will determine whether the external review should be conducted by a single committee, a joint committee acting at times together and at times separately, or by separate review committees. The Dean of the Graduate School and APAA leadership will make this determination in consultation with their respective councils and the leadership of the academic program unit involved. Should a single committee prove advisable, membership will still meet the requirements specified in the policies of the two councils. The Dean of the Graduate School and APAA leadership will jointly appoint the chair and will specify the format for a joint Evaluation Report that meets the requirements of both the Graduate Council and the Curriculum Council.

| Back to Table of Contents |

The Undergraduate Academic Program Review Committee

Oversight of the composition of the review committee belongs to the APAA leadership, with input from the Curriculum Council and the unit undergoing review.

At the minimum, the Undergraduate Academic Program Review (UAPR) Committee is to be comprised of:

  • One member of the Curriculum Council (who will serve as the Review Committee chair).
    • Assignment of Curriculum Council member(s) to the Review Committee is the responsibility of the chair of the Curriculum Council;
  • One-to-two additional members of the Faculty (who may or may not be members of the Curriculum Council);
    • On-campus members of the Review Committee should be from a college other than that of the program under review and may be chosen from a department, school, or college with strong undergraduate instruction connections to the program under review: e.g., from an academic unit that requires course work in the undergraduate academic unit under review.
    • While the Review Committee members may vary in their familiarity with the subject matter of the program, all should be well versed in undergraduate education.
  • A minimum of two, and preferably three, disciplinary peers external to OSU.   No more than one external reviewer can be from inside the state of Oregon;
    • The external reviewers should be a highly knowledgeable and reputable leader in the field under review (see Appendix I). Suggestions for external reviewers are provided by the unit undergoing review. The APAA leadership contacts and finalizes the external reviewers.
  • Additional participants may be desirable, for example, the inclusion of a representative of the employing profession may be particularly helpful;
    • Suggestions for additional reviewers are provided by the unit undergoing review. The APAA leadership contacts and finalizes the external reviewers.
  • APAA leadership accompanies the Review Committee during the site visit, but is not an official member of the Committee.

Travel for External Reviewers

Travel arrangements and expenses for the external reviewers, including travel, lodging, meals, and any honorarium, are the responsibility of the college dean or deans whose undergraduate academic program is being reviewed.

| Back to Table of Contents |

Self-Study Report

The primary benefit of an academic program review process lies in the opportunity for self-analysis and the use of this analysis (along with the feedback provided by the Review Committee) in subsequent program improvement. In order to derive maximum benefits from the external review effort, it is crucial that the self-study be approached as a process of communication and planning rather than as an exercise in generating paper.

The self-study should be prepared collectively by the faculty and administration of the academic program unit, and should be an opportunity for colleagues to review departmental, school, or college accomplishments, to share concerns and aspirations, and to develop a long-term vision and strategies for improvement and enhancement. The Self-Study Report simply documents the matters considered during the self-study and describes the conclusions emerging from that process.

Appendix II provides a suggested outline for the Self-Study Report and additional detail on the type of data to be included. The report should begin by presenting the review context and go on to describe the mission statement of the academic unit and its relationship to the mission of the school/college and the University. The methods and extent to which the academic unit collects evidence of student learning should be discussed. The major short- and long-term goals of the academic unit and the undergraduate degree program should be presented. Issues, challenges, and opportunities confronting the unit/program should also be described.

All undergraduate programs should regularly collect and monitor the data needed in the Self-Study Report. In addition, the Institutional Research Office can assist in providing statistical information to units preparing the Self-Study Report. The final report should present assessment data, along with narrative interpretation, in the following areas:

  1. Input Assessment - the adequacy of the infrastructure supporting the program (quality of students; quality of the curriculum; physical facilities; accessibility of courses, technology, and physical facilities; resources; personnel; faculty profile; general infrastructure).
  2. Program Performance - the quality of the undergraduate program including the breadth and depth of its capacity to fulfill its mission and goals (student learning outcomes and activities to measure achievement of these outcomes; student and faculty performance metrics; recruitment and retention of students; affirmative action; student persistence (retention); honors and awards; academic honesty; research and scholarly activity; and metrics for operational effectiveness).
  3. Evaluative and Learning Outcomes Assessment - the quality of the outcomes that result from the existence of the undergraduate program (placement of graduates, student and alumni satisfaction level, employer assessment of program quality, and program improvements resulting from assessment feedback).

Questionnaires are often useful in soliciting impressions of academic program strengths and weaknesses. If used, questionnaires should be anonymous, and designed and administered with care. [Note:  The OSU Survey Research Center can help provide assistance regarding the design and application of a survey as well as an analysis of the survey responses.]

Departments/schools may offer multiple baccalaureate majors, minors, and options and may contribute to interdisciplinary majors and minors, the Honors College, general education (Baccalaureate Core), and provide service courses for other majors. APAA leadership, obtaining guidance from the Curriculum Council, will work with the program leader to clearly define the scope of undergraduate instruction to be considered in the review.

Nine (9) copies of the self-study are to be submitted to APAA at least three weeks in advance of the scheduled site visit.

| Back to Table of Contents |

Pre-review Meeting

Prior to the site visit, APAA leadership will meet with the on-campus members of the Review Committee. The Self-Study Review document will be reviewed, and the Review Committee will receive advice on procedures and significant issues to be examined during the site visit. During this orientation, individual Review Committee members may be assigned responsibility for specific topics of inquiry and the agenda of the on-site visit will be reviewed. If the college dean has requested that attention be given to specific aspects of the undergraduate program, these issues will be presented for incorporation.

| Back to Table of Contents |

Site Visit

Following review of the Self-Study Report, the Review Committee conducts a site visit in the department/school/college. The site visit is scheduled as follows:

  •  

Responsibility for the schedule and agenda of the site visit should be assumed jointly by the chair/director of the program being reviewed and the chair of the Review Committee.

The agenda should include a facility tour and interview meetings with:

  • The college dean(s)
  • The department chair/school director
  • The program director (if different from the department chair/school director)
  • Faculty
  • Staff
  • Students
  • Time should also be arranged for any faculty or staff member or undergraduate student who wishes to have a private meeting with the Review Committee

Report-back meeting with:

  • The dean(s), department chair/school director, program director (if different from the department chair/school director), Senior Vice Provost for Academic Affairs

Confidentiality must be assured in these discussions. Additional materials may be requested and reviewed at this time, if appropriate. 

A sample site visit agenda is provided in Appendix V.

| Back to Table of Contents |

Review Committee Report

The Review Committee will complete a draft of an Evaluation Report on the evening of the full day visit (see Appendix III). It is essential that all Committee members agree upon the structure and nature of the Evaluation Report, and the responsibility for preparation of its various sections. The Review Committee will identify the lead report writer, who will assemble all parts of the report and generate a final version.

This Evaluation Report provides a review of the Committee findings, along with recommendations for undergraduate program improvements and enhancements.

  • No format is specified for this report, but it should address the quality, vitality, and direction of the undergraduate academic program being reviewed.
  • The Evaluation Report should comment on the quality of the students in the program, the productivity of the faculty, the leadership of the program, and the continued relevance of the program.
  • The Evaluation Report should include recommendations concerning the future of the department's programs, structure, and activities. These could range from a recommendation to discontinue a program to a recommendation to greatly expand the scope of the department and its academic programs.
  • A recommendation might be to change the direction, structure, or activities of the department's undergraduate program in order to improve its quality, to increase its effectiveness, or to utilize the University's resources more efficiently.

The draft version of the Evaluation Report will be provided to the department chair/school director, program director, and the college dean to assess the factual information contained in the report. After factual information has been confirmed, the Evaluation Report will be formally reviewed by the Review Committee ot generate a final draft.

A final draft of the Review Committee's Evaluation Report should be completed within 15 days of the site visit and circulated to Review Committee members for their input. The final draft is submitted to APAA and the Curriculum Council Chair no more than 30 days after the site visit.  

The Curriculum Council chair will arrange for the Evaluation Report to be presented at a regular meeting of the Curriculum Council. The program chair/director and college dean may be invited to that meeting to review factual issues in the report. The Curriculum Council, after considering the report and any responses, will make recommendations to the APAA leadership, the Faculty Senate, Executive Committee or other appropriate individuals or groups. After the Curriculum Council has "accepted" the Evaluation Report and attached any supplement comments regarding the review and the report, a copy will be forwarded to the Provost.

| Back to Table of Contents |

Implementation and Follow-up

Completion of the undergraduate academic program review should lead to positive actions to enhance program quality. Accordingly, there must be action to implement each of the Review Committee's recommendations. Following a careful review of the Self-Study Report and the Evaluation Report, the Provost or Senior Vice Provost will convene a meeting that will include the APAA leadership, chair of Curriculum Council, the college dean, and the department chair/school director to discuss the recommendations and findings of the program review. This review should result in an Action Plan prepared by academic unit chair or director and agreed to by the college dean. The Action Plan will identify specific actions to be taken, by whom, and in what time frame relative to the recommendations in the Evaluation Report.

This Action Plan must be agreed upon by the Provost or Senior Vice Provost, the college dean, the Chair of the Curriculum Council, and the departmental administrator. It becomes a part of the review record and is used by the Curriculum Council to guide follow-up activities.  APAA will call a meeting three years following the meeting with the Provost to review the progress that has been made (or not made) with regard to the implementation of the Action Plan.

The sequence of events discussed above is summarized in Appendix IV.

| Back to Table of Contents |

Appendix I: Guidelines for Selection of External Reviewers

  1. External reviewers should be respected peers with proficiency in the areas of specialization that are important to the unit being reviewed.
  2. Reviewers should be experienced academics who understand university operations and undergraduate education, who are able to realistically evaluate the unit's operations, plans for growth and development, professional activities of faculty members, and who can assess the unit's strengths and weaknesses relative to similar units at other comparable higher education institutions.
  3. If an undergraduate program review is being conducted in conjunction with another review (e.g., professional accreditation or graduate academic program review), the reviewers should be aware of the objectives of the undergraduate program review as well as those of the related review.
  4. It is preferable to avoid former mentors or close personal friends of faculty members, former employees or individuals who have applied or are likely to apply for a position at OSU, or individuals from institutions substantially different in character from OSU who will be less likely to understand specific circumstances here.
  5. The long-term viability of the unit and the credibility of the undergraduate academic program review will be enhanced by identifying thoughtful, knowledgeable, and objective external reviewers.
  6. When nominating individuals to serve as external reviewers, the dean and/or department chair/school director should provide the Office of Academic Programs, Assessment, and Accreditation with names, highest degree, professional or academic title, current and complete contact information (including mailing address, telephone number, and email address), and a brief statement of the individual's qualifications (several sentences summarizing the individual's expertise and any previous experience with the unit or faculty).

| Back to Table of Contents |

Appendix II: Guidelines for the Self-Study Report

Sample Outline for Undergraduate Academic Program Reviews:

I.    Introduction and Context (include process by which self-study was written)
II.   Mission Statement and Goals
III.  Input Assessment
IV.   Program Performance
V.    Evaluative and Learning Outcomes Assessment
VI.  Trends and Forecasts
VII. Summary

This outline represents the content that is viewed to be essential to Self-Study Report. Additional information can be included if it will enhance the effectiveness of the presentation. Materials that do not relate to the objectives of the program review process should not be included.

Suggested Metrics for Undergraduate Academic Program Reviews:

Input Assessment

  • Student Admission Information - incoming high school GPA of admitted student and of matriculated students; SAT scores, TOEFL (where applicable), comparison to national pools; enrollment changes and trends  during the last 10 years
  • Selectivity - applicant/admit ratio; applicant/matriculation ratio; departmental admission criteria, demand (application, acceptance, and yield)
  • Enrollment and Demographics - number of students by degree sought, age, gender, ethnicity, residency, other relevant demographics
  • Faculty characteristics (head count, FTE, gender, ethnicity, academic rank/tenure status)
  • Ratios - student/faculty ratio; advisor/student ratio
  • Facilities - utilization of space (including lecture rooms, workshops, labs, studio, offices, etc), e.g. capacity, responsibility for maintenance, etc.; computing and technology (accessibility of technology), etc.; accessibility of technology and physical facilities
  • Budget - expenditures (teaching, research, salaries vs. supplies, etc.), revenue (federal/state/private, grants); indicate by actual and percent how budget has changed over the past 10 years
  • Quality of Curriculum - list course offerings, core curriculum requirements per major, opportunity for non-traditional instruction (e.g. distance education); Honors Miscellaneous--college course offerings; opportunity for student mentoring; participation in the Baccalaureate Core
  • Issues - restrictions on enrollment and current capacity, etc.
  • Organization Chart: school/department/program and its relation to the college and institution); other infrastructure issues

Program Performance

  • Program recruitment and retention of students - demographic changes (success at attracting underrepresented groups); first-to-second year freshman persistence (including six-year trend)
  • Student performance metrics - pass rates, GPA, failure/academic suspension rate
  • Other evidence of student learning - capstone courses, senior thesis/poster/presentation; internship performance; student self-assessment
  • Student involvement - in research and internships; awards and honors
  • Assessment - student learning outcomes assesssment results and impacts on the program
  • Credit hour production - student credit hour analysis; enrollment of students outside of major (service courses)
  • Faculty - performance metrics (citation indexing, awards and honors)
  • Electronic student evaluation of teaching (aggregate)
  • Operational effectiveness - service and supply expenditures; FTE in administration
  • Capacity and access to classes

Outcomes

  • Graduation rates - number of graduates for last six years (graduation rates by ethnicity)
  • Professional viability of graduates - % employed within one year of graduation, % employed in position directly related to degree, type of positions held by graduates
  • Student and alumni satisfaction level
  • Employer assessment of quality of graduates
  • National program rankings
  • Student performance on licensure/certificate/professional exams, where applicable (compared to national data)
  • Program improvements as a result of learning outcomes assessment activities

| Back to Table of Contents |

Appendix III: Guidelines for Review Committee Report

1. Summary of Findings and Recommendations
This section serves as an executive summary of the report. A narrative style is common, but a numerical listing of key issues and findings may be useful. It also includes and summarizes all the recommendations found in the body of the main report. They should be organized by major category, preferably following the same organization as the main report.

2. Detailed Findings
This is the main body of the report. It highlights the strengths and weaknesses of the program. It provides the details of the review findings and provides the basis for the recommendations. Specific recommendations are often interspersed throughout the narrative of the report, but should be highlighted in some manner so they can easily be identified. The subsections of the report will vary depending upon the unit being reviewed. There is no set format for this report, but a typical report might include:

Introduction
Specify the objectives of the review, the participants, the order of events, and the organization of the report.

Undergraduate Teaching and Advising
Review course offerings, teaching loads, curriculum, advising loads, the characteristics of the undergraduate program, methods and outcomes of assessment of student learning, and stature.

Faculty
Review size, diversity, and productivity of faculty and their involvement in undergraduate education programs of the university. Assess the use of graduates in the undergraduate instructional programs.

Students
Review the quality and quantity of students, recruitment and retention procedures, graduate rates, and post-graduate placement.

Facilities
Review the amount and quality of space and facilities that pertain to undergraduate education, including accessibility.

Administration
Review the internal governance of the unit, including committee structure, interactions between the administrator(s) and faculty and students, student involvement in governance, and support staff functions.

Other

| Back to Table of Contents |

Appendix IV: Sequence of Review Procedures

  • Review requested in appropriate time frame (approximately ten years after last review)
  • College dean and program administrator notified; Self-Study Report requested
  • Review Committee appointed
  • Site-visit date established and scheduled to include concurrent external reviews (if appropriate)
  • Self-Study Report received by Office of Academic Programs, Assessment and Accreditation at least two weeks prior to site visit; report distributed to Review Committee members
  • Review Committee meets with the leadership of Academic Programgs, Assessment, and Accreditation to discuss agenda and review materials
  • Site visit with administrators, faculty, graduate students, staff
  • Preparation of Review Committee Report
  • Reviewer Report and Self Study are presented to Curriculum Council with comments by program administrator and college dean
  • Review and acceptance of the Self Study and Reviewer Report by the Curriculum Council with appropriate recommendations and responses
  • Academic unit prepares Action Plan based on recommendation contained in the Evluation Report
  • Reviewer Report and Action Plan are forwarded to the Provost
  • Provost calls meeting with the leadership of Academic Programs, college dean, department chair/school director, chair of Curriculum Council, and chair of the Review Committee to go over the Evaluation Report and agree to the proposed Action Plan.
  • Follow-up review in three years following the meeting with the Provost with the same participants to assure implementation.

| Back to Table of Contents |

Appendix V: Sample Site Visit Agenda

Sample Site Visit Agenda [pdf]

| Back to Table of Contents |