Department and Program Editor Roles
UB Faculty and staff who can edit department and program content
Users with a department editor role can edit the department pages and all program reports in the department. Users with a program editor role can only edit the program reports they are assigned to.
Department Editor Roles:
- Dean/Assoc. Dean,
- Assistant to the Chair, and
- University Assessment Council
Program Editor Roles:
- Program Coordinator
General Tips – Add or Modify an Editor:
If an editor is incorrectly listed or listed with the wrong role they can be modified by selecting the edit icon next to the editor:
- Option 1: Add an editor selected role – select the edit icon next to that editor and enter a UB IT Name into option 1 of the edit window.
- Option 2: Replace an existing editor with a new editor – select the edit icon next to that editor and enter a UB IT Name into option 2 of the edit window.
- Option 3: Remove an existing editor – select the edit icon next to that editor and choose option 3 of the edit window
– this will only remove the editor from the currently viewed department or program.
If a person does not currently have access to the AAPAR service, you will get an error instructing you to request the user be added to AAPAR. Submit a help request with the persons UB IT Name and which department or program editor role they should have.
Primary Assessed Program
The primary program sets the learning outcomes that all associated programs are assessed against.
The programs and advanced certificates listed in the AAPAR service have been approved through the appropriate undergraduate or graduate new and revised program approval process. Program and degree listings within AAPAR are informed by results of the program approval process.
Program Title and Degree Listing:
It is the policy of AAPAR to list programs and degrees using the official program title and degree abbreviation provided through SUNY SED.
Request A Change to Title or Degree:
If you find that a primary program or associated program has a title or degree that is listed incorrectly please use the edit icon next to that listing to submit a change request. All change requests will be confirmed with the appropriate resource and a staff member will respond with details.
Annual Program Reporting Requirements
About the AAPAR Service:
The Annual Academic Program Assessment database is available to faculty and staff who have assigned accounts. Contact your department chair or program assessment coordinator to be given editing capabilities in the service.
Annual assessment report deadline:
In order to ensure compliance with the Middle States Commission on Higher Education's accrediting standards, all academic programs should assess at least one learning outcome per year, with the entire set of program learning outcomes being assessed in a 3 to 5 year cycle.
Each year, the program coordinator must submit a report detailing the assessment activities for the year, including results and identified improvements. Annual assessment reports are due by Wednesday, September 30, 2020.
Program Reporting Work Reduction:
Regular review and update of program assessment reports will reduce the work load required to complete reports by the annual September due date. The recommended practice is to include review of program assessment reports at regular department chair and program coordination meetings.
A learning outcome is a statement that defines what knowledge and skills students will be assessed for when they complete a study - what students will know and be able to do because they have completed a program. While a student learning outcome (SLO) can be measured at many levels: course (CLO), program (PLO), and institutional (ILO); for the purposes of the AAPAR student learning outcomes are measured at the program level.
Systematic and periodic course and program assessment with thoughtful review of results helps ensure that the university is meeting its goals for academic excellence and a transformative student experience. The Office of Educational Effectiveness provides leadership and oversight for assessment of student learning at UB.
How the outcome will be measured. Who will be assessed, when, and how often: Student learning outcomes have been written for this program in terms of what students will know or be able to do as a result of successfully completing the program and they are clearly measurable. The PLOs are transparently consistent with the sponsoring department's/school's mission and goals. The statement of PLOs is publicly available and incorporated into program-specific publications, including web-based and hard copy, for students.
Statement of PLOs
Publishing PLOs (while part of the MSCHE requirements) is a recommended activity but is not yet tracked in the UB AAPAR. This requirement will become part of AAPAR reporting during the 2021-22 reporting cycle.
The level of performance which represents mastery and what proportion of students should be achieving mastery: Direct measures of learning are used to assess each PLO and these are consistently supplemented by indirect measures, as appropriate. Assessment of student learning outcomes takes place on an established schedule, and all PLOs are assessed over time.
Direct and Indirect Measures
Direct assessments collect evidence of student learning or achievement directly from students and the works that they submit such as:
- Capstone Course Evaluation;
- Course-Embedded Assessment;
- Faculty Generated Tests and Examinations that collect work samples;
- Commercially Produced Standardized Tests and Examinations;
- Portfolio Evaluation;
- Pre-Test/Post-Test Evaluation; and
- Video And Audio Evaluation.
Indirect assessments are analysis of reported perceptions, provided by students, faculty, third parties or other vehicles or proxies such as:
- Standardized testing;
- Student Surveying and Exit Interviewing;
- Co-Op and Internship Employer Surveying;
- Curriculum and Syllabus Analysis;
- Test and Assessment Vehicle Reports, and
- Course Evaluations.
The assessment results and what they show about student learning: Assessment results are collected and recorded in a place accessible by program faculty and administrators, and there is evidence that faculty are considering the results for the purpose of program modification and improvement. In addition, conclusions and recommendations based on consideration of assessment results are regularly shared with administrators and/or in public forums.
Public statement of PLOs and consistent consideration of assessment results by program faculty (while part of the MSCHE requirements) is not yet required in the UB AAPAR. This requirement will become part of AAPAR reporting during the 2021-22 reporting cycle. These activities could be reported in the PLO actions based on findings report field or in the planning section of the program assessment report.
Actions Based OnF indings
A brief description of actions that will be taken to improve the program or courses in the program based on the assessment findings: Assessment results (data) are regularly examined and there is evidence the results have served as the basis of program modifications designed to improve learning assessment results.
Actions Based on Findings
Each PLO assessment should include a statement of actions that will be taken to improve the program or courses in the program based on the assessment findings and should be linked to program planning activities reported in the planning section of the program assessment report.
UB ILO Mapping
A list of institutional learning outcomes (ILO) that map to this learning outcome: All program PLOs are linked to university ILOs and a clear and specific depiction of how the PLOs contribute to the fulfillment of ILOs exists. Appropriate detail about the ILOs linked to a PLO should be included in the methods and criteria sections for that PLO.
A list of courses from this program that directly address this learning outcome: A comprehensive curriculum map has been created showing clear linkages between all PLOs and programmatic requirements. The curriculum map also shows progressions (degrees of learning), as appropriate to the program, from introduction to mastery of PLOs and ties these progressions to successive course requirements. Appropriate detail about the courses linked to a PLO should be included in the methods and criteria sections for that PLO. In the 2020-21 cycle a dynamic curriculum map tool will be added to AAPAR which will use the ILO and course mapping fields completed in program outcomes to compile and display a program curriculum map that can be exported.
Course Mapping and Degrees of Learning
- Introduced: Learning outcome is introduced with some emphasis and a degree of learning intended to introduce basic knowledge, facts, concept, and/or ideas that support the learning outcome, but is not assessed.
- Reinforced: Learning outcome is reinforced with moderate emphasis, an opportunity to practice, and a degree of learning intended to strengthen and support the development of knowledge relevant to the learning outcome, but is not assessed.
- Mastered: Learning outcome is presented with a degree of learning intended to show achievement of the outcome, mastery at the senior or exit level, but is not assessed.
- Introduced and Assessed: Learning outcome is introduced with some emphasis and a degree of learning intended to introduce basic knowledge, facts, concept, and/or ideas that support the learning outcome, and assessment evidence is collected.
- Reinforced and Assessed: Learning outcome is reinforced with moderate emphasis, an opportunity to practice, and a degree of learning intended to strengthen and support the development of knowledge relevant to the learning outcome, and assessment evidence is collected.
- Mastered and Assessed: Learning outcome is presented with a degree of learning intended to show achievement of the outcome, mastery at the senior or exit level, and assessment evidence is collected.
Thinking About Course Mapping
Mapped courses should be selected based on curriculum design and at least one course should be mapped to each PLO. In some cases mapping all courses in a curriculum may not be desirable – instead mapping a representational set of courses that align with PLO assessment methods and criteria is recommended. Please contact your Assessment Council Representative and Educational Effectiveness Liaison to discuss your course mapping questions.
Program Assessment Frameworks
Assessment within an academic program can be easily organized and implemented by selecting one of the following assessment frameworks as a starting point for the design of the program assessment plan.
How do I get access to the Annual Academic Program Assessment
If a user is unable to access the login tab or after logging in does not see a department that should be assigned to them, the user should contact the department chair or program assessment coordinator to be provided with editing access.
Lexicon for Annual Academic Program Assessment
Data Fields in the interface
Each field in the AAPAR data has been configured to hold as concise a piece of data as possible so display of data in the user interface can be consistent and provide users with a common set of display and editing expectations.
The About Page
General details and information about the service are displayed here.
Program Report User Dashboard
The dashboard provides a set of user tools for accessing the AAPAR service. Listed on the dashboard are:
Service Access Details
- Service Alerts: details about the operational state of the AAPAR will be displayed here.
- Access Responsibility: details about conditions and responsibilities associated with being an editor in this service.
This section will display and provide updates on any help requests that have submitted through the interface. If no help requests have been submitted then this section will not display.
Units and Departments You have Access To
When the service is available this section displays units and departments which the user has been assigned program reports in. If a user does not see a department that should be assigned to them, then contact the department chair or program assessment coordinator to have the user added.
Program Report Department Homepage
The department homepage displays general department details and a listing of the programs within the department the user has been been assigned to. If a user does not see a program that should be assigned to them, then contact the department chair or program assessment coordinator to have the user added.
Assessment reporting is completed using the program report page. Each of the report areas and the fields within them are individually editable.
User Interface Icons
The following icons are tools that allow you to interact with page items in AAPAR reports.
- Display Item Selecting this icon will display the content below the page item the icon is displayed near.
- Hide Item Selecting this icon will hide the content displayed by selecting a 'Display Item' icon.
- Show Help for Item Selecting this icon will pop-up information window on the page item the icon appears near.
- Edit Item Selecting this icon will pop-up an editing window for the page item the icon appears near.
- Item Request Selecting this icon will pop-up a request window for the page item the icon appears near.
- Open Item Selecting this icon will open the page item in a new window or download it if it is a document.
- Command Button Selecting a command button will process the command displayed in the button oval.
Editing Content Tips
When composing new content or completing large updates:
- Navigate to current item to update in AAPAR.
- Copy item from AAPAR and paste into a text editor.
- Compose edits and new content in text editor.
- When composition/edits are complete copy content from text editor.
- Navigate to current item in AAPAR.
- Select the edit icon next to the item.
- Paste updated content into editing area.
- Submit edits.
- Follow any instructions.
When making short edits and grammar corrections:
- Navigate to current item to update in AAPAR.
- Make required edits.
- Select the edit icon next to the item.
- Submit edits.
- Follow any instructions.
- Connection to server times out or is otherwise lost. If you made edits, copy text from edit area and reload webpage. Otherwise reload webpage.
- Pasted text has illegal characters. These characters are replace with a place holder text. Locate place holder text and type in new characters.
- Pasted text is too long. Edit text to comply with length requirement.
Unit and Department Overview Statistics
Purpose of the overview statistics
The unit and department overview statistics is intended to provide a quick look at unit assessment report activity. Report editors can get a quick view of reporting levels for programs.
Number of Programs with Complete Assessment Reports
An assessment is considered complete when the following aspects of a report have content entered:
- The program has three or more learning outcomes listed.
- At least one learning outcome is completely assessed. An outcome is completely assessed when of the outcome dimensions are completed:
- Assessment Method;
- Success Criteria;
- Assessment Findings;
- Actions Based on Findings;
- UB ILO Mapping; and
- Course Mapping.
- The planning section is complete with content provided for:
- Program Plan;
- Budget Plan; and
- Assessment Plan.
Programs that are assessed using the learning outcomes of a primary program.
Types of associated programs listed in this tool are:
- Concentration: a specialized area of study consisted of a coordinated grouping of courses. May be SED-registered or informal
- Track: a sequence of study determined by either (1) admissions criteria (e.g. doctoral/post-MS or first professional/post-BS), or (2) certification goals (e.g. resulting in NYS teacher certification or not), or (3) length or intensity of program (e.g. part-time, full-time). May be SED-registered or informal.
- Format: An alternative format of study (e.g. hybrid, online). Must have distance-learning approval from SED.
- Combined: a program resulting in more than one degree (e.g. BA/MA or MA/CAS). May be SED-registered or locally-approved.
- UBTeach Combined: a program administered by the Graduate School of Education combining a content-oriented BA with a related EdM degree that results in NYS teacher certification.
- Non-degree Program: programs that do not result in a degree award (e.g. internships, residencies, continued education).
- Various Combined: programs that combine various degree programs in a customizable, ad-hoc manner (e.g. Medical Scientist Training program, Special Studies)
- Subject Area: programs that address a specific subject area of a primary program.
Not all programs listed will have associated programs. When associated programs are listed the listing is displayed in the following manner: Program Title - Degree Abbreviation - Additional Program Area and Type
If you find that a program listed as an associated program should be elevated to a primary program with its own set of learning outcomes, or you note that a program is missing from the associated listing, or should be demoted from a primary program to an associated program, please use the an associated program edit icon to request a change.
Printing and Exporting Reports
The AAPAR web-app is optimized to allow for printing/exporting using tools installed in all major web browsers.
You can print or export using the following steps:
- Login to the AAPAR
- Navigate to the department home page or program report you would like to print or export.
- Expand or collapse sections of the page, using the or icon, that you would like to print or export.
- From your web browsers main menu choose the 'Print' option or type the 'Print' command from your keyboard. In the print dialog box choose the printer you would like to print to or the “save as PDF” if you would like to export and choose a location to save the file to. If you would like to export while using a mobile or tablet OS you may have to choose the 'Print' option from the device and then 'long-press' over the 'print preview' of the page you would like to export to get the option to save as a PDF document.
Annual Academic Program Assessment Template
This document provides a template for the information required to be reported in the annual assessment report.
General Tips – We Recommend using Adobe Reader or Pro Desktop Application
- Download AAPAR Template from the buffalo.edu/uboee website to your desktop.
- Download Acrobat Reader or Pro from the getreader.com/ website and install on your computer.
- Rename the AAPAR Template with a name that identifies the program you are collecting content for.
- Fill Form with your content. As you work Adobe Reader or Pro desktop application will save your work.
- Close Form when you are finished with your work session.
– Acrobat will remind you to save and will maintain the fillable form and content.
– Reopening the form in Acrobat will allow you to continue to fill the form and edit previously saved edits.
Academic Program Learning Outcomes Assessment Rubric
This rubric, adapted from the SUNY Council on Assessment (SCoA) Academic Program Assessment Rubric, is used by the University Assessment Council to share feedback to academic departments undergoing Comprehensive Program Review. This rubric can also be used within departments and by program coordinators/directors to review their own assessment activities.
The AAPAR is provided as a place to report on program learning assessment activities and to manage clarifying updates to program learning outcomes and the dimensions by which they are measured. Substantial alterations that alter the learning outcomes of the program such that the program is no longer targeting or assessing the same learning should first go through the appropriate undergraduate or graduate program change processes. Please contact your University Assessment Council(UAC) Representative and Educational Effectiveness Liaison if you have questions about program or learning outcome alterations.