INSTITUTIONAL REPORT: CONTINUOUS IMPROVEMENT OPTION
Standard 2: Assessment System
The unit has an assessment system that collects and analyzes data on applicant qualifications,candidate and graduate performance, and unit operations to evaluate and improve the performance of candidates, the unit, and its programs.
2.1 How does the unit use its assessment system to improve candidate performance, program quality and unit operations?
The continuous assessment plan for the educator preparation program at Campbellsville University is the system by which candidates at the initial and continuing levels, all preparation programs, and the unit are evaluated. It serves as an internal quality control to ensure that candidates in the program develop the knowledge, skills and dispositions to have a positive impact on student learning. The major question driving the system is derived from the unit’s conceptual framework theme Empowerment for Learning: “How do we know if our candidates have a positive impact on student learning?” To answer this, data are collected from several internal and external assessments and used to improve candidate performance, program quality and unit operations.
The unit’s continuous assessment plan includes official, monitoring checkpoints, known as candidate assessment points or CAPs. These checkpoints enable the candidates and the unit to assess strengths and to identify growth areas. There are four CAPs (1, 2, 3, and 4) at the initial level of certification (undergraduate) and three CAPs (5, 6, and 7) at the advanced level. CAP 1 is intent to enter teacher education, CAP 2 is admission, CAP 3 is approval for student teaching and CAP 4 is exit from the undergraduate program. CAP 5 is admission for advanced programs, CAP 6 is mid-point check/candidacy and CAP 7 is exit from the graduate program.
Multiple criteria are designated at each CAP, based on varied combinations of formative and summative assessments that are aligned with the national and Kentucky Teacher Standards (KTS) or KTS for Interdisciplinary Early Childhood Education (KTS/IECE) or Interstate School Leaders Licensure Consortium Standards (ISLLC). As candidates matriculate through their respective programs, they must meet the expectations of each CAP. They submit CAP applications with all required documentation. Details about the specific CAP process, criteria and forms are included in the Continuous Assessment Plan. Faculty members of designated courses, during which candidates submit their CAP applications, serve as the respective CAP coordinators. These coordinators work closely with candidates, assisting them in completing their applications and offering suggestions for remediation, if necessary.
A major assessment for initial certification is the capstone portfolio required at CAP 3 and CAP 4 at the undergraduate level and CAP 7 at the graduate level. In 2011-2012, the unit adopted an electronic portfolio platform, LiveText, for collecting and scoring candidate work at the CAPs through common rubrics. LiveText also allows data analysis on scorer reliability. These data are entered into an Access® relational database which is linked to Campbellsville University’s main Jenzabar® data system. Common rubrics, adapted from those used for the Kentucky Teacher Internship Program (KTIP), are used to assess the candidates’ portfolios. Through use of these rubrics, faculty, cooperating teachers and university supervisors provide feedback to candidates on their performance in the classroom. Additionally, P-12 teachers and administrators evaluate CAP 4/7 portfolios using LiveText technology.
In addition to CAPs, candidates complete critical, course-embedded assessments throughout their program. Candidates develop skills in instruction, assessment and analysis of student learning through specific coursework as well as a continuum of field based experiences. These assessments are designed to assist candidates in meeting the KTS and consist of Teacher Performance Tasks (TPA tasks) adapted from those used in KTIP. Many of the tasks focus on student learning, such as Tasks C and J1, which involve an analysis of performance on individual lessons and the pre/post assessments of unit objectives. These tasks include individual student performance data on learning targets aligned to state standards (Kentucky Core Academic Standards and the P-12 Program of Study). Candidates are asked to analyze their data by gap groups and reflect on improvements in their instructional practices, identifying growth areas and professional development needed to improve in these areas. Candidates are required to maintain samples of student work and their analysis. At the advanced level, candidates implement a project in their P-12 classes to determine impact on student learning. Data are collected and presented in their Master’s action research project or the Rank 1 culminating project.
Faculty members use standard rubrics adapted from the KTIP Tasks to provide feedback to candidates as they complete course embedded assignments. These assessments are included in the Portfolio Development Plan and the rubrics have been developed in LiveText so professors and candidates can use the electronic portfolio system for these assignments. Candidates receive feedback concerning how well they are meeting the designated standards.
When candidates teach lessons, P-12 teachers or the education professor observe and evaluate their instruction through use of a common rubric. Often, debriefing sessions provide opportunities for feedback with the candidate. During student teaching, the cooperating teachers and the university supervisors also observe and evaluate candidates’ instruction using the same rubrics. Additionally, advisers assist candidates as they matriculate through their programs. Each term, during advising sessions, advisers and candidates discuss progress and upcoming CAPs.
For KTS 9, Professional Development, candidates are asked to self-assess their capabilities regarding all KTS and disposition(s) to develop a Pre-Professional Growth Plan (PPGP) which includes action plans and timelines for growth in identified areas. Candidates submit editions of their PPGPs at CAP 2, 3 and 4 at the initial level and at designated CAPs at the advanced level.
Program accountability occurs through multiple venues. Data from internal and external assessments on candidate performance serve as accountability measures providing longitudinal feedback regarding assessment of the program. Examples of internal assessments generating data include those associated with the CAPs (including the portfolios); examples of data from external sources include cooperating teacher evaluations, exit portfolio evaluations/interviews, Praxis II scores, New Teacher Survey conducted by the Kentucky Education Professional Standards Board (EPSB) and follow-up surveys of graduates and employers. All data, aggregated and summarized for annual retreats, are used to analyze overall program effectiveness and to address identified areas for growth. Data are also disaggregated by program, location and online to determine individual program quality (candidates pursuing the ESL endorsement are tracked through the CU ESLI program). One major outcome of each annual faculty retreat is the development of a Program Improvement Plan which includes several growth areas of focus, along with action plans, for each upcoming year.
To illustrate alignment with and integration of the unit’s conceptual framework and its theme ‘Empowerment for Learning’, an Empowerment Index is calculated to determine unit and program quality. The Empowerment Index is based on aggregated candidate performance data at program exit, CAP 4 at the undergraduate level and CAP 7 at the graduate level. The index involves an overall average of aggregated KTS data according to the three types of learning experiences in the empowerment theme—content, process and self-efficacy. The respective KTS, IECE or ISLLC standards are identified according to these experiences in the conceptual framework and in the continuous assessment plan. The critical assessments selected for the Empowerment Index at the undergraduate level currently consist of CAP 4 performance evaluations by the cooperating and supervising teachers during student teaching and KTS scores on the exit portfolio. Although initial calculations are based on these two major program assessments, it is anticipated that the Empowerment Index will evolve to include other major assessments, such as Praxis II scores. Similar calculations are underway for programs at the advanced level. Such an index provides an overall average of how candidates in the programs have progressed toward ‘being empowered’ to positively impact student learning.
Results of data analysis at annual retreats impact decisions regarding unit operations: governance, planning, budget, personnel, facilities, services and procedures. The Program Improvement Plans resulting from the analyses of data at retreats may necessitate changes in advising, revisions to CAPs and available resources. For example, one of the recent improvement goals called for increased focus on assessment. As a result, the unit purchased copies of Stiggin’s Classroom Assessment for Student Learning: Doing It Right - Using It Well for faculty and provided time during workshops to address critical assessment approaches. Another improvement plan called for an increased focus on reflection and necessitated changes in field hour guidelines, development of reflective questions for field hours, and development of pre-professional development (PPDs) workshops on reflection for students with mandated attendance. Finally, another improvement plan concerned improved Praxis II scores. For this focus, the NCATE Standard 2 committee developed a Praxis II policy determining when students take their Praxis II exams and providing a study preparation plan template.
Through regular analysis and review of all data from internal and external sources at annual retreats, goals are developed, implemented, monitored and updated. Such practice improves candidate performance, program quality and unit operations in an ongoing feedback loop. During annual retreats (retreat PowerPoint presentations), faculty analyze continuous assessment system data to develop and update annual improvement goals; the faculty also review goals at monthly faculty meetings. The improvement goals and results are also periodically evaluated by other stakeholder groups, such as the Teacher Education Committee, which includes arts and science faculty and students, and the Teacher Education Advisory Council or TEAC, which meets annually and consists of education faculty and public school practitioners.