CAEP 4 Annual Reporting Measures

CAEP Accountability Measures

Measure 1:
Completer Effectiveness (R4.1)

 
 

Initial Programs

The EPP obtained i-Ready Reading and Math benchmark data from one county tied to 12 recent program completers currently teaching in grades 1-5 and a second county with 12 recent program completers teaching in grades kindergarten through grade 9. The presented data represents assessment results from two diagnostic assessments and reflects school schedules and structure changes due to the COVID-19 pandemic. The counties went fully remote in March 2020 and did not permit new instruction to occur after transitioning to digital delivery due to inequities in reliable internet access across the state. When students returned in Fall 2020, the counties operated under a 2-day in-person and 3-day distance model and also gave students the option to do fully remote learning. This schedule continued throughout the year. Additionally, many schools dealt with numerous full closures for various periods due to quarantines, fires, and flooding. Students also had the option to take the benchmark tests at home. Due to these circumstances, the reliability and validity of the data cannot be assumed as an accurate representation of completer impact in normal circumstances.  

I-Ready Math data from County One reflect that an average of 21% of students in classrooms of EPP completers scored “On or Above Grade Level” on these benchmarks with a range of 6% to 44%. Data reflect that an average of 41% of students in classrooms of EPP completers scored “One Grade Level Below” with a range of 20% to 91%. “Two or More Grade Levels Below” represented an average of 31% of students with the range of 0% to 60%. On average, 7% of students did not complete the assessment, ranging from 0% to 24% across classrooms.  

I-Ready Reading data from County One reflect an average of 21% of students scored “On or Above Grade Level” with a range of 0% to 44%. Data indicates that an average of 34% of students scored “One Grade Level Below” with a range of 11% to 82%. “Two or More Grade Levels Below” represented 30% of students with a range of 0% to 67%, with 14% of students not completing the assessment with a range of 0% - 46%.

I-Ready Benchmark data from County Two reflect an average of 41% of students scored “On or Above Grade Level” with a range of 24% to 60%. Data indicates that an average of 35% of students scored “One Grade Level Below” with the range of 11% to 72%. “Two or More Grade Levels Below” represented 31% of students with a range of 0% to 79%.

To assess completer effectiveness in applying professional knowledge, skills, and dispositions, the EPP asked mentors and administrators to complete classroom observations of completers. The EPP Observation Form includes ten competencies with completers rated by employers on each competency as Distinguished (4), Proficient (3), Basic (2), or Unsatisfactory (1). The form collectively includes competencies across all four of the West Virginia Professional Teaching Standards (WVPTS) and the four InTASC categories and is also aligned to CAEP standards. Mentors or school administrators completed the Observation Form for 29 EPP completers representative of 9 content areas. The overall mean was 3.12 on all competencies, ranging from 2.9 to 3.38. The only outlier was one Unsatisfactory score on “Plans, prepares, and implements standards-based learning experiences that make the content matter meaningful for students.” The competency with the lowest average score (2.9) was “Demonstrates knowledge of the content and its inter-relatedness within and across disciplines,” and the competency with the highest average score (3.38) was “Creates a positive, supportive environment that manages student behavior and emphasizes the importance of learning.” The competencies at the lowest range and the highest range indicate that our EPP completers can manage and facilitate a positive learning environment, but they need intentional opportunities in identifying and designing cross-disciplinary connections.

EPP Employer Observation Data 2020-21

EPP Observation Form

i-Ready and Benchmark Assessment Data 2020-21

Measure 2:
Satisfaction of Employers and Stakeholder Involvement (R4.2|R5.3|RA4.1)

 
 

Initial Programs

The EPP Employer Survey (Standard R4.2) collected information on supervisor evaluations of new teacher performance. The EPP Employer Survey competencies are explicitly aligned to CAEP, InTASC, and the WVPTS and reflect the shared values and beliefs of the EPP. The survey includes four separate sections that match the four WVPTS: Curriculum and Planning, the Learner and the Learning Environment, Teaching, and Professional Responsibilities for Self-Renewal, School, and Community. Each competency has a stem which reads, “The recent FSU graduates in my building are well prepared to...” The employer rated each competency on the survey via a Likert scale: Strongly Agree (4), Agree (3), Disagree (2), or Strongly Disagree (1). The CAEP Coordinator and co-coordinators distributed an Employee Survey to school administrators in four local school districts where one or more of our program completers are employed. Nine administrators from three of the four school districts completed the survey. The overall mean for the four WVPTS sections was 3.61, ranging from 3.42 to 3.86. Overall survey ratings suggest that employers are satisfied with the EPP graduates’ preparation for their assigned responsibilities in working with K-12 students.

With the implementation of the yearlong residency initiative that was mandated by West Virginia Board of Education Policy 5100 for all EPPs by Fall 2021, stakeholder involvement was a priority. Teachers and administrators from the EPP’s professional development school (PDS) network were involved throughout the curriculum development process and plans for implementation. Specifically, EPP faculty and staff from the PDS office and teachers and administrators from the PDS network met to discuss macrolevel changes needed to implement the residency model in the EPPs undergraduate elementary education program. These meetings resulted in the formation of a summer workgroup. In Summer 2021, that workgroup—composed of EPP faculty, staff, teachers, and administrators from four school districts in the PDS network—collaborated to co-design the field experience checklists for Residency 1 and Residency 2/student teaching. Co-teaching is an essential, connective strand of the residency model. This summer workgroup aligned the assignments within and across the Residency 1 and Residency 2 field experiences, so co-teaching opportunities were purposefully included. This purposeful placement of co-teaching opportunities led to other needed revisions in the EPP program, including restructuring the lesson plan format to include the space to articulate co-teaching strategies. The EPP faculty also collaborated to sequence assessments and learning opportunities in the two courses elementary education majors take during their Residency 1 semester.

EPP Employer Survey 2020-21

EPP Employer Survey - Completers F20-S21

EPP Professional Development Schools - Coordinator Meeting Minutes S21

Summer Work Group - PDS Partnership Meeting Agendas - Summer 2021

Advanced Programs

Stakeholder involvement has been essential to planning the Educational Leadership curriculum. At the start of the rollout of this Advanced level program (with an option for certification), individuals from across the professional development school network co-designed course proposals based on their level of expertise. After the proposals for courses were finalized, the stakeholders met to vote on the final curriculum proposals for the superintendent certificate and principal certificate tracks. The stakeholder group also made plans to market the program within the professional development school network. Due to the barriers that COVID-19 created from our K-12 partners and EPP, our Reading Specialist Advisory Board did not meet, which impacted the collection of Employer Survey data. Data will be collected in Spring 2022.

Educational Leadership Advisory Council Minutes - October 2020

Measure 3:
Candidate Competency at Completion (R3.3)

 
 

Initial Programs

The EPP tracks the number of completers of Initial programs as defined by CAEP through our Certification Officer and the Office of Institutional Effectiveness and Strategic Operations. Initial certification numbers represent all candidates completing the undergraduate education programs, MAT graduate programs, and the M.Ed. in Multi-Categorical Special Education with Autism graduate program. 

At the completion of the program, the EPP documents that candidates achieve a high standard for content knowledge and can teach effectively to promote student learning and development. The EPP demonstrates that candidates meet the standard for high academic achievement and ability through the Student Teaching Assessment (STA), Praxis II Content Area exams, and Principles of Learning and Teaching Praxis II exams. In addition, during student teaching, candidates design and implement an action research project to demonstrate their impact on student learning and present findings to faculty and peers.

Candidates enrolled in Initial programs complete 16 weeks of student teaching at West Virginia required developmental levels and in all teaching specializations in which they expect to be licensed. During the student teaching experience, candidates in the undergraduate and MAT programs demonstrate the ability to teach college and career-ready standards in lesson and unit plans and are evaluated in each placement using the Student Teaching Assessment (STA). The STA is aligned with the West Virginia Professional Teaching Standards (WVPTS), the Council for the Accreditation of Educator Preparation (CAEP) standards, and the Interstate Teacher Assessment and Support Consortium (InTASC) standards. Candidates are assessed on four standards that include 20 specific competencies: Standard 1: Curriculum and Planning; Standard 2: The Learner and the Learning Environment; Standard 3: Teaching; and Standard 4: Professional Responsibilities for Self-Renewal, School, and Community.  

Data from Fall 2020 (n=20) and Spring 2021 (n=19) indicate that candidates performed well on all standards, with average scores on standards ranging from 3.69 out of 4.00 on Standard 2 in Spring 2021 to 3.85 out of 4.00 on Standard 2 in Fall 2020. Review of the competencies for each standard indicates that average scores were above the proficient (3) level for all standard competencies. Strengths include Competency 2F: Organizing the Learning Environment (Fall 2020: 3.99 out of 4.00; Spring 2021: 3.91 out of 4.00) and Competency 3B: Communicating with Students (Fall 2020: 4.00/4.00; Spring 2021: 3.82 out of 4.00). Relative weaknesses include Competency 3C: Questioning and Discussion Techniques (Fall 2020: 3.18 out of 4.00) and Competency 1B: Content Pedagogy (Fall 2020: 3.49 out of 4.00). Average candidate scores were 3.50 out of 4.00 or higher on 18 out of 20 competencies in Fall 2020 and 20 out of 20 competencies in Spring 2021, providing strong evidence that candidates meet EPP and West Virginia Department of Education expectations at program completion.

Analysis of data from the action research project suggests that candidates provide effective instruction through the application of content knowledge and pedagogical skills, and they positively impact student learning. Candidates in undergraduate elementary and secondary programs performed well on the final report and presentation with mean scores of above 90% on the Fall 2020 final report and presentation. Scores were slightly lower in Spring 2021, with a final report mean score of 88% on the report and 90.6% on the presentation. In Fall 2020, candidates in the MAT program scored above the proficient level with a 3.09 out of 4.00 average on all rubric criteria on the report and a 3.0 out of 4.00 average on the presentation. Only one student completed the action research project Spring 2021, but scores were high for that student with a mean of 3.73 out of 4.00 on the action research project and a 4.00 out of 4.00 on the presentation.

Overall, Praxis test data from Fall 2021 and Spring 2022 indicates that most candidates in the Initial programs are meeting EPP expectations. An analysis of tests taken by candidates in Initial level programs shows that 13 out of 18 Praxis tests or subtests had a pass rate of 80% or higher, with nine tests at a 90% or higher pass rate. However, the pass rate did not meet the 80% criteria on two elementary subtests and three Praxis content tests taken by candidates in Initial programs. The lower pass rate was due to repeated tests by a small number of test-takers and a small number of test-takers who did not pass.  For example, two out of three content tests with a pass rate below 80% had only one test-taker. While EPP pass rates were below the 80% pass rate criteria on the elementary science and social studies subtests, they were above the state average pass rates of 59% on the elementary science subtest (EPP=75%) and 48% on the social studies subtest (EPP=53%). However, faculty recognized that the EPP needed to support candidates who struggled to pass these elementary subtests and added a class for content test preparation. In addition, an online module was made available to all candidates with test preparation materials.         

Student Teacher Assessment F20-S21 - Standards 1-4

Praxis Test Pass Rates - Initial Programs 2020-21

Action Research Final Report Directions

Action Research Rubrics - Final Report and Presentation

Action Research Data - Initial Programs F20-S21

Advanced Programs

The Advanced programs include candidates completing the M.Ed. in Reading Specialist program and the M.Ed. in Educational Leadership. At the completion of the program, the EPP documents that candidates in the Reading Specialist program achieve a high standard for content knowledge and can teach literacy skills effectively to promote student learning and development. The EPP documents that candidates in the Educational Leadership program obtain the professional knowledge and skills needed to promote the success and well-being of students and adults in K-12 schools. The EPP demonstrates that candidates meet the standard for high academic achievement through program assessments and Praxis II Content Area exams. In addition, candidates design and implement an action research project to demonstrate their impact on student learning.

Overall, Praxis test data from Fall 2021 and Spring 2022 indicates that candidates in the Advanced programs are meeting EPP expectations. Both advanced programs had a pass rate of 80% or higher on all tests. All Educational Leadership candidates passed the Ed. Leadership: Administration and Supervision exam (n=20) and 85.7% (6 out of 7) Reading Specialist candidates passed the Praxis exam required. Analysis of action research data from candidates in the advanced Reading Specialist program shows that candidates performed well above the proficient level of 3.00 out of 4.00 on all rubric criteria with an average of 3.24 out of 4.00 (n=11) for all candidates in Fall 2021 and Spring 2022. Candidates scored an average of 3.33 or higher on all rubric criteria except for the criteria on implications for personal practice (Fall 2020: 2.88 out of 4.00; Spring 2021: 2.33 out of 4.00). 

Praxis Test Pass Rates - Advanced Programs 2020-21

Action Research Data - Advanced Program F20-S21

Measure 4:
Ability of Completers to be Hired (in positions for which they have prepared)

 
 

Initial Programs

The EPP's PDS Office systematically tracks the number of completers hired after they complete their respective program. The PDS Office regularly checks the PDS County Board of Education meeting agendas and personnel directories to identify the location of hired completers and communicates with completers who provided the EPP with contact information. Current data reflect completers from Spring 2018 to Spring 2021. Data reflects that 71% to 100% of candidates from Initial programs who completed during these semesters have been employed in Education. It is worth noting that the lowest percentage is reflected for Spring 2021 completers, but that is also after school systems moved to virtual instruction in Spring 2020 followed by a hybrid model of virtual and face-to-face instruction for the Fall 2020 – Spring 2021 school year due to COVID-19. The two semesters with the highest percentage employment rate (100% in Fall 2018 and 90% in Spring 2019) are the semesters that the EPP knows the status of all completers.

Initial Programs Employed Completers F18-S21

Advanced Programs

All candidates accepted into the M.Ed. in Reading Specialist advanced program are certified teachers. Upon completion of the program in Fall 2020 and Spring 2021, every graduate was employed as a teacher in a West Virginia public-school and pursued employment as a Reading Specialist in West Virginia public-schools. In addition, five candidates graduated from the EPP’s new Educational Leadership Advanced program in 2020- 2021. Two out of the five candidates in the Educational Leadership program were employed as principals. Discussions with candidates indicate that many teachers add the Educational Leadership certification with plans to apply for administrative positions in the future.

Reading Specialist Hired Completers F18-S21

 

CAEP Annual Report

2021 Annual Report

2021 Annual Report