CAEP 4 Annual Reporting Measures
CAEP Accountability Measures
Measure 1:
Completer Effectiveness (R4.1)
-
Initial Programs
-
The EPP obtained I-Ready English/Language Arts and Math benchmark data from eight schools in one
county tied to 12 recent program completers currently teaching in grades kindergarten through eighth grade. The presented data represents assessment results from two diagnostic assessments. I-Ready Benchmark data reflect an overall average of 38% of students scored "On or Above Grade Level" with a range of 21% to 65%. Data indicates that an average of 37% of students scored "One Grade Level Below" with a range of 18% to 67%. "Two or More Grade Levels Below" represented 25% of students with a range of 0% to 76%. Overall, students of program completers performed higher in 2021-2022 than in 2020-2021. An average of 38% of students of program completers scored "On or Above Grade Level" in 2021-2022 while only 28% of students scored "On or Above Grade Level" in 2020-2021. However, the improvements could be attributed to changes in instruction. In 2020- 2021 the counties operated under a 2-day in-person and 3-day distance model and gave students the option to do fully remote learning. Students of program completers scored slightly lower in English/Language Arts than Math. I-Ready
English/Language Arts data reflect an average of 36% of students scored "On or Above Grade Level" with a range of 21% to 82%. Data indicates that an average of 37% of students scored "One Grade Level Below" with a range of 18% to 67%. "Two or More Grade Levels Below" represented 27% of students with a range of 0% to 52%, with an average of one student who did not complete the assessment and a range of 0-5 students who did not complete the assessment for each program completer. I-Ready Math data reflect that an average of 45% of students in classrooms of EPP completers
scored "On or Above Grade Level" on these benchmarks with a range of 29% to 64%. Data reflect that an average of 42% of students in classrooms of EPP completers scored "One Grade Level Below" with a range of 29% to 64%. "Two or More Grade Levels Below" represented an average of 13% of students with the range of 0% to 25% with an average of less than one student who did not complete the assessment and a range of 0-1 students who did not complete the assessment for each program completer. To assess completer effectiveness in applying professional knowledge, skills, and dispositions, the
EPP reviewed case study data on five program completers. Each of these completers were enrolled in a graduate class through the EPP that required use of the action research process to assess impact of instruction on K-12 students' performance. Completers from Initial undergraduate programs in English (5-Adult), Elementary (K-6), and Elementary with a concentration in Special Education (K-6) were included. In addition, case study data from one program completer from the initial M.Ed. in Multi-categorical Special Education program was evaluated. Completers were asked to submit an Action Research proposal that included a description of the learning need based on relevant data, the question that guided the project, the instructional methods, and data collection process. In the Action Research Report, completers shared the methods implemented, analysis of data, and implications for future instruction. Results indicate that all program completers saw improvements in students' performance. Greater gains were noted for program completers employed as general educators (mean = 35%) compared to those employed as special educators (mean = 13%). While this case study data is limited to five completers, it provides some evidence that candidates effectively apply professional knowledge and skills. The state of West Virginia purchased surveys through Skyfactor, a survey company that creates Exit
surveys, Alumni surveys, and Employer surveys for teacher education programs. Twenty-two program completers responded to questions on how well the Fairmont State Teacher Education program prepared them with the knowledge, skills, and dispositions they would need to be effective in the classroom. Graduates were asked to rate each question on a scale of 1-7 with one described as "not at all" and seven as "extremely satisfied." Overall, on 37 questions related to lesson planning, instruction, content knowledge, assessment, classroom management, diversity, and professional development, program completers rated the Fairmont State Teacher Education program with 5.80 out of 7.00. Program completers rated Fairmont State higher than other institutions for overall program effectiveness. The Fairmont State program was rated 5.90 out of 7.00 compared to 5.56 out of 7.00 for all institutions and 5.44 out of 7.00 compared to six similar regional institutions. Relative strengths noted by program completers included the program's positive impact on the development of professional skills (5.94 out of 7.00) and effective instructional strategies (5.96 out of 7.00). Positive student comments included, "I truly believe that I picked a great school to continue my education and study to become a teacher. I had absolutely wonderful professors that truly cared about me and taught me what it means to be a teacher and how to form those relationships with students." Program completers also noted relative weaknesses including classroom management (5.5 out of 7.00) and lesson planning (5.47 out of 7.00). When asked what the Fairmont State Teacher Education program could do to improve the program, five out of ten completers who responded stated that the lesson plan requirements were too long and detailed. One summarized concerns stating that "the long lesson plans took away from the actual experience of student teaching." Program completers provided the most feedback about classroom management, with seven out of ten stating that more tips and strategies would be helpful. Several suggested that providing different scenarios to respond to would be beneficial. Comments also indicated that program completers would benefit from more strategies to support social emotional learning. i-Ready Assessment Data 2021-22
Measure 2:
Satisfaction of Employers and Stakeholder Involvement (R4.2|R5.3|RA4.1)
-
Initial Programs
-
In spring 2022, the Skyfactor Employer Survey (Standard R4.2) was used to collect information on
Employer evaluations of new teacher performance. When completing the survey, employers of EPP graduates were asked to rate their satisfaction with the graduates' preparation for their assigned responsibilities in working with diverse P-12 students and their families. Of the ten principals that responded, 20% rated Fairmont State graduates as among the best prepared, 60% stated that they were better prepared than others, 10% stated that they were as prepared as others, and 10% stated that they were unable to make a comparison. The Skyfactor Employer Survey competencies are explicitly aligned to CAEP and InTASC and reflect
the shared values and beliefs of the EPP. Employers of EPP graduates were asked to respond to questions on graduates' abilities to apply professional knowledge, skills, and dispositions. Employers rated graduates on a scale of 1-7 with one described as "not at all" and seven as "extremely satisfied." Eighty percent or more of employers rated graduates as a six or seven on their abilities related to the following knowledge, skills and dispositions: understanding of assessment strategies (82%); building collaborative professional relationships (91%); creating a productive classroom environment (80%); demonstrating effective classroom instruction (80%), developing effective lesson plans (90%); displaying appropriate professional skills (91%); exhibiting mastery of relevant content (90%); integrating technology into the teaching experience (100%); and reflecting the value of diversity in teaching (80%). No Employer rated candidates below a score of 4 (moderately satisfied) in any category. Employer responses indicate that our graduates apply professional knowledge, skills, and dispositions that their preparation experiences were designed to achieve. With the implementation of the yearlong residency initiative that was mandated by West Virginia
Board of Education Policy 5100 for all EPPs by Fall 2021, stakeholder involvement was a priority. Teachers and administrators from the EPP's professional development school (PDS) network were involved throughout the curriculum development process and plans for implementation. In Summer 2021, that workgroup-composed of EPP faculty, staff, teachers, and administrators from four school districts in the PDS network-collaborated to co-design the field experience checklists for Residency 1 and Residency 2/student teaching. In Summer 2022, the PDS Advisory Board met again for two days to review data on the teacher candidates who piloted the residency. The Advisory Board also co- designed the requirements and checklist for the new field experience that would be rolled out with the High Incidence Disabilities class in Fall 2022. Skyfactor Employer Assessment of Program Completers 2021-22_CAEP Accreditation Standards
Skyfactor Employer Demographics and Overall Rating of Program Completers
-
Advanced Programs
-
Stakeholder involvement has been essential to the development of the Educational Leadership
curriculum. At the start of the rollout of this Advanced level program (with an option for certification), individuals from across the professional development school network co-designed course proposals based on their level of expertise. After the proposals for courses were finalized, the stakeholders met to vote on the final curriculum proposals for the superintendent certificate and principal certificate tracks. Due to transitions in faculty and department chair in spring 2021, the stakeholder group did not meet in Fall 2021 and Spring 2022. However, during the spring and summer of 2022, a formal Educational Leadership Advisory Board was formed that met in September 2022. Additional meetings were planned in Spring 2023 to provide input on the Specialized Professional Association (SPA) report and plans to add an internship to the program. The EPP will provide notes on those meetings in the 2023 CAEP Annual Report. Due to a transition in the Program Coordinator for the M.Ed. Reading Specialist, the Advisory Board
did not meet during the 2021-2022 academic year. The program coordinator resigned mid-year due to family obligations, however, she continued to teach the program's online courses. Surveys were sent to Employers to determine satisfaction with program graduates. Employers were asked to rate their satisfaction with graduate performance on the ILA standards as Strongly Agree, Agree, Disagree, or Strongly Disagree. Only one employer completed the survey, and this employer marked strongly agree or agree to questions on all standards. A new program coordinator was hired in Fall 2022 and surveys will be conducted this year with follow-up to try to increase employer responses to the survey.
Measure 3:
Candidate Competency at Completion (R3.3)
-
Initial Programs
-
The EPP tracks the number of completers of Initial programs as defined by CAEP through our
Certification Officer and the Office of Institutional Effectiveness and Strategic Operations. Initial certification numbers represent all candidates completing the undergraduate education programs, MAT graduate programs, and the M.Ed. in Multi-categorical Special Education with Autism graduate program. At the completion of the program, the EPP documents that candidates achieve a high standard for
content knowledge and can teach effectively to promote student learning and development. The EPP demonstrates that candidates meet the standard for high academic achievement and ability through the Student Teaching Assessment (STA), Praxis II Content Area exams, and Principles of Learning and Teaching Praxis II exams. In addition, during student teaching, candidates design and implement an action research project to demonstrate their impact on student learning and present findings to faculty and peers. Candidates enrolled in Initial programs complete 16 weeks of student teaching at West Virginia
required developmental levels and in all teaching specializations in which they expect to be licensed. During the student teaching experience, candidates in the undergraduate and MAT programs demonstrate the ability to teach college and career-ready standards in lesson and unit plans and are evaluated in each placement using the Student Teaching Assessment (STA). The STA is aligned with the West Virginia Professional Teaching Standards (WVPTS), the Council for the Accreditation of Educator Preparation (CAEP) standards, and the Interstate Teacher Assessment and Support Consortium (InTASC) standards. Candidates are assessed on four standards that include 20 specific competencies: Standard 1: Curriculum and Planning; Standard 2: The Learner and the Learning Environment; Standard 3: Teaching; and Standard 4: Professional Responsibilities for Self-Renewal, School, and Community. Data from Fall 2021 (n= 10) and Spring 2022 (n= 21) indicate that candidates performed well on all
standards, with average scores on standards ranging from 3.62 out of 4.00 on Standard 3 in Spring 2022 to 3.82 out of 4.00 on Standard 1 in Fall 2021. Review of the competencies for each standard indicates that average scores were above the proficient (3) level for all standard competencies. Strengths include Competency 1C: Setting Goals and Objectives for Learning (Fall 2021: 3.84 out of
4.00; Spring 2022: 3.87 out of 4.00); Competency 1D: Designing Instruction (Fall 2021: 3.84 out of 4.00; Spring 2022: 3.88 out of 4.00); and Competency 2F: Organizing the Learning Environment (Fall 2021: 3.86 out of 4.00; Spring 2022: 3.82 out of 4.00). Relative weaknesses include Competency 2E: Managing Student Behaviors (Fall 2021: 3.58 out of 4.00; Spring 2022: 3.60 out of 4.00) and Competency 3C: Questioning and Discussion Techniques (Fall 2021: 3.60 out of 4.00; Spring 2022: 3.55 out of 4.00). Average candidate scores were 3.50 out of 4.00 or higher on 20 out of 20 competencies in both Fall 2021 and Spring 2022, providing strong evidence that candidates meet EPP and West Virginia Department of Education expectations at program completion. Analysis of data from the action research project suggests that candidates provide effective
instruction through the application of content knowledge and pedagogical skills, and they positively impact student learning. Candidates in undergraduate elementary and secondary programs performed well on the final report and presentation with mean scores of 3.57 out of 4.00 in Fall 2021 on all final report criteria and a mean score of 3.70 out of 4.00 on presentation criteria. Scores were slightly lower in Spring 2022, with a final report mean score of 3.48 out of 4.00 on all report criteria and 2.69 out of 4.00 on presentation criteria. In Fall 2021, one candidate in the MAT program scored a 98% on the report and in Spring 2022, two candidates in the MAT scored an average of 97% on the final report. An analysis of tests taken by candidates in Initial level programs from Fall 2021-Spring 2022 reveals
12 out of 24 Praxis tests or subtests had a pass rate of 80% or higher, with eight tests at a 90% or higher pass rate. However, the pass rate did not meet the 80% criteria overall on the elementary subtests and six out of sixteen Praxis secondary or subject area content tests taken by candidates in Initial programs. In many cases for secondary content tests, the lower pass rate was due to repeated tests by a small number of test-takers and a small number of test-takers who did not pass. For example, four content tests with a pass rate below 80% had less than four test takers, and in three cases, a candidate did not pass the test one or more times. While EPP pass rates were below the 80% pass rate criteria on the elementary science and social studies MS subtests, they were at 100% on both the CKT Social Studies and CKT Science subtests. These tests were added as options for Elementary majors due to the low state average pass rates of 59% on the Elementary MS Science subtest (EPP= 54%) and the state average of 48% on the Elementary MS Social Studies subtest (EPP= 51%). However, the faculty recognized that the EPP needed to support candidates who struggled to pass these elementary subtests and added a class for content test preparation. In addition, an online module was made available to all candidates with test preparation materials. Student Teacher Assessment 21-22
Praxis Test Pass Rates - Initial Programs 2021-22
Action Research Final Report Directions
-
Advanced Programs
-
The Advanced programs include candidates completing the M.Ed. Reading Specialist program and
the M.Ed. in Educational Leadership. At the completion of the program, the EPP documents that candidates in the Reading Specialist program achieve a high standard for content knowledge and can teach literacy skills effectively to promote student learning and development. The EPP documents that candidates in the Education Leadership program obtain the professional knowledge and skills needed to promote the success and well-being of students and adults in K-12 schools. The EPP demonstrates that candidates meet the standard for high academic achievement through program assessments and Praxis II Content Area exams. Reading Specialist candidates complete a practicum during the final semester of the program. The
activities and assignments in the practicum are designed to help candidates meet the qualifications of a Reading Specialist/Literacy Coach as defined by the International Literacy Association (ILA) standards. During the semester candidates assess, plan, and implement instruction for students who struggle with literacy. The candidates are evaluated twice a semester by their supervisors. Review of data indicates that candidates performed well above the proficient level of 3.00 out of 4.00 with an average of 3.42 out of 4.00 in Fall 2021 (n=6) and 3.75 out of 4.00 in Spring 2022 (n=4). During the final practicum, candidates also design and implement an action research project to demonstrate their ability to plan instruction that positively impacts student learning. Analysis of action research data from candidates in the Advanced Reading Specialist program shows that candidates performed above the proficient level of 3.00 out of 4.00 on all rubric criteria with an average of 3.40 out of 4.00 (n=6) for all candidates in Fall 2021 and 3.82 out of 4.00 (n=4) in Spring 2022. Candidates scored an average of 3.50 or higher on all rubric criteria except for the criteria on implications for personal practice (Fall 2021: 2.50 out of 4.00; Spring 2022: 3.00 out of 4.00). Praxis test data from Fall 2021 and Spring 2022 indicates that candidates in the Education
Leadership program are meeting EPP expectations. All Education Leadership candidates passed the Ed. Leadership: Administration and Supervision exam (n=21). Challenges were noted for candidates on the Reading Specialist exam. The Praxis test required by the state of West Virginia changed this year and the pass rate was only 40%. This was a change from the five previous years when the pass rate was 80% or higher. As a result, the new Reading Specialist program coordinator hired in Fall 2022 will evaluate the changes between the previously required exam and the new exam to determine changes in content and make adjustments to the curriculum. Praxis Test Pass Rates - Advanced Programs 2021-22
Reading Specialists Action Research Data Analysis and Implications Scores F21
Reading Specialists Action Research Data Analysis and Implications Scores S22
Measure 4:
Ability of Completers to be Hired (in positions for which they have prepared)
-
Initial Programs
-
The EPP PDS Office systematically tracks the number of completers hired after they complete their
respective program. The PDS office regularly checks the PDS County Board of Education meeting agendas and personnel directories to identify the location of hired completers and communicates with completers who provided the EPP with contact information. Current data reflect completers from Fall 2019 to Spring 2022. Data reflects that 71% to 100% of candidates from Initial programs who completed during these semesters have been employed in Education. It is worth noting that the lowest percentage is reflected for Spring 2021 completers, but that is also after school systems moved to virtual instruction in Spring 2020 followed by a hybrid model of virtual and face-to-face instruction for the Fall 2020 - Spring 2021 school year due to Covid-19. The two semesters with the highest percentage employment rate (100% in Fall 2019 and 83% in Fall 2021) are the semesters that the EPP knows the status of all completers. -
Advanced Programs
-
All candidates accepted into the M.Ed. Reading Specialist Advanced program are certified teachers.
Upon completion of the program in Fall 2021 and Spring 2022, every graduate was employed as a teacher or substitute in a WV public school. In addition, 23 candidates graduated from the EPP's new Education Leadership Advanced program in 2021- 2022. Our records indicate that only three graduates in the Education Leadership program are employed as principals or administrators, but this information has been difficult to track. As in previous years, discussions with candidates indicate that many teachers add the Education Leadership certifications with plans to apply for administrative positions in the future. With significant turnover in the College and with state delivered surveys changing providers each year, tracking graduates has become more challenging. To remedy this, the CAEP team decided the Administrative Assistant in the Clinical Office will begin tracking graduates (regardless of state surveys administered) and data will be provided in the 2023 CAEP Annual Report.