Standard 2: Aspect D

Standard 2d: Program completers engage in professional practice in educational settings and show that they have the skills and abilities to do so in a variety of additional settings and community/cultural contexts. For example, candidates must have broad and general knowledge of the impact of culture and language on learning, yet they cannot, within the context of any given program, experience working with the entire diversity of student identities, or in all types of school environments. 

Candidate preparation includes first-hand professional experience accompanied by reflection that prepares candidates to engage effectively in different contexts they may encounter throughout their careers.

---

Data Sources & Analysis

Data Source 1

CSU Educator Quality Center Year One Completer Survey

Description of Data Source: 
The California State University’s Education Quality Center (EdQ) oversees the administration of a survey of all individuals who completed a CSU teacher-preparation program after their first year on the job. The survey is administered annually April through July. In April, the EdQ Center emails an initial survey invitation to all completers of Multiple Subject Credential Programs serving as first-year teachers in public schools, charter schools, or private schools in all locations. Follow-up reminders are sent every two weeks throughout the duration of the survey window. 

In addition to asking questions about the completer’s demographics and educational  background, the survey also contains items to capture data about the school where the completer is employed. Additionally, the survey includes items asking about candidates’  perceptions of various aspects of the preparation program and the field placement experience. Campuses have access to annual results from the survey by utilizing the EdQ Dashboard. Results can be disaggregated by various measures including campus, year of completion, respondent race/ethnicity, and type of credential. Note: the California Commission on Teacher Credentialing (CCTC) also distributes a Credential  Program Completer Survey which gives an overall view of CA Educator Preparation Programs. 

Perspective Captured from Data Source: Completer (one year after program completion)

Rationale for Using Data Source:
For candidates who completed a CSU educator preparation program each academic year from 2014-2015 to 2017-2018, the CSU Educator Quality Center administered a year-out survey to learn how well the educators, after their first year in the field, believed the program had prepared them. Due to the challenges of educators in Spring 2020, the CSU Educator Quality chose not to administer the survey at that time, meaning data are not available for candidates who completed in 2018-2019. Survey administration began again in Spring 2021, and we look forward to analyzing those data when they become available.

Still, the routine, annual collection of these data offer the program to look at Multiple Subject Program Completer perceptions of preparedness longitudinally. Moreover, the survey items are also aligned to the Teaching Performance Expectations (TPE) for which our program curriculum and clinical experiences are also aligned. This makes the survey a good tool for the face validity of the survey as it measures what our program intends to understand about the quality of our teacher credential program. Overall this survey tool aims to capture their perceptions of preparation after having been a teacher of record for one year. 

We selected six (6) items from the survey that focused on the Multiple Subject Program Completers’ (MSPC) one-year out perspectives of how the program prepared them to support Tk-6 students’ growth in international and global perspectives.     

Specific Elements of Data Source:
The items we selected to analyze include candidate responses for how well prepared they were to begin each aspect of a teacher’s job listed below. The items available to provide evidence for this aspect are not as obvious as the items for other aspects therefore a brief rationale is provided:

• To help students build a strong knowledge base through content rich texts. This item demonstrates that the program has good curricular coverage through the use of a wide array of literature. 
• To provide opportunities for students to learn about critical environmental issues and questions facing the world today. This item demonstrates evidence of how the program prepares candidates to help their TK-8 students develop global awareness. 
• To help students use digital media and environments to communicate and work collaboratively. This item demonstrates that the program prepares teacher candidates to use digital media that provides TK-8 students with content from across the globe and to learn how to share perspectives and work collaboratively across boundaries. 
• To help students gather, evaluate, and use information from a variety of print, media, and online sources. This item is intended to measure the program’s ability to prepare candidates to encourage their TK-8 students use of a variety of online sources that often grant easy access to global perspectives on multiple subjects. 
• To teach students to apply the mathematics they know to model real-life situations and solve realistic problems. This item demonstrates that the program prepares teacher candidates to develop lessons that are connected to real-world problems. These lessons can include connections to the global economy such as problem-posing projects which might range from connecting students to the gentrification happening in their neighborhoods to studying patterns of displacement of people in other global cities. 
• To teach the History-Social Studies Content Standards for California Public Schools. This item demonstrates that the program has good curricular coverage through the program’s ability to prepare elementary teachers to teach social studies.  

Definition of Success for Each Element:
It is our program's goal to have 75% or more of candidates indicate positive preparedness perceptions within the 3 - 5 range or from “Adequately Prepared” to “Well Prepared” to “Very Well Prepared.” 

Displays of Analyzed Data:

Figure 1: 2015-2016 Multiple Subject Program Completers One Year Out (N 44)

Figure 2: 2016-2017 Multiple Subject Program Completers One Year Out (N 61)

Figure 3: 2017-2018 Multiple Subject Program Completers One Year Out (N 59)

Link to Full Dataset: The link to the full dataset is unavailable. However, if reviewers would like to view the CSU Educator Quality Center Data Dashboards, we are happy to set up a time to provide them access by sharing screens in a Zoom session.

Interpretation of Data:
The data collected from 2015-2017 show a consistent trend that most of our candidates felt well prepared to support students’ growth in international and global perspectives. Nearly all items met the program’s goal of 75% or more espousing positive perceptions. However, a slide back in positive preparation perception occurred in 2017-2018 as nearly all items just met or did not meet the program’s goal of 75% reporting positive perceptions. This could be attributed to the program redesign that was newly implemented. 2017-2018 program completer journey map data indicated high levels of stress felt by students in the program due to ineffective program communications where during data discussions the program instructors expressed frustration with building the plane while flying it which translated into students “feeling like a guinea pig” due to the program changes being implemented. Overall, the CSU Survey data also revealed that the program completers one-year-out from finishing the program consistently rated being least prepared to teach History-Social Studies Content Standards for California Public Schools which parallels the findings from the CSU completer survey at the end of the program. This also indicates that their first year of induction support did not seem to advance their skills related to teaching History-Social Student Content Standards. It is also plausible that because this content is not included in standardized testing cycles that the least amount of instructional time is included in the hiring district’s pacing calendar. 

Data Source 2

Formative & Summative New Teacher Evaluation

Description of Data Source: 
Fresno Unified is the third-largest school district in California, educating just over 73,000 K-12 students in the 2019-2020 academic year (CDE, n.d.). Of those students, 68% identified as Latinx, 10% identified as Asian, 9% identified as White, 8% identified as Black, and 2% identified with two or more races, and 60% of those children are dual-language learners (CDE Dataquest, n.d.).  Additionally, 85.9% of the students in the local school district receive free or reduced meals (CDE, n.d.). Fresno Unified also consistently places the largest number of Multiple Subject Credential Candidates with experienced mentor teachers in their district for their clinical experience (N 73; 26%). Moreover, Fresno Unified hires the majority of our MS program completers; 60-70 new hires each year. For these reasons, we launched a plan in March 2020 where Fresno Unified agreed to strengthen their data collection systems so that we could receive their teacher induction program data. 

Perspective Captured from Data Source: District Instructional Coach

Rationale for Using Data Source:
The Formative & Summative New Teacher Evaluation is recognized as a desirable method to achieve the improvement of instruction, to identify skills and abilities that contribute to the success of the educational program, and to redirect skills and abilities that do not result in optimum student growth. The goals of the evaluation are as follows: 

• To recognize the performance of outstanding employees 
• To enhance and improve performance through communications that is direct, clear, honest, immediate, frequent, and evidence based 
• To align professional growth to employees’ strengths and areas of improvement
• To provide avenues for informal and formal communications that builds relationships.  

District Instructional Coaches are trained and calibrated to use the district’s formative and summative evaluation rubric that is aligned with the California Standards for the Teaching Profession (CSTP), which is also aligned with the Teaching Performance Expectations that serves as a guide to our program’s curriculum and clinical experiences. New teachers are formally observed twice per year. The first formal observation takes place by the end of November and it is formative in nature. A full lesson is observed and followed up by a debrief within five days from the observation. The second formal observation takes place by the end of May and it is summative in nature.  

Specific Elements of Data Source:
The items we selected to analyze align to TPE/CSTP 1: Engaging and Supporting All Students in Learning because this Standard also assesses how teachers encourage students to ask critical questions and consider diverse perspectives about subject matter such as raising awareness about international contexts. The Standard asks teachers to consider how and why they:

• Promote critical thinking through inquiry, problem solving, and reflection. This item is intended to measure how program completers promote critical thinking among TK-8 students which is a skill essential to developing TK-8 students' understanding of global perspectives.

Definition of Success for Each Element:

The Formative and Summative New Teacher Evaluation consists of a scale of 1-4: 

• 1 - “Does Not Meet Standards” 
• 2 - “Growth Expected”  
• 3 - “Meeting Standards”
• 4 - “Demonstrates Expertise” 

A 2 of growth expected still meets the goals of the CSTP measured by the rubric. It is our program's goal to have 85% or more of the program’s completers in their first year of hire within the 3 - 4 range or “Meeting Standards” to “Demonstrates Expertise” by the summative evaluation. Our program would also want to observe an increase in completers’ scores from the formative to summative evaluation. 

Displays of Analyzed Data:
Table 1: 2020-2021 Formative and Summative Outcomes on CSTP 1 

Link to Full Dataset: FUSD Fresno State Induction Data

Interpretation of Data:
The data collected over the past year indicates that 91% of  program’s completers in their first year of hire are Meeting Standards or higher on CSTP 1 for their summative evaluation thereby meeting the program’s goal for new teachers' ability to have good curricular coverage that provides opportunities for developing international perspectives on content. The program met its secondary goal as a positive trend was also observed in scores from the formative to summative evaluations.  

Data Source 3

CCTC Employer Survey 

Description of Data Source: 
The California Commission on Teacher Credentialing (CCTC) administers the Employer Survey annually from October 1 to December 31. Employers who have hired at least 2 completers from the same institution over the past 3-5 years are asked to complete the survey to provide information about their perception of how well new teachers were prepared by their preparation programs. The survey items align with the California Teaching Performance Expectations (TPE) and the California Standards for the Teaching Profession (CSTP).  Since both set of standards inform the knowledge and skills embedded in the Multiple Subject Credential Program, this survey gathers valuable information for our program to reflect on regarding the strengths and weaknesses of the preparation program from Employers’ perspectives. Moreover, the intention is that the individual who has seen the new teachers teach is the person who completes this survey, which also provides our program with insight into how the completers of our program are doing 3-5 years out.

Perspective Captured from Data Source: Employer

Rationale for Using Data Source:
The CSU Educator Quality Center administered a survey to employers of 2014-2015 and 2015-2016 graduates of CSU educator preparation programs. Results were able to be disaggregated by both preparatory institution and program. The survey was discontinued after evaluating the 2015-2016 completers, meaning that all available data are at least five years old. The California Commission on Teacher Credentialing (CCTC) began its own employer survey that it first administered in Fall 2019. However, while it is possible to disaggregate results by institution, it is not possible to do so by preparation program. Although the data from the CCTC Employer Survey are not disaggregated, we chose to use the results as one of our three data sources because they do provide an employer perspective on our candidates. In the future, we plan to advocate to the CCTC for the addition of items to the survey that would allow data to be disaggregated in this way.

Specific Elements of Data Source:
The item we selected to analyze align to TPE/CSTP 1: Engaging and Supporting all Students in Learning include employer responses to how well-prepared program completers are to do the following as a teacher compared to other beginning teachers with whom the employer has worked:

• Engage students in inquiry, problem-solving, and reflection to promote their critical thinking

This item was selected because engaging students in inquiry, problem-solving, and reflection to promote critical thinking are skills that can help TK-8 students understand and make sense of the world around them and to fully participate in the global economy in the future. 

Definition of Success for Each Element:
The CCTC Employer Survey responses range from 1 - “Not At All” to 5 - “Very Well prepared.”  It is our program's goal to have 80% or more of employers indicate positive preparedness perceptions within the 3 - 5 range or from “Adequately” to “Well” to “Very Well” prepared. 

Displays of Analyzed Data:

Figure 4: 2018-2019 CCTC Employer Survey (N 32)

Figure 5: 2019-2020 CCTC Employer Survey (N 33)

Link to Full Dataset:  MS Employer Master Teacher Survey Results

Interpretation of Data:
The data collected over the last two years show a consistent positive trend that the employers of our program completers feel that those that they hired from our program are well prepared to support student global perspectives as measured by engaging students in inquiry, problem-solving, and reflection to promote their critical thinking. No items fell below 80% in positive perception thereby meeting the program’s goal over the past two years. Moreover, an increase in positive perception is noticed from 2018-2019 to 2019-2020. 

Next Steps:
Need to address the program completers’ ability: 

• To provide opportunities for students to learn about critical environmental issues and questions facing the world today.
• To teach students to apply the mathematics they know to model real-life situations and solve realistic problems.
• To teach the History-Social Studies Content Standards for California Public Schools.

In order to address what we found, we will . . .

• Invite program faculty and district partners into conversations about how we prepare program completers to be day one ready to support students in their development of global perspectives. Work together to reflect on… What does this look like in practice? How can we build in more clinical experiences that build new teacher confidence in these areas? What change ideas can we test out and study? Review data together and discuss what additional data is needed to help us develop interpretations that increase our confidence and decision-making. 

To evaluate our efforts in this area, we will . . .

• Strengthen the datasets that we currently have to assess this area:
  • Assess if there may be additional data sources that better measure the intended content of Standard 2 Aspect D, and if not, build out those measures over time. 
  • Invite two more district partners to support data sharing of induction data including discussing their induction practices and data collection procedures.
  • Ask Fresno Unified about receiving induction data by item-level analysis and for program completers by year of service in the district (Year 1, Year 2, Year 3 and Year 5).   
  • Work with our district partners to develop a strategy to increase CCTC employer survey response rates.

Aspect E →