Standard 2 Aspect B

Standard 2b: Program completers engage in professional practice in educational settings and show that they have the skills and abilities to do so in a variety of additional settings and community/cultural contexts. For example, candidates must have broad and general knowledge of the impact of culture and language on learning, yet they cannot, within the context of any given program, experience working with the entire diversity of student identities, or in all types of school environments. Candidate preparation includes first-hand professional experience accompanied by reflection that prepares candidates to engage effectively in different contexts they may encounter throughout their careers. Engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts

---

Data Sources & Analysis:

Data Source 1

CalAPA Leadership Cycle 2 (Rubric 2.4)
After completion of the first semester of the PASC program, candidates exhibit skills and abilities as educational leaders who are culturally responsive to the diverse needs of students in grades P-12 through their fieldwork and coursework experiences that culminate with the submission of CalAPA Leadership Cycle 2. Rubric 2.4 evaluates the candidate’s ability to explain the collaborative process used to select the relevant evidence-based strategy and work with the group to learn about and monitor implementation of that strategy to address the selected problem of practice, as well as describe the potential impact on student learning and/or well-being. Candidates also work towards describing in detail how the collaboratively determined evidence-based strategy will improve student learning and/or well-being across specific student groups. 

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CCTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 2.4 is aligned to summative assessment of candidate performance (completers of first semester program content and skills) of the following AAQEP 2b aligned CAPEs elements: 3C.3) Recognize discriminatory practices, signs of trauma, manifestations of mental illness, and promote culturally responsive, positive and restorative strategies to address diverse student and school needs, and; 5B.2) Recognize any possible institutional barriers to student and staff learning and use strategies that overcome barriers that derive from economic, social-emotional, racial, linguistic, cultural, physical, gender, gender identity, sexual orientation, or other sources of educational disadvantage or discrimination. Thus, Rubric 2.4 has been chosen as an appropriate measure for the demonstration of program completer skills and abilities to engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts.

Although Rubric 2.4 evaluates candidates’ knowledge and skills while they are still in the program, our hope is that they will be able to draw on the knowledge and skills they develop once they have completed the program and are employed as administrators. 

Definition of Success for Each Element:
Semester one program completer success would be measured at Level 2 or Level 3 as proficiency with the CCTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 1

CalAPA Leadership Cycle 2, Rubric 2.4 for program level and state-wide mean scores by submission year

Link to Full Dataset:  CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Three years of data in relation to Rubric 2.4 highlights that first semester program completers are overall meeting the current program expectation of 2 and above. Semester one completers are also consistently performing above the State-wide average.  Overall, responses show candidates ability to  explain the collaborative process used to select the relevant evidence-based strategy and work with the group to learn about and monitor implementation of that strategy to address the selected problem of practice, as well as describe the potential impact on student learning and/or well-being. 

Moving forward, faculty’s focus for supporting first semester program completers at higher levels includes supporting more detail in how the collaboratively determined evidence-based strategy will improve student learning and/or well-being across specific student groups. Faculty should also collaborate regarding instructional strategies and resources to better assist first semester completers ability to draw on and cite evidence-based practices and/or related research to thoroughly show understanding of the potential implementation implications and/or challenges. 

Data Source 2

CalAPA Leadership Cycle 1 (Rubric 1.2)
After the second semester of the PASC program, candidates exhibit knowledge, skills, and abilities through their fieldwork and coursework experiences culminating with the submission of CalAPA Leadership Cycle 1.  Rubric 1.2 evaluates the candidates’ ability to collect and analyze relevant qualitative data and explain their relation to quantitative data findings and the student group equity issue. Candidates also work to provide a comprehensive analysis collecting additional qualitative data, as appropriate, to deepen their understanding of the chosen California state indicator and student equity issue. 

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CCTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 1.2 is aligned to summative assessment of candidate performance (completers of second semester program content and skills) of the following AAQEP 2b aligned CAPEs elements: 1A.3) Analyzing and applying political, social, economic, and cultural contexts to inform the school’s vision and mission, and; 1A.5) Explaining how school plans, programs, and activities support the school’s vision to advance the academic, linguistic, cultural, aesthetic, social-emotional, behavioral, and physical development of all students. Thus, Rubric 1.2 has been chosen as an appropriate measure for the demonstration of program completer skills and abilities to engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts.

Although Rubric 1.2 evaluates candidates’ knowledge and skills while they are still in the program, our hope is that they will be able to draw on the knowledge and skills they develop once they have completed the program and are employed as administrators. 

Definition of Success for Each Element:  Semester two program completer success would be measured at Level 2 or Level 3 as proficiency with the CCTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 2

CalAPA Leadership Cycle 1, Rubric 1.2 for program level and state-wide mean scores by submission year

Link to Full Dataset:  CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Three years of data in relation to Rubric 1.2 highlights that program candidates are overall meeting the current program expectation of 2 and above, but are not yet performing consistently above the State-wide average, as based on initial data reporting. Based on the previous trend and once the entire data set is available, the program believes semester two completer mean scores will be above the State-wide scores.

Evidence shows that candidates' overall meeting proficiency in demonstrating the ability to collect and analyze somewhat of a range of relevant qualitative data and explain their relation to quantitative data findings and the student group equity issue. 

Although candidates are performing at current expectations, further focus is needed to support completers to provide a more comprehensive analysis and collect additional qualitative data, as appropriate, to deepen their understanding of the chosen California state indicator and student equity issue. Candidates could also benefit from increased faculty support on how to demonstrate a responsive qualitative data collection approach, taking into account the complexity of the school context with cultural sensitivity and appreciation for diverse viewpoints. 

Data Source 3

CalAPA Leadership Cycle 3 (Rubric 3.2)
After the last semester of the PASC program candidates exhibit knowledge, skills, and abilities of professional educators through their fieldwork and coursework experiences culminating with the submission of CalAPA Leadership Cycle 3.  The focus of Rubric 3.2 is to evaluate the discussion between the candidate and volunteer teacher regarding learning goals, classroom context, and student assets--including those of students from culturally and linguistically diverse backgrounds--and learning needs prior to the teaching and learning classroom observation.

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CCTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 3.2 is aligned to assessment of candidate performance of the following AAQEP 2b aligned CAPEs elements: 2B.3) Supporting and promoting effective instruction and a range of instructional methods and supporting practices that address the diverse educational needs of all students; 2D.3) Providing timely, constructive suggestions about instructional strategies and assessments, available resources, and technologies to refine and enhance instruction and assessment that supports student learning, safety, and well-being; and; 5C.1) Apply policies and practices that both support student learning and protect the rights and confidentiality of students, families, and staff. Thus, Rubric 3.2 has been chosen as an appropriate measure for the demonstration of program completer skills and abilities to engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts.

Although Rubric 3.2 evaluates candidates’ knowledge and skills while they are still in the program, our hope is that they will be able to draw on the knowledge and skills they develop once they have completed the program and are employed as administrators. 

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3 as proficiency with the CCTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 3

CalAPA Leadership Cycle 3, Rubric 3.2 for program level and state-wide mean scores by submission year

Link to Full Dataset:  CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Three years of data in relation to Rubric 3.2 highlights that program completers are overall meeting the current program expectation of 2 and above and are overall performing just at or above the State-wide average, with the exception of year two.

Current evaluation of performance based on mean scores highlights the ability of program completers to engage the teacher in a mostly detailed pre-observation discussion about the classroom context, student assets and learning needs, and content-specific learning goals of the lesson, including student work. 

Current student mean scores also demonstrate an area of need in the program completers ability to jointly select, with the teacher, the CSTP element(s), and together determine evidence to be collected, including student work to plan for the observation as well as use facilitative questioning strategies to engage in a two-way discussion to also draw out specific areas of need based on the volunteer teacher’s context, including student assets and learning needs, in order to focus the observation. Candidates also need further faculty support on when and how to re-direct the conversation as appropriate, to deepen discussion regarding equitable opportunities for all students in the class to learn.

Data Source 4

CCTC Completer Survey (Question 7)
After completion of the PASC program, the CCTC initiates a 35-item demographic and Likert-type self-report survey to all program completers to assess new administrator knowledge, skills, and abilities in the framework of the CAPEs based on their program fieldwork and coursework experiences.  The CCTC Completer Survey is administered to all program completers within 3 to 6 months of finishing the program to gather information about their perceptions of the PASC program, including those which relate to AAQEP standard 2b, how prepared they are to engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts. 

Perspective Captured from Data Source: Recent (3 months) program completers.

Rationale for using Data Source:
Specifically, Question 7 of the CCTC survey asks program completers the following on a scale from 1 (not at all) to 5 (very well): How well did your administrator preparation program prepare you to do the following as an administrator: demonstrate understanding of the school and community context, including the instructional implications of cultural/linguistic, socioeconomic, and political factors?  

Specific Elements of Data Source:
Q7 is a self-report item scaled from 1 (not at all) to 5 (very well) self-report founded in CalAPAs 2B: Instructional Leadership. Students rate how well prepared they feel they are to understand and engage in the school and community context, including the instructional implications of cultural/linguistic, socioeconomic, and political factors, which includes culturally responsive and linguistically appropriate approaches by the completion of the PASC program.

Definition of Success for Each Element:
Candidate success would be measured by 1) mean rating of 3 (adequately) or higher, and; 2) maintaining mean ratings above the State-wide average.

Displays of Analyzed Data:

Table 4

Table of CCTC Completer Survey program and state-wide self-report scale means for Q7 by academic year 

Link to Full Dataset: California Commission on Teacher (CCTC) Credentialing Completer Survey, Q7

Interpretation of Data:
Based on three years of data, evidence highlights that PASC completers consistently feel the program prepared them overall well to very well for this construct. While this exceeds our success criteria of a mean score 3 and above, the program has yet to realize a consistent mean score above the State-wide level.

Data Source 5

P12 PASC Program AAQEP Candidate Self-Assessment 

Perspective Captured from Data Source:
Candidate retrospective self-report of AAQEP Standard 2 at each semester end interval. 

Specific Elements of Data Source:
Item 2B: As an aspiring school leader, I engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts. 

Rationale for using Data Source:
The P12 PASC Program AAQEP Candidate Self-Assessment was designed to capture student reflection and growth after each semester in the program in alignment with AAQEP aspects. The results capture one cohort (program graduates in May 2021) over the 3-semester program using a retrospective design to pilot the self-assessment and obtain initial data. 

Definition of Success for Each Element:
Candidate success would be measured by 1) average of 3 or higher for each semester and 2) evidence of mean growth from semester 1 to semester 3. 

Displays of Analyzed Data:

Table 5

Table of AAQEP Standard 2b Candidate Self-Assessment Response Means for Cohort 0 (Pilot)

Link to Full Dataset: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 2 by Semester 

Interpretation of Data:
Overall, student reported mean scale scores for Standard 2b currently shows some growth from initial to end of program data points. Evidence also demonstrates that PASC program completers overall feel they can engage in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community contexts at high levels (between a 4 and 5) which exceeds our success criteria of level 3. 

Next Steps for 2b

Based on the analysis of the four data point interpretation sections, the following next steps are recommended: 

After the completion of the PASC program, candidates exhibit skills and abilities as educational leaders who are culturally responsive to the diverse needs of students in grades P-12 through their fieldwork and coursework experiences that culminate with the submission of CalAPA Leadership Cycles.  This includes an emphasis on the intersectionality of race, ethnicity, class, gender identity and expression, and sexual identity, and addressing the needs of diverse student groups in schools who are acquiring academic English language skills and are in the various stages of developing literacy. 

Overall, students perform at the rubric expectation.  The 2020 data is for half of the year therefore full analysis is difficult.  Candidate evidence shows that candidates can collect data and identify patterns or gaps that highlight student inequities.  Current student mean scores demonstrate an area of need in a candidate’s ability to explain the relationship between qualitative data collected to the quantitative data chosen in a specific student equity issue.

Furthermore, although this data set and the assessment measure candidates’ understanding of data collection that highlights inequity, that does not completely measure a candidate’s ability to engage in culturally responsive educational practices with diverse learners.  Therefore, one of the goals for the Fresno State PASC program is to develop a competency task during semester 2 of the program that would specifically highlight a candidate’s ability to engage in culturally responsive educational practices as part of the program redesign process.

Several steps have been identified to further develop candidates’ knowledge, skills, and abilities both academically and professionally in engagement in culturally responsive educational practices with diverse learners and do so in diverse cultural and socioeconomic community context.  Faculty will continue to function as a professional community of practice to develop additional learning experiences in providing more specificity and rationale for specific strategies identified in Leadership Cycle 2 of the CalAPA which will demonstrate a deeper level of understanding for underperforming students.  

Further, monthly faculty discussion will include the development of additional learning experiences for candidates to strengthen their ability to draw relationships between qualitative data collected to the quantitative data chosen in a specific student equity issue.  Lastly, there is a need to develop a competency task within the program that would specifically highlight a candidate’s ability to engage in culturally responsive educational practices and then reflect on the experience as part of the program redesign process.

To evaluate our efforts in the engagement in culturally responsive educational practices with diverse learners, faculty will set SMART goals based on data collected from the results of the consequential years of the CalAPA for Rubrics 1.2, 2.4 and 3.2 and then progress monitor the goals on a quarterly basis.

Lastly, the Fresno State PASC program will develop a competency task within semester 2 of the redesign scope and sequence that is specific to the demonstration of culturally responsive educational practices as part of the equity focus. 

Aspect C →