Standard 1 Aspect A

Standard 1a: Evidence shows that, by the time of program completion, candidates exhibit knowledge, skills, and abilities of professional educators appropriate to their target credential or degree, including: Content, pedagogical, and/or professional knowledge relevant to the credential or degree sought

---

Data Sources & Analysis:

Data Source 1

CalAPA Leadership Cycle 2 (Rubric 2.3)
After the first semester of the PASC program, candidates exhibit content, pedagogical, and professional knowledge through their fieldwork and coursework experiences that culminate with the submission of CalAPA Leadership Cycle 2.  The focus of Rubric 2.3 is on the facilitation of a community of practice to develop a problem of practice related to student learning and/or well-being that is based on school data where candidates exhibit knowledge, skills, and abilities of aspiring school leaders.

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Specifically, Rubric 2.3 is aligned to assess candidate performance on the following AAQEP 1a aligned CAPEs: 1C)  Recognizing and explaining to staff and other stakeholders how the school vision guides planning, decision-making, and the change processes required to continuously improve teaching and learning; 2A) Recognizing that professional growth is an essential part of the shared vision to continuously improve the school, staff, student learning, and student safety and well-being; 2D) Knowing and understanding P–12 student content standards and frameworks, P–12 performance expectations, and aligned instructional and support practices focused on providing equitable learning opportunities so that all students graduate ready for college and careers; 5B) Developing and knowing how to use professional influence with staff, students, and community to develop a climate of trust, mutual respect, and honest communication necessary to consistently make fair and equitable decisions on behalf of all students, and; 5C) Understanding that how they carry out professional obligations and responsibilities affects the entire school community. Thus, Rubric 2.3 has been chosen as an appropriate measure for the candidate demonstration of content, pedagogical, and professional knowledge necessary for aspiring school leaders.

Specific Elements of Data Source Using:
The CalAPA rubrics are designed as 5-level rubrics, with the lowest score of 1 through 5.  The first data source is from Leadership Cycle 2, Rubric 2.3, which evaluates the candidate’s ability to collaboratively work with a community of practice and build group ownership to select a problem of practice based on the agreed-upon area of educational focus and related to student learning and/or well-being. Candidates also work towards differentiating the academic performance for student groups and issue of well-being, providing supporting student data and explaining how they co-facilitated the group to collaboratively address these differences in defining the problem of practice. Candidates at the highest level also explain how and why they co-facilitated the collaborative process with the group members to analyze student data, respected diverse viewpoints of the group, and co-identified the problem of practice. 

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3, as proficiency with the CTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 1
CalAPA Leadership Cycle 2, Rubric 2.3 for program level and state-wide mean scores by submission year

Link to Full Dataset: CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Three years of data in relation to Rubric 2.3 highlight that program candidates are overall meeting the current program expectation of 2 and above and performing just above the State-wide average each year.  Overall, candidate responses show the candidates’ have acquired a proficient level of content, pedagogical, and professional knowledge to collaboratively work within a community of practice and build some group ownership to select a problem of practice based on the agreed-upon area of educational focus and related to student learning and/or well-being. 

Current student mean scores also demonstrate supporting candidates at higher levels includes the differentiation of academic performance for student groups and issue of well-being, and providing supporting student data. In addition, scores indicate candidates need support on more clearly explaining how they co-facilitated the group to collaboratively address these differences in defining the problem of practice. Further, faculty will need to continue to find resources and plan explicit instruction to support candidate understanding and explanation of how and why they co-facilitated the collaborative process with the group members to analyze student data,  respected diverse viewpoints of the group, and co-identified the problem of practice in order to deepen program acquired content, pedagogical, and professional knowledge relevant for aspiring school leaders.

Data Source 2:

CalAPA Leadership Cycle 1 (Rubric 1.1)
After the second semester of the PASC program, candidates exhibit knowledge, skills, and abilities through their fieldwork and coursework experiences culminating with the submission of CalAPA Leadership Cycle 1. 

The second data source is from the CalAPA Leadership Cycle 1, Rubric 1.1 which is to used evaluate the candidate’s ability to select a California State indicator (student absenteeism, suspension rate, English learner progress, graduates rate, academic performance, college/career readiness) and analyze quantitative data across three years to identify trends related to school equity for one student group. Candidates are also evaluated on how clearly they make connections between the data analysis and specific components of the school’s vision, mission, and/or goals. From this foundation, candidates work towards exploring additional data linked to the indicator to support their trend analysis and further understand the group differences (i.e., language, ethnicity, gender) within the selected state indicator. Candidates at the highest level also cite relevant research that supports the trend analysis related to equity and clearly explains why the cited research informs their understanding of the observed equity gap.   

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 1.1 is aligned to assess candidate performance on the following AAQEP 1a aligned CAPEs elements: 1A.2) Analyzing available student and school data from multiple sources to develop a site-specific vision and mission, and; 1C.3) Collecting, analyzing, and using multiple sources of data for ongoing monitoring to determine whether the plan is helping staff and stakeholders move toward the school’s vision. Thus, Rubric 1.1 has been chosen as an appropriate measure for the demonstration of candidate  content, pedagogical, and professional knowledge as aspiring school leaders.

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3 as proficiency with the CTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 2

CalAPA Leadership Cycle 1, Rubric 1.1 for program level and state-wide mean scores by submission year

Link to Full Dataset: CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Overall, students perform at the rubric expectation.  The 2020 data is for half of the year; therefore, full analysis is difficult.  Candidate evidence shows that candidates are able to collect data and identify patterns or gaps that highlight student inequities.  Further instruction on how to triangulate quantitative and qualitative data to demonstrate a sophisticated analysis of equity disparity among groups of students would be a future emphasis for Leadership Cycle 1.  Based on specific instruction for data triangulation and data analysis, we anticipate the mean average for Rubric 1.1 will increase.

Data Source 3:

CalAPA Leadership Cycle 3 (Rubric 3.4) 
After the last semester of the PASC program candidates exhibit content, pedagogical, and professional knowledge, skills, and abilities through their fieldwork and coursework experiences culminating with the submission of CalAPA Leadership Cycle 3.  The third data source is from the CalAPA Leadership Cycle 3, Rubric 3.4, which is used to evaluate how the candidate fosters a learning conversation in a post-observation meeting using CSTP-focused observation evidence, lesson observation video, and student work with the volunteer teacher regarding strengths and area(s) for growth. The candidate is working to cultivate a two-way conversation citing the CSTP specific observation notes and lesson video as well as student work. In addition, Candidates should demonstrate respectful and supportive joint identification of volunteer teacher strengths and areas for growth based on gathered observation evidence and student work. Candidates work to establish a clear partnership with the volunteer teacher as they discuss connections between the pre-observation meeting considerations, the lesson observation evidence, and the student learning. 

The focus of Rubric 3.4 is to measure candidate content, pedagogical and professional knowledge and skills using instructional coaching tools during the post-observation meeting between the candidate and volunteer teacher. These meetings incorporate the California Standards for the Teaching Profession (CSTP), lesson observation videos, and student work to identify teaching strengths and areas for growth which are all expected knowledge, skills, and abilities exhibited by aspiring school leaders.

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 3.4 is aligned to assessment of candidate performance of the following AAQEP 1a aligned CAPEs elements: 2A.1) Using the California Standards for the Teaching Profession (CSTP) for teachers to describe and set expectations for growth and performance; 2B.3) Supporting and promoting effective instruction and a range of instructional methods and supporting practices that address the diverse educational needs of all students; 2C.2) Using state-adopted professional standards (e.g., CSTP) with staff as a foundation to guide professional learning; 2D.3) Providing timely, constructive suggestions about instructional strategies and assessments, available resources, and technologies to teachers in order to refine and enhance instruction and assessment that supports student learning, safety, and well-being, and; 5B.2) Guiding staff in examining issues that may affect accomplishment of the school’s vision, mission, and goals, including issues that may be related to race, diversity, and access. Thus, Rubric 3.4 is a suitable measurement for candidate program acquired content, pedagogical, and professional knowledge, skills, and abilities as aspiring school administrators.

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3 as proficiency with the CTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 3

CalAPA Leadership Cycle 3, Rubric 3.4 for program level and state-wide mean scores by submission year

Link to Full Dataset: CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Three years of data in relation to Rubric 3.4 highlights that program candidates are overall meeting the current program expectation of 2 and above and are overall performing just above the State-wide average, with the exception of year two. 

Current evaluation of performance based on mean scores highlights the ability of candidates to hold a focused conversation with a volunteer teacher they have observed teaching.  During the conversation, candidates have provided CSTP-focused evidence or feedback to help identify strengths and weaknesses for the observed teacher.  

Current student mean scores also demonstrate an area of need to further faculty instruction on strengthening candidate skills related to fostering a two-way conversation and more explicitly connect the components of an observation cycle, such as the pre-observation, CSTP specific evidence and student learning outcomes. Further faculty planning for instruction and resources is also needed to assist candidates in preparing and then sharing evidence-based instructional practices related to observation findings to support the feedback conversation and to offer choices to the volunteer teacher that may lead to improvements in practice or meet student needs.

Data Source 4

P12 PASC Program AAQEP Candidate Self-Assessment 

Perspective Captured from Data Source:
Candidate retrospective self-report of AAQEP Standard 1a at each semester end interval. 

Specific Elements of Data Source Using:
Item 1a: I believe the content, pedagogical, and professional knowledge is relevant to the credential or degree sought.

Rationale for using Data Source:
The P12 PASC Program AAQEP Candidate Self-Assessment was designed to capture student reflection and growth after each semester in the program in alignment with AAQEP aspects. The results capture one cohort (program graduates in May 2021) over the 3-semester program using a retrospective design to pilot the self-assessment and obtain initial data. This will now be used at the end of each semester as an additional program assessment data point used for reflection and continuous improvement.

Definition of Success for Each Element:
Candidate success would be measured by 1) average of 3 or higher for each semester and 2) evidence of mean growth from semester 1 to semester 3. 

Displays of Analyzed Data:

Table 4

Table of AAQEP Standard 1a Candidate Self-Assessment Response Means for Cohort 0 (Pilot)

Link to Full Dataset: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 1 by Semester 

Interpretation of Data:
Overall, student reported mean scale scores for Standard 1a show some growth. Evidence demonstrates that candidates believe the content, pedagogical, and professional knowledge gained is relevant to the credential or degree sought at a high levels (between a 4 and 5) which exceeds our success criteria of level 3. 

Next Steps for 1a
Based on the analysis of the four data point interpretation sections, the following next steps are recommended: 

Next Steps Narrative:
In each data point, evidence exists that candidates are able to demonstrate the current level of proficiency of the knowledge and skills required for aspiring administrators. Specifically, the Fresno State PASC program is supporting academic content, pedagogical, and/or professional knowledge relevant to the credential or degree is evident within candidate’s course work and field work experiences as evidenced across data points. 

To support our cycle of continuous improvement for Standard 1a, the program will focus on three key areas as noted in the Table above. Over the past several years faculty have engaged in an intense realignment process to intentionally backwards map the California Administrator Performance Expectations and CalAPA skills.  The revised course outcomes, scope and sequence, and embedded fieldwork expectations will be implemented starting with the Fall 2021 cohorts. 

Faculty will also work as a learning community to improve candidate outcomes, specifically mean rubric scores by evaluating individual and cohort CalAPA data by rubric and overall candidate scores.  Currently, out of the six full-time tenure track faculty that teach in this program, four professors have gone through the CalAPA assessor training in order to score candidate responses.  Other faculty have participated in CTC specific professional development to help develop additional instructional practice to teach the knowledge and skills necessary for aspiring administrators as per the CAPEs and CalAPA.  Lastly, faculty that teach common courses work together from a common syllabus, with common competency tasks and rubrics that have been developed for equity throughout the PASC program.

Additional actions faculty will take to improve mean rubric scores is to continue monthly collaboration meetings with an instructional focus on best practices, the development of instructional lessons to increase the average rubric scores for candidates.

To monitor program efforts in this area, faculty will analyze CalAPA results based on SMART goals on a quarterly basis and discuss instructional strategies that have produced results.  Furthermore, candidates will be surveyed for feedback on instructional practices and experiences in each course that supports the three leadership cycles.

Aspect B →