Standard 1 Aspect D

Standard 1d: Evidence shows that, by the time of program completion, candidates exhibit knowledge, skills, and abilities of professional educators appropriate to their target credential or degree, including: Assessment of and for student learning, assessment and data literacy, and use of data to inform practice

---

Data Sources & Analysis:

Data Source 1

CalAPA Leadership Cycle 1 (Rubric 1.2) 
After the second semester of the PASC program, candidates exhibit knowledge, skills, and abilities in the area of analyzing the assessment of and for student learning, assessment and data literacy, and use of data to inform practice through their fieldwork and coursework experiences that culminate with the submission of CalAPA Leadership Cycle 1. Rubric 1.2 evaluates the candidate’s ability to collect and analyze relevant qualitative data and explain their relation to quantitative data findings and the student group equity issue. Candidates work towards demonstrating a comprehensive analysis of a range of quantitative data sources over three years as well as follow up qualitative data to deepen their understanding of the equity issues surrounding the selected State-wide indicator and the student group equity issue.

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Skills assessed in Rubric 1.2 include how the candidate collects and analyzes relevant qualitative data and explain their relations to quantitative data findings and student group equity issues. Specifically, Rubric 1.2 is aligned to assessment of candidate performance of the following AAQEP 1d aligned CAPEs elements: 1A.2) Analyzing available student and school data from multiple sources to develop a site-specific vision and mission; 1B.1) Engaging staff and diverse community stakeholders in a collaborative process, including consensus building and decision making, to develop a vision of teaching and learning that is shared and supported by all stakeholders, and; 3B.3) Engaging staff and other stakeholders in using data to help establish, monitor, and evaluate the alignment and effectiveness of organizational processes to meet school goals and provide equitable access to opportunities for all students.

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3 as proficiency with the CTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 1

CalAPA Leadership Cycle 2, Rubric 1.2 for program level and state-wide mean scores by submission year

Link to Full Dataset:  CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Our analysis demonstrates that, overall, students are performing at the program expectation of 2 and above, and remain close to the State-wide average. Once the complete 2020-21 results are available, we are hopeful the mean scores on this rubric will continue to remain above the State-wide average. 

Evaluation of performance shows that candidates are able to collect data and identify patterns or gaps that highlight student inequities.  Current student mean scores demonstrate an area of need in a candidate’s ability to collect a wider range of data and more clearly analyze the relationship between qualitative data collected to the quantitative data chosen to show a deeper understanding of the context surrounding the selected State-wide indicator and the student group equity issue. Faculty will work together to explore ways of teaching and broaden resources to best support candidates on the aforementioned need as well as continue to work towards demonstrating a qualitative data collection strategy that is both responsive to the complex context in which they are completing their fieldwork and culturally sensitive with an appreciation for diverse viewpoints.

Data Source 2

CalAPA Leadership Cycle 3 (Rubric 3.4)
After the last semester of the PASC program candidates exhibit knowledge, skills, and abilities of professional educators through their fieldwork and coursework experiences culminating with the submission of CalAPA Leadership Cycle 3.  The focus of Rubric 3.4 is on the use of data to determine student learning and inform the teaching and learning process through the observation process with a volunteer teacher. Data include CSTP-focused candidate observation evidence, lesson observation video evidence, and student work samples. 

Perspective Captured from Data Source:
Assessment of candidate performance is evaluated by CTC selected, Pearson calibrated, and CA Administrative Services Credentialed faculty and school/district/county educational leaders involved in overseeing or preparing school administrators. 

Rationale for using Data Source:
Rubric 3.4 is aligned to assessment of candidate performance of the following AAQEP 1d aligned CAPEs elements: 2A.2) Involving staff in identifying areas of professional strength and development that link to accomplishing the school’s vision and goals to improve instruction and student learning; 2B.1) Identifying and using multiple types of evidence- based assessment measures and processes to determine student academic growth and success; 2C.3) Building a comprehensive and coherent system of professional learning focused on reaching the shared vision of equitable access to learning opportunities and resources and positive outcomes for all students, and; 5B.3) Involving stakeholders in reviewing aggregated and, where appropriate, disaggregated student data and evidence-based best practices to identify and address actual and anticipated challenges that can affect student success.

Definition of Success for Each Element:
Candidate success would be measured at Level 2 or Level 3 as proficiency with the CTC is currently at a Level 2; however, Fresno State faculty instruct toward all 5 levels with an emphasis on Level 3 or 4. In addition, maintaining mean scores above the state average by rubric will be considered as an element of success criteria.

Displays of Analyzed Data:

Table 2

CalAPA Leadership Cycle 3, Rubric 3.4 for program level and state-wide mean scores by submission year

Link to Full Dataset:  CalAPA Date AY 2018-2019, AY 2019-2020, YTD 2020-2021

Interpretation of Data:
Overall, candidates are meeting program assessment expectations scoring above an average of 2. Additionally, in two of the three cycles of data, candidates perform above the state-wide average for Rubric 3.4.  Evidence demonstrates the ability of candidates to have a focused two-way conversation with a volunteer teacher they have observed teaching to support professional growth.  Scores also indicate that, during the conversations with volunteer teacher, candidates are able to provide standard and evidence-based feedback to help identify strengths and weaknesses of the teaching and learning process through the analysis of multiple data sources including video lesson evidence, focused observation notes, and student work samples.  Further faculty planning for instruction and resources on how candidates can better demonstrate the ability to connect feedback from the previous components of the observation cycle, such as the pre-observation, CSTP specific evidence and student learning outcomes, to the post-lesson coaching meeting as well as how to offer evidence-based professional learning choices to the volunteer teacher that may lead to improvements in practice or meet student needs will continue to be explored.

Data Source 3

P12 PASC Program AAQEP Candidate Self-Assessment 

Perspective Captured from Data Source:
Candidate retrospective self-report of AAQEP Standard 1a at each semester end interval. 

Specific Elements of Data Source:
Item 1D: I understand assessment of and for student learning, and incorporate other assessments with data literacy to inform my leadership practices and approaches.

Rationale for using Data Source:
The P12 PASC Program AAQEP Candidate Self-Assessment was designed to capture student reflection and growth after each semester in the program in alignment with AQQEP aspects. The results capture one cohort (program graduates in May 2021) over the 3-semester program using a retrospective design to pilot the self-assessment and obtain initial data. This will now be used at the end of each semester as an additional program assessment data point used for reflection and continuous improvement.

Definition of Success for Each Element:
Candidate success would be measured by 1) average of 3 or higher for each semester and 2) evidence of mean growth from semester 1 to semester 3. 

Displays of Analyzed Data:

Table 3

Table of AAQEP Standard 1d Candidate Self-Assessment Response Means for Cohort 0 (Pilot)

Link to Full Dataset: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 1 by Semester 

Interpretation of Data:
Overall, student reported mean scale scores for Standard 1d show some growth from initial to end of program data points. Evidence demonstrates that candidates feel they understand assessment of and for student learning, and incorporate other assessments with data literacy to inform their leadership practices and approaches at a high level (between a 4 and 5) which exceeds our success criteria of level 3. 

Next Steps for 1d
Based on the analysis of the four data point interpretation sections, the following next steps are recommended:

Next Steps Narrative:
In each data point, evidence exists that candidates are able to demonstrate the current level of proficiency of the knowledge and skills required for aspiring administrators. Specifically, the Fresno State PASC program is supporting knowledge, skills, and abilities of aspiring school leaders regarding assessment of and for student learning, assessment and data literacy, and use of data to inform practice as evident within candidate’s course work and field work experiences as evidenced across data points. 

To support our cycle of continuous improvement for Standard 1d, the program will focus on three key areas as noted in the Table above. Over the past several years faculty have engaged in an intense realignment process to intentionally backwards map the California Administrator Performance Expectations and CalAPA skills, which includes deeper alignment to becoming data-informed and student-learning centered school leaders who work to cultivate systems which are monitored for student-equity by data-driven cycles of continuous improvement. The revised course outcomes, scope and sequence, and embedded fieldwork expectations will be implemented starting with the Fall 2021 cohorts. 

Several steps have been identified to further develop candidates’ knowledge, skills, and abilities both academically and professionally in the area of the assessment of and for student learning, assessment and data literacy, and use of data to inform practice. Faculty will continue to function as a professional community of practice to develop additional learning experiences in the following areas: the collection and analysis of qualitative and quantitative data to support the identification of a problem of practice, the identification and the relationship between qualitative data collected to the quantitative data chosen in a specific student equity issue, and how to connect the CSTPs and student outcomes from a teacher observation.

To monitor program efforts in this area, faculty will analyze CalAPA results based on SMART goals on a quarterly basis and discuss instructional strategies that have produced results. Furthermore, candidates will be surveyed for feedback on instructional practices and experiences in each course that supports the three leadership cycles.

Aspect E →