Skip to main content Skip to main navigation Skip to footer content

AAQEP Accreditation 2022

Appendix E: Evidence of Data Quality

Preliminary Administrative Services Credential

Quantitative Data Measure: California Administrative Professionals Assessment
Description of Measure

Candidates in each administrative services credential program in California must demonstrate content and skill knowledge on three performance assessments prior to the attainment of a preliminary credential.  Each performance assessment or leadership cycle contains required knowledge and skills found in the California Administrator Performance Expectations (CAPEs).  The CalAPA consists of three leadership cycles that focus candidates on specific content knowledge, a different set of skills, and knowledge to demonstrate new learning relevant to school administration.  Each cycle includes professional dispositions that school administrators should know and be able to demonstrate.  There are 4 steps in each CalAPA performance assessment which include the following actions:  Investigate, Plan, Act, Reflect in a continuous improvement model based on collected data from fieldwork school sites.
Consequential implementation of the CalAPA began in Fall 2019.  Based on this new assessment the Fresno State PASC program has gone through a series of redesign processes to better align student expectations and outcomes to the state standards and assessment.   Each leadership cycle includes either 7 or 8 analytic rubrics that measures knowledge and skills candidates must evidence in narrative writing, document artifacts, and/or video. Each rubric is aligned to specific CAPEs organized by a scale from Level 1 to Level 5.  Based on the overall passage score for each cycle, candidate success or proficiency would be measured at Level 2.  As of 2019, each CalAPA leadership cycle has a summative passing score as follows:

  • Cycle 1 (8 rubrics): A final cut score of 14 points.
  • Cycle 2 (7 rubrics): A final cut score of 12 points.
  • Cycle 3 (7 rubrics): A final cut score of 12 points.
Evidence (or plans) regarding trustworthiness The CalAPA was developed and field-tested through a collaboration of the CCTC and Pearson. School administrators with clear credentials front throughout the state were brought together in working groups to determine what content would be included on the assessment. Once items were determined, the assessment then went through a pilot year during which each cycle was field-tested. Results were analyzed and the cycles were revised. The assessment then went through a second, nonconsequential year of testing to ensure validity and reliability of items.
Annually, CalAPA administrators seek input from preliminary Administrative Services Credential coordinators throughout the state to learn how students are doing on the exam. This feedback is then used to revise items to ensure the assessment is as valid and fair as possible.
Evidence (or plans) regarding reliability All raters are calibrated at least annually to ensure reliable scoring of assessment measures.
Raters must hold a California Clear Administrative Credential.
All faculty from within the Fresno State PASC program are fully calibrated scorers who participate in scoring of the state-level exams.
Evidence (or plans) regarding fairness Cut-off scores are determined annually to ensure pass rate does not disproportionately impact administrative credential candidates from diverse backgrounds.
Fresno State PASC program faculty use what they learn from their involvement in the scoring of the CalAPA to inform their instruction in the PASC program

Quantitative Data Measure: President’s Commission of Teacher Education Questionnaire
Description of Measure President’s Commission of Teacher Education Questionnaire. During university and local partners annual face-to-face meetings, the President's Commission on Teacher Education, a pilot 3-item open response feedback questionnaire was used to collect employer perceptions based on observed strengths of the Fresno State PASC graduates. The responses are analyzed for themes in relation to the CAPEs and AAQEP standards.
Evidence (or plans) regarding trustworthiness We let completers know their responses are anonymous and only utilized for program improvement to help with trustworthiness. 
Evidence (or plans) regarding fairness In order to get a true representation of superintendent voices across the central valley, we would like to send this questionnaire or an adapted version of this questionnaire via email too for those who were not in attendance to the President’s Commission of Teacher Education. In addition, we want to also share this questionnaire in the P12 Superintendent Advisory Council as well. 

Measure: CCTC Completer Survey
Evidence regarding validity The survey was designed to align with the California Professional Standards for Education Leaders (CPSEL), which are aligned with the California Administrative Performance Expectations (CAPEs). All survey items have been validated by the administrative bodies to determine their validity and are standardized across the larger systems
Evidence regarding reliability All survey items are assessed annually by the CTC to determine their reliability based on the responses received from program completers.
Evidence regarding Fairness/Trustworthiness

Because this instrument is administered statewide to candidates of all programs (CSU, University of California, private institutions, and local education agencies), we believe in its fairness as an instrument. Additionally, we believe the data the instrument yields are trustworthy because the survey is administered by the CTC, the statewide accrediting agency.

Fresno State has initiated a college-wide data summit to consider the findings of this statewide survey and triangulate them with campus data, including the percentage of First Generation students, access to resources like scholarships, and culture and context of the cohorts in which prospective teachers are placed. Through this triangulation process, we are able to determine the alignment of the finding from the survey with our other measures, further assuring us of the survey’s trustworthiness as an instrument. In the process, we are also able to inform the impact on program changes on our own students with respect to the unique diversity of culture and needs in the Central Valley.  

Quantitative Data Measure: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 1 by Semester 
Description of Measure P12 PASC Program AAQEP Candidate Self-Assessment. The purpose of this self-assessment is to have students reflect on their perceptions of personal growth in the CAPEs and AAQEP standards over time in the PASC program. The information from this self-assessment is utilized to inform both areas of strength and opportunities for growth for students, as well as inform programmatic and instructional decision-making for ongoing continuous improvement. Each semester students are provided the opportunity to grow professionally as an aspiring equity driven school leader. Based on overall program coursework, activities, readings, and discussions, as well as fieldwork experiences through the CalAPA and other supervised activities with your field mentor supervision. This assessment is designed to help students reflect on a wide range of knowledge and experiences essential to their preparation. This self-assessment is divided into three sections: 1) AAQEP Program Standards Assessment, 2) CTC CAPEs Self-Assessment , and 3) Written Narrative Reflective Response.
Evidence (or plans) regarding validity Measure items are aligned with CTC and AAQEP standards. We informally tested the content with a pilot group of candidates who were finishing their final semester in the program to ensure understanding of the items. We can conduct a more formal pilot study of the items with a random group of candidates to ensure content clarity and understanding of what the items are asking. 
Evidence (or plans) regarding reliability We utilize a controlled testing setting where all students are required to complete the assessment within a course structure during course time. Also, as more candidates respond to the survey and the sample size increases, we can consider running some basic statistical analysis on the items. 
Evidence (or plans) regarding fairness Ensuring all candidates who have completed have an opportunity to share reflections and other needs every semester consistently. 

Appendix E: