AAQEP Accreditation 2022
Appendix E: Evidence of Data Quality
Preliminary Administrative Services Credential
|Quantitative Data Measure: California Administrative Professionals Assessment|
|Description of Measure||
Candidates in each administrative services credential program in California must demonstrate
content and skill knowledge on three performance assessments prior to the attainment
of a preliminary credential. Each performance assessment or leadership cycle contains
required knowledge and skills found in the California Administrator Performance Expectations
(CAPEs). The CalAPA consists of three leadership cycles that focus candidates on
specific content knowledge, a different set of skills, and knowledge to demonstrate
new learning relevant to school administration. Each cycle includes professional
dispositions that school administrators should know and be able to demonstrate. There
are 4 steps in each CalAPA performance assessment which include the following actions:
Investigate, Plan, Act, Reflect in a continuous improvement model based on collected
data from fieldwork school sites.
|Evidence (or plans) regarding trustworthiness||The CalAPA was developed and field-tested through a collaboration of the CCTC and
Pearson. School administrators with clear credentials front throughout the state were
brought together in working groups to determine what content would be included on
the assessment. Once items were determined, the assessment then went through a pilot
year during which each cycle was field-tested. Results were analyzed and the cycles
were revised. The assessment then went through a second, nonconsequential year of
testing to ensure validity and reliability of items.
Annually, CalAPA administrators seek input from preliminary Administrative Services Credential coordinators throughout the state to learn how students are doing on the exam. This feedback is then used to revise items to ensure the assessment is as valid and fair as possible.
|Evidence (or plans) regarding reliability||All raters are calibrated at least annually to ensure reliable scoring of assessment
Raters must hold a California Clear Administrative Credential.
All faculty from within the Fresno State PASC program are fully calibrated scorers who participate in scoring of the state-level exams.
|Evidence (or plans) regarding fairness||Cut-off scores are determined annually to ensure pass rate does not disproportionately
impact administrative credential candidates from diverse backgrounds.
Fresno State PASC program faculty use what they learn from their involvement in the scoring of the CalAPA to inform their instruction in the PASC program
|Quantitative Data Measure: President’s Commission of Teacher Education Questionnaire|
|Description of Measure||President’s Commission of Teacher Education Questionnaire. During university and local partners annual face-to-face meetings, the President's Commission on Teacher Education, a pilot 3-item open response feedback questionnaire was used to collect employer perceptions based on observed strengths of the Fresno State PASC graduates. The responses are analyzed for themes in relation to the CAPEs and AAQEP standards.|
|Evidence (or plans) regarding trustworthiness||We let completers know their responses are anonymous and only utilized for program improvement to help with trustworthiness.|
|Evidence (or plans) regarding fairness||In order to get a true representation of superintendent voices across the central valley, we would like to send this questionnaire or an adapted version of this questionnaire via email too for those who were not in attendance to the President’s Commission of Teacher Education. In addition, we want to also share this questionnaire in the P12 Superintendent Advisory Council as well.|
|Measure: CCTC Completer Survey|
|Evidence regarding validity||The survey was designed to align with the California Professional Standards for Education Leaders (CPSEL), which are aligned with the California Administrative Performance Expectations (CAPEs). All survey items have been validated by the administrative bodies to determine their validity and are standardized across the larger systems|
|Evidence regarding reliability||All survey items are assessed annually by the CTC to determine their reliability based on the responses received from program completers.|
|Evidence regarding Fairness/Trustworthiness||
Because this instrument is administered statewide to candidates of all programs (CSU, University of California, private institutions, and local education agencies), we believe in its fairness as an instrument. Additionally, we believe the data the instrument yields are trustworthy because the survey is administered by the CTC, the statewide accrediting agency.
Fresno State has initiated a college-wide data summit to consider the findings of this statewide survey and triangulate them with campus data, including the percentage of First Generation students, access to resources like scholarships, and culture and context of the cohorts in which prospective teachers are placed. Through this triangulation process, we are able to determine the alignment of the finding from the survey with our other measures, further assuring us of the survey’s trustworthiness as an instrument. In the process, we are also able to inform the impact on program changes on our own students with respect to the unique diversity of culture and needs in the Central Valley.
|Quantitative Data Measure: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 1 by Semester|
|Description of Measure||P12 PASC Program AAQEP Candidate Self-Assessment. The purpose of this self-assessment is to have students reflect on their perceptions of personal growth in the CAPEs and AAQEP standards over time in the PASC program. The information from this self-assessment is utilized to inform both areas of strength and opportunities for growth for students, as well as inform programmatic and instructional decision-making for ongoing continuous improvement. Each semester students are provided the opportunity to grow professionally as an aspiring equity driven school leader. Based on overall program coursework, activities, readings, and discussions, as well as fieldwork experiences through the CalAPA and other supervised activities with your field mentor supervision. This assessment is designed to help students reflect on a wide range of knowledge and experiences essential to their preparation. This self-assessment is divided into three sections: 1) AAQEP Program Standards Assessment, 2) CTC CAPEs Self-Assessment , and 3) Written Narrative Reflective Response.|
|Evidence (or plans) regarding validity||Measure items are aligned with CTC and AAQEP standards. We informally tested the content with a pilot group of candidates who were finishing their final semester in the program to ensure understanding of the items. We can conduct a more formal pilot study of the items with a random group of candidates to ensure content clarity and understanding of what the items are asking.|
|Evidence (or plans) regarding reliability||We utilize a controlled testing setting where all students are required to complete the assessment within a course structure during course time. Also, as more candidates respond to the survey and the sample size increases, we can consider running some basic statistical analysis on the items.|
|Evidence (or plans) regarding fairness||Ensuring all candidates who have completed have an opportunity to share reflections and other needs every semester consistently.|