Appendix E: Evidence of Data Quality

Bilingual Authorization Program

BAP Completer Survey

Quantitative Data Measure: BAP completer survey
Description of Measure The BAP completer survey has four (4) sections: past education, experience in credential, experience in BAP, and employment. The main purpose of this survey is to attain feedback from recent graduates regarding their experiences in the BAP and credential programs. 
Evidence (or plans) regarding validity The first draft of this survey was created and disseminated in spring 2021 and plans to achieve validity for this survey include workshopping in the BAP advisory committee so that all stakeholders, who represent district partners, clinical practice coaches, counselors, multiple subject credential program coordinator, liberal studies chair, and faculty, can assist and ensure that state bilingual authorization standards, AAQEP standards, teacher performance expectations (TPEs), and other related standards and measures used to  assess bilingual authorization programs are met. 
Evidence (or plans) regarding reliability The plan to achieve reliability for this survey includes solidifying a final draft to disseminate every year. During the 2021-2022 academic year, the BAP advisory committee will work on several drafts of this survey to ensure everything from questions asked, to wording, are agreed upon by all stakeholders. It will also be workshopped for feedback among non-advisory stakeholders, like current students, to make sure that respondents understand questions clearly. This will ensure that the survey results will not vary too drastically each time data is collected. In addition, once this survey is dessimmated, reliability for this instrument will be confirmed by analyzing the number of respondents and the extent that responses are answered similarly. For example, the percentage of specified answers to each question will be analyzed in order to estimate reliability by using confidence intervals. 
Evidence (or plans) regarding fairness Fairness for this survey will be established because it will be disseminated among program graduates upon completion who have completed all the same requirements as indicated by state bilingual authorization standards, moreover, the rigorous engagement of the program advisory committee in creating and finalizing this survey further strengthens its fairness measure.

BAP Employer Survey

Quantitative Data Measure:  BAP employer survey
Description of Measure The current rough draft of the employer survey has two (2) sections, district context, and teacher performance. The main purpose of this survey is to get feedback from employers that have hired BAP graduates in their district/school. Focus is on graduate preparedness as indicated by AAQEP standard 1a-f. 
Evidence (or plans) regarding validity The plan to achieve validity for this draft survey includes workshopping in the BAP advisory committee so that all stakeholders, who represent district partners, clinical practice coaches, counselors, multiple subject credential program coordinator, liberal studies chair, and faculty, can assist and ensure that state bilingual authorization standards, AAQEP standards, teacher performance expectations (TPEs), and other related standards and measures used to  assess bilingual authorization programs are met. 
Evidence (or plans) regarding reliability The plan to achieve reliability for this survey includes solidifying a final draft to disseminate every year. During the 2021-2022 academic year, the BAP advisory committee will work on several drafts of this survey to ensure everything from questions asked, to wording, are agreed upon by all stakeholders. It will also be workshopped for feedback among non-advisory stakeholders, like current students, to make sure that respondents understand questions clearly. This will ensure that the survey results will not vary too drastically each time data is collected. In addition, once this survey is dessimmated, reliability for this instrument will be confirmed by analyzing the number of respondents and the extent that responses are answered similarly. For example, the percentage of specified answers to each question will be analyzed in order to estimate reliability by using confidence intervals. 
Evidence (or plans) regarding fairness Fairness for this survey will be established because it will be disseminated among district partners and additional known districts that hire BAP graduates, moreover, the rigorous engagement of the program advisory committee, which include employing districts, in creating and finalizing this survey further strengthens its fairness measure.

BAP Alumni Survey

Quantitative Data Measure:  BAP alumni survey
Description of Measure The current rough draft of the alumni survey has two (2) sections, employment, and professional competence and growth. The main purpose of this survey is to follow up with BAP graduates one (1+) years after completing the program to determine if they are employed as bilingual/dual immersion teachers as well as assessing the maintenance of AAQEP professional standards 1a-f. 
Evidence (or plans) regarding validity The plan to achieve validity for this draft survey includes workshopping in the BAP advisory committee so that all stakeholders, who represent district partners, clinical practice coaches, counselors, multiple subject credential program coordinator, liberal studies chair, and faculty, can assist and ensure that state bilingual authorization standards, AAQEP standards, teacher performance expectations (TPEs), and other related standards and measures used to  assess bilingual authorization programs are met. 
Evidence (or plans) regarding reliability The plan to achieve reliability for this survey includes solidifying a final draft to disseminate every year. During the 2021-2022 academic year, the BAP advisory committee will work on several drafts of this survey to ensure everything from questions asked, to wording, are agreed upon by all stakeholders. It will also be workshopped for feedback among non-advisory stakeholders, like current students, to make sure that respondents understand questions clearly. This will ensure that the survey results will not vary too drastically each time data is collected. In addition, once this survey is dessimmated, reliability for this instrument will be confirmed by analyzing the number of respondents and the extent that responses are answered similarly. For example, the percentage of specified answers to each question will be analyzed in order to estimate reliability by using confidence intervals. 
Evidence (or plans) regarding fairness Fairness for this survey will be established because it will be disseminated among program graduates upon completion who have completed all the same requirements as indicated by state bilingual authorization standards, moreover, the rigorous engagement of the program advisory committee in creating and finalizing this survey further strengthens its fairness measure.

LEE 136 Spanish Lesson Plan

Quantitative Data Measure: LEE 136 Spanish Lesson Plan
Description of Measure A key assignment from the LEE 136 course for the Spanish BAP pathway is the creation of a Spanish lesson plan that encompasses the tenets of culturally and linguistically sustaining pedagogy around a central instructional focus or theme and at least one children’s text. The central focus takes into account knowledge of students’ language development, backgrounds, interests, and learning levels that might further influence students’ thinking and learning. This lesson assignment also embodies the four (4) learning goals of the course, which include: 1) the impact of language and culture on teaching and learning in the elementary school, 2) language acquisition theory, socio-cultural context in teaching and instructional strategies for Emergent Bilinguals in the classroom, and 3) strategies to promote student success, including achievement of Common Core state-adopted content and English Language Development (ELD) standards. 
Evidence (or plans) regarding validity The lesson plan rubric demonstrates alignment between CCSS and language objectives, as well as the learning tasks, and assessments that are related to an identifiable theme or topic.
Evidence (or plans) regarding reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric. 
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Evidence (or plans) regarding fairness All students have access to the assignment requirements and rubric ahead of time. 

LEE 136 Literacy, Language, and Culture Story/Project

Quantitative Data Measure:  LEE 136 Literacy, Language, and Culture Story/Project
Description of Measure To further develop their learning about learners, in LEE 136, candidates complete a literacy, language, and culture project with a focal student. In this assignment, candidates identify one focal student to observe and get to know, preferably a child who is labeled Emergent Bilingual (also called English Language Learner (ELL)). Candidates work alongside their focal student and learn about her/his interests, literacy, language, and culture. They are required to take detailed notes about their conversations after all interactions and then discuss the student’s profile during class time. At the end of the semester, candidates prepare  a shadowbox with their student and submit a 1-page, single-spaced narrative about the student.
Evidence (or plans) regarding validity The focus of this course is teaching content in Spanish. We believe this assignment is valid for this focus because it asks the candidate to work with an emergent bilingual student, speaking with them in Spanish to build a relationship with the student, learn about her/his interests, and create the project together.
Evidence (or plans) regarding reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric. 
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Evidence (or plans) regarding fairness All students have access to the assignment requirements and rubric ahead of time. 

Appendix E: