Appendix E: Evidence of Data Quality

Preliminary Administrative Services Credential

California Administrative Professionals Assessment (CalAPA)

Quantitative Data Measure: California Administrative Professionals Assessment
Description of Measure

Candidates in each administrative services credential program in California must demonstrate content and skill knowledge on three performance assessments prior to the attainment of a preliminary credential.  Each performance assessment or leadership cycle contains required knowledge and skills found in the California Administrator Performance Expectations (CAPEs).  The CalAPA consists of three leadership cycles that focus candidates on specific content knowledge, a different set of skills, and knowledge to demonstrate new learning relevant to school administration.  Each cycle includes professional dispositions that school administrators should know and be able to demonstrate.  There are 4 steps in each CalAPA performance assessment which include the following actions:  Investigate, Plan, Act, Reflect in a continuous improvement model based on collected data from fieldwork school sites.
Consequential implementation of the CalAPA began in Fall 2019.  Based on this new assessment the Fresno State PASC program has gone through a series of redesign processes to better align student expectations and outcomes to the state standards and assessment.   Each leadership cycle includes either 7 or 8 analytic rubrics that measures knowledge and skills candidates must evidence in narrative writing, document artifacts, and/or video. Each rubric is aligned to specific CAPEs organized by a scale from Level 1 to Level 5.  Based on the overall passage score for each cycle, candidate success or proficiency would be measured at Level 2.  As of 2019, each CalAPA leadership cycle has a summative passing score as follows:

  • Cycle 1 (8 rubrics): A final cut score of 14 points.
  • Cycle 2 (7 rubrics): A final cut score of 12 points.
  • Cycle 3 (7 rubrics): A final cut score of 12 points.
Evidence (or plans) regarding trustworthiness The CalAPA was developed and field-tested through a collaboration of the CCTC and Pearson. School administrators with clear credentials front throughout the state were brought together in working groups to determine what content would be included on the assessment. Once items were determined, the assessment then went through a pilot year during which each cycle was field-tested. Results were analyzed and the cycles were revised. The assessment then went through a second, nonconsequential year of testing to ensure validity and reliability of items.
Annually, CalAPA administrators seek input from preliminary Administrative Services Credential coordinators throughout the state to learn how students are doing on the exam. This feedback is then used to revise items to ensure the assessment is as valid and fair as possible.
Evidence (or plans) regarding reliability All raters are calibrated at least annually to ensure reliable scoring of assessment measures.
Raters must hold a California Clear Administrative Credential.
All faculty from within the Fresno State PASC program are fully calibrated scorers who participate in scoring of the state-level exams.
Evidence (or plans) regarding fairness Cut-off scores are determined annually to ensure pass rate does not disproportionately impact administrative credential candidates from diverse backgrounds.
Fresno State PASC program faculty use what they learn from their involvement in the scoring of the CalAPA to inform their instruction in the PASC program

President’s Commission on Teacher Education (Employer Questionnaire)

Quantitative Data Measure: President’s Commission of Teacher Education Questionnaire
Description of Measure President’s Commission of Teacher Education Questionnaire. During university and local partners annual face-to-face meetings, the President's Commission on Teacher Education, a pilot 3-item open response feedback questionnaire was used to collect employer perceptions based on observed strengths of the Fresno State PASC graduates. The responses are analyzed for themes in relation to the CAPEs and AAQEP standards.
Evidence (or plans) regarding trustworthiness We let completers know their responses are anonymous and only utilized for program improvement to help with trustworthiness. 
Evidence (or plans) regarding fairness In order to get a true representation of superintendent voices across the central valley, we would like to send this questionnaire or an adapted version of this questionnaire via email too for those who were not in attendance to the President’s Commission of Teacher Education. In addition, we want to also share this questionnaire in the P12 Superintendent Advisory Council as well. 

CCTC Completer Survey

Measure: CCTC Completer Survey
Evidence regarding validity The survey was designed to align with the California Professional Standards for Education Leaders (CPSEL), which are aligned with the California Administrative Performance Expectations (CAPEs). All survey items have been validated by the administrative bodies to determine their validity and are standardized across the larger systems
Evidence regarding reliability All survey items are assessed annually by the CTC to determine their reliability based on the responses received from program completers.
Evidence regarding Fairness/Trustworthiness

Because this instrument is administered statewide to candidates of all programs (CSU, University of California, private institutions, and local education agencies), we believe in its fairness as an instrument. Additionally, we believe the data the instrument yields are trustworthy because the survey is administered by the CTC, the statewide accrediting agency.

Fresno State has initiated a college-wide data summit to consider the findings of this statewide survey and triangulate them with campus data, including the percentage of First Generation students, access to resources like scholarships, and culture and context of the cohorts in which prospective teachers are placed. Through this triangulation process, we are able to determine the alignment of the finding from the survey with our other measures, further assuring us of the survey’s trustworthiness as an instrument. In the process, we are also able to inform the impact on program changes on our own students with respect to the unique diversity of culture and needs in the Central Valley.  

P12 PASC Program AAQEP Candidate Self-Assessment

Quantitative Data Measure: P12 PASC Program AAQEP Candidate Self-Assessment Descriptives Data for Standard 1 by Semester 
Description of Measure P12 PASC Program AAQEP Candidate Self-Assessment. The purpose of this self-assessment is to have students reflect on their perceptions of personal growth in the CAPEs and AAQEP standards over time in the PASC program. The information from this self-assessment is utilized to inform both areas of strength and opportunities for growth for students, as well as inform programmatic and instructional decision-making for ongoing continuous improvement. Each semester students are provided the opportunity to grow professionally as an aspiring equity driven school leader. Based on overall program coursework, activities, readings, and discussions, as well as fieldwork experiences through the CalAPA and other supervised activities with your field mentor supervision. This assessment is designed to help students reflect on a wide range of knowledge and experiences essential to their preparation. This self-assessment is divided into three sections: 1) AAQEP Program Standards Assessment, 2) CTC CAPEs Self-Assessment , and 3) Written Narrative Reflective Response.
Evidence (or plans) regarding validity Measure items are aligned with CTC and AAQEP standards. We informally tested the content with a pilot group of candidates who were finishing their final semester in the program to ensure understanding of the items. We can conduct a more formal pilot study of the items with a random group of candidates to ensure content clarity and understanding of what the items are asking. 
Evidence (or plans) regarding reliability We utilize a controlled testing setting where all students are required to complete the assessment within a course structure during course time. Also, as more candidates respond to the survey and the sample size increases, we can consider running some basic statistical analysis on the items. 
Evidence (or plans) regarding fairness Ensuring all candidates who have completed have an opportunity to share reflections and other needs every semester consistently. 

Reading/Literacy Specialist Credential

LEE 213 Inquiry Project

Key Data Measure: LEE 213 Inquiry Project
Description of Measure In LEE 213-Teaching the Language Arts K-12, students complete an inquiry project that has three main components. First, students select a topic of inquiry driven by their professional experiences teaching language arts and write a research paper describing the different theoretical perspectives and respective instructional implications. Second, students use the research examined to develop and implement instructional lessons in the classroom setting. Third, students develop a presentation to share with colleagues that presents the theory of their report, the practical applications from their lessons, and a critical reflection on the experiences.
Process to Ensure Validity Assignment aligns with student learning outcomes and CTC standards (see Reading and Literacy Standards course Matrix)
Process to Ensure Reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the assignment requirements and rubric ahead of time. 

LEE 224 Case Study Report

Key Data Measure: LEE 224 Case Study Report
Description of Measure In LEE 224-Assessing & Developing Reading Abilities, students administer a variety of literacy assessments to an individual struggling reader in K-12, analyze the assessment results, and use the results to develop an individualized instructional plan. The students prepare a case study report that details the assessment tools and results, provides an analysis of the results, and provides instructional recommendations.
Process to Ensure Validity Assignment aligns with student learning outcomes and CTC standards (see Reading and Literacy Standards course Matrix)
Process to Ensure Reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the assignment requirements and rubric ahead of time. 

LEE 244 Literature Review

Key Data Measure: LEE 244 Literature Review
Description of Measure In LEE 244-Research for Reading Professionals, students review research from the emergent reading, comprehension, and English Learner fields of literacy and construct a Wiki page. On this wiki page, students provide summaries of the research reviewed, including context, methods, and implications as well as a synthesis comparing the various theoretical perspectives that were examined.
Process to Ensure Validity Assignment aligns with student learning outcomes and CTC standards (see Reading and Literacy Standards course Matrix)
Process to Ensure Reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the assignment requirements and rubric ahead of time. 

LEE 254 Program Evaluation Report

Key Data Measure: LEE 254 Program Evaluation Report
Description of Measure In LEE 254-Supervised Field Experiences in Reading, students use evaluation tools and school data to construct an evaluation report of their school site’s literacy program. In these reports, students provide analysis of data regarding school instructional procedures and curriculum materials, the strengths and weaknesses of these elements, and conclusions regarding program enhancement and professional development.
Process to Ensure Validity Assignment aligns with student learning outcomes and CTC standards (see Reading and Literacy Standards course Matrix)
Process to Ensure Reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the assignment requirements and rubric ahead of time. 

LEE 254 Coaching Presentation

Key Data Measure: LEE 254 Coaching Presentation
Description of Measure In LEE 254-Supervised Field Experiences in Reading, students collaborate with a colleague in 3 peer-coaching cycles, consisting of pre-consultation, observation/modeling, and debriefing consultation. The students prepare presentations for two of the cycles. Presentations include lessons learned about the coaching process, critical reflective insights about professional growth, and plans for future goals. 
Process to Ensure Validity Assignment aligns with student learning outcomes and CTC standards (see Reading and Literacy Standards course Matrix)
Process to Ensure Reliability Assignment and rubric were created by program faculty. All program faculty who teach this course use this assignment and rubric.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the assignment requirements and rubric ahead of time. 

LEE 230 Discussion Posts

Key Data Measure: LEE 230 Discussion Posts
Description of Measure The LEE 230 Supervised Teaching in Reading/Language Arts discussion assignments highlights students’ ability to identify and reflect on the ideological/philosophical underpinnings of teaching reading, to assess their own stances, and to critically reflect on the implications of that assessment.
Process to Ensure Validity This assignment invites candidates to make connections between course readings and activities related to literacy instruction and their current beliefs and instructional practices. Because the focus of the course is on supervising reading/language arts instruction, the focus of this assignment is valid.
Process to Ensure Reliability The fact that multiple candidates demonstrated similar understandings and were able to reflect on the content in thoughtful and meaningful ways indicates the discussion posts are reliable measures for evaluating their understanding of the focal content.
Process to Ensure Fairness All candidates received full credit for the assignment, as long as they demonstrated understanding of the content and made an effort to connect the content to their instruction. These guidelines were provided in the discussion prompt.

LEE 215 Language Issues in Reading Course Grades

Key Data Measure: LEE 215 Language Issues in Reading Course Grades
Description of Measure The goals of this course are to prepare students to be knowledgeable about literacy development and instructional practices through the study of theoretical perspectives and scientific research on literacy processes and language development; and to prepare students with the capacity to plan, implement, evaluate, and modify literacy instruction to meet the needs of diverse struggling readers and English Language Learners.
Key assignments in the course include reflection papers on course readings, reflections on key course content, and a final research paper on a topic relevant to the course.
Process to Ensure Validity Course grades are calculated based on students’ successful completion of all assignments. Because the assignments are specifically linked to course content, the grades serve as a valid measure of mastery of the course content.
Process to Ensure Reliability In order to establish reliability, as a program faculty, we will engage in joint scoring of student work using the established rubric to ensure that we are all in agreement on how the assignments should be scored. Where disagreements arise, we will work to come to a common understanding. We will repeat this process on (an annual, biannual, regular) cycle to ensure the ongoing reliability of the measure.
Process to Ensure Fairness To ensure fairness, we will analyze assignments instructions to make sure expectations are clear. One way to engage in this process is to have candidates articulate their understanding of what the assignment asks. Where discrepancies exist between what we intend candidates to do and what they understand, we will revise instructions to ensure they are clear to all. We will also be sure that clear details are provided about how the assignment will be assessed, including specific rubrics and, whenever possible, samples of previous students' successful work.

Reading/Literacy Program Comprehensive Exam

Key Data Measure: Reading/Literacy Program Comprehensive Exam
Description of Measure The comprehensive exam is one culminating experience option for students in the Reading/Literacy MA program. There is a bank of ten questions addressing key information in five of the required courses (LEE 213, LEE 215, LEE 224, LEE 244, and LEE 278). Students will randomly receive five questions on the exam (one question from each course), and then select three to write a response. 
Process to Ensure Validity Exam aligns with student learning outcomes
Process to Ensure Reliability Exam questions and rubric were created by program faculty.
Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this exam in the same way.
Process to Ensure Fairness/ Trustworthiness All students have access to the exam requirements, question, and rubric ahead of time. 

Reading, Language, Literacy Specialist Credential Completer Survey

Key Data Measure: Reading, Language, Literacy Specialist Credential Completer Survey
Description of Measure The Reading, Language, Literacy Specialist Credential Completer Survey was created and piloted in Spring 2021 as a way to gauge completers’ perceptions of the program. Included in the survey are five likert-response items focused on how well the program prepared candidates: with knowledge of varied instructional approaches in the development of oral language, reading, and writing; to understand and use a variety of assessment techniques and instruments; to understand and use a variety of instructional techniques for ELL students; to understand the means of interacting with other professionals, parents, and community members concerning reading instruction; and to effectively meet the reading/language arts needs of their school.
Process to Ensure Validity Because each of the areas addressed represented a focal area of the program, we believe this is a valid measure to evaluate program effectiveness from the perspective of candidates at the point of completion.
Additionally, responders were given the opportunity to provide additional information if they chose to do so.
Process to Ensure Reliability This survey was administered as a pilot. As a program faculty, we will evaluate responses to look for overall alignment as a way to ensure it is a reliable measure.
Process to Ensure Fairness The survey was sent to all candidates upon their completion of the program, providing all with the same opportunity to respond. Additionally, responses were collected anonymously, giving completers the opportunity to share their honest opinions without fear of retribution.

Reading, Language, Literacy Specialist Credential Year+ Completer Pilot Survey

Key Data Measure: Reading, Language, Literacy Specialist Credential Year+ Completer Pilot Survey
Description of Measure The Reading, Language, Literacy Specialist Credential Year + Completer Survey was created and piloted in Spring 2021 as a way to gauge completers’ perceptions of the program one to five years after they finished. Included in the survey are items to collect background and demographic information on responders, including type of employment, along with six likert-style items, each of which was aligned with an AAQEP Standard 2 aspect. For each item, respondents were also given the opportunity to provide additional comments.
Process to Ensure Validity Because each of the items represented a focal area of the program in line with AAQEP Standard 2, we believe this is a valid measure to evaluate program effectiveness from the perspective of candidates at the point of completion.
Additionally, responders were given the opportunity to provide additional information if they chose to do so.
Moving forward, the program intends to hold focus group discussions with completers as a way to further validate the information provided in the survey responses.
Process to Ensure Reliability This survey was administered as a pilot. As a program faculty, we will evaluate responses to look for overall alignment as a way to ensure it is a reliable measure.
Process to Ensure Fairness The survey was sent to all individuals who completed the program between 2015 and 2020, providing all with the same opportunity to respond. Additionally, responses were collected anonymously, giving completers the opportunity to share their honest opinions without fear of retribution

School Counseling

COUN 201 (Multicultural Counseling) Reflective Assignments

Quantitative Data Measure: Reflective assignments in COUN 201 that require research-based information to understand students’ experiences with intersectionality.
Description of Measure In order to prepare culturally sensitive and responsive candidates, students are required to understand their own social experience in our societal context. Students assess the interaction of race, ethnicity, class, gender identity and expression, sexual identity, language acquisition, disability, learning style among other variables that impact behaviors and roles. 
Process to Ensure Validity For these assignments, students were expected to evaluate their own identity as well as the intersectionality of multiple factors affecting their own experience and learning. Additionally, students explored the impact of privilege and oppression on their own development and future practice. Since the assignment’s expectations are in line with enhancing students’ cultural competence, it is a valid data source for std. 1 C
Process to Ensure Reliability To assess the reliability of this assignment, we analyzed students’ assignments and found that students scored above satisfactory in all areas with scores higher than 12 points total (M=13.39). Overtime, we need to continue to analyze the data to ensure the assignment’s reliability. Please refer to the following link to access full dataset: COUN201Co.pdf
Process to Ensure Fairness/ Trustworthiness We believe this assignment demonstrates students’ understanding about intersectionality and cultural competence. There is also a clear rubric that assesses students’ cultural competence based on this assignment, contributing to the assignment’s fairness. Please see the following link for rubric: Rubric COUN201

COUN 201 Research Project Scores

Quantitative Data Measure: Research based projects in COUN 201 to analyze and understand the experience of marginalized groups and how experiences are shaped by environmental forces.
Description of Measure Through engaging in this assignment, students were expected to explore and understand the experience of marginalized groups to understand their lived experiences. Additionally, students researched the impact of systematic forces on this group as well as ways to enhance the life of these marginalized groups.
Process to Ensure Validity In order to prepare culturally sensitive and responsive candidates, students are required to increase their awareness about the struggles and challenges of socially marginalized groups. Since this assignment encourages students to explore, understand, research, and articulate the lived experiences of people belonging to marginzalized groups, it is a valid source for analyzing std. 1 c
Process to Ensure Reliability

Students were evaluated in their ability to follow instructions, write their ideas clearly, organize material, present content effectively, and engage in self-work. Students were required to incorporate research based resources to improve their understanding of the experience of marginalized groups and to learn about interventions that could improve the lived experiences of these groups. Providing students these clear directions supported with ensuring this assignment’s reliability in supporting students to engage in the specific requirements of this assignment. To analyze the effectiveness of this assignment, we examined the following dataset: COUN201Co.pdf 

Overall, students scored above satisfactory in all areas with scores higher than 12 points total (M= 13.32). It will be important to keep analyzing students’ responses to this assignment to assess its reliability overtime in expanding students’ cultural competence. 

Process to Ensure Fairness/ Trustworthiness

We strive to ensure that the assignment is fair for all students by using the following rubric: Rubric COUN201

Through engaging students in this assignment, our hope was to evaluate students’ understanding of the impact of external and internal forces on marginalized groups, which will be important for strengthening all candidates’ ability to provide counseling to a diverse group of K-12 students, hence making this assignment a fair choice.

COUN 206 (Counseling through the lifespan) Midterm Exam

Quantitative Data Measure: Mid-Term Exam in COUN 206. Please refer to the following link for syllabus: COUN 206 Syllabus for AAQEP Standard 1b 

Description of Measure The Midterm exam consists of a clinical vignette component. The clinical vignette requires students to respond to one clinical vignette in an essay format.  Students are expected to demonstrate competency with the relevant material taught to that point in the course. Specifically, students need to be able to connect developmental theory to a case example as well as identify normative and non-normative developmental factors that impact the individual(s) behavior. Students are also asked to discuss how developmental models across lifespan development explain the behavior seen in the vignette.
Process to Ensure Validity As indicated in COUN 206 syllabus, the purpose of this course is to provide an overview of various developmental concepts and principles as they relate to the practice of counseling. This course explores topics related to different aspects of development (i.e. social, emotional, moral, cognitive, physical, and other forces of development). This course also covers the elements of the life cycle process and supports students to understand how individual, familial, and social developmental forces interact to create both health and dysfunction. Since this exam assesses students’ ability to apply learning and developmental theories to meet the developmental needs of all individuals including K-12 students, it is a valid measure.
Process to Ensure Reliability Requirements to pass in the mid-term exam in COUN 206 stays relatively consistent over time since they are aligned with the CCTC program standards and CACREP standards given that COUN 206 is a core-counseling class in CACREP accredited specializations. In addition, the courses are staffed by the same faculty, on the whole to ensure reliability in content that is covered and assessed.
Process to Ensure Fairness/ Trustworthiness To examine our progress in preparing candidates to learn and apply theory--specifically, learning theory including social, emotional, and academic dimensions-- we use a midterm exam in COUN 206. Since the exam is same for all students, it is fair. 

COUN 241 Assignment: Research Paper and School Counselor Interview

Quantitative Data Measure: COUN 241 (Seminar in Organization of Counseling Services) Assignment titled “Research Paper and School Counselor Interview
Description of Measure “Research Paper and School Counselor Interview” assignment provides students an opportunity to write and submit a research paper on a topic related to school counseling.
Process to Ensure Validity This paper directly supports students pursuing their Masters in School Counseling with PPS credential to actively read and reflect upon the research-based, peer-reviewed articles that can help them to gain pedagogical content. In addition, students are required to interview a professional school counselor on the topic of their interest in relation to school counseling and further reflect upon how they can apply the knowledge gained from this interview. Therefore, this paper is helpful for students’ professional development and directly related to the course content and std. 1 objective of the pedagogical, and/or professional knowledge relevant to the credential/degree sought by students, hence it a valid assessment for std. 1 A.
Process to Ensure Reliability

Faculty are able to ensure the assignment’s reliability across different semesters by providing clear and consistent directions:

a. Papers will include two parts: A school counselor interview and literature review. Students are required to critically analyze, synthesize, and evaluate research articles on the topic of their interest. Students should be able to suggest their own conclusions and implications for school counseling based on an interview with school counselor and literature review.
b. This paper is intended to be helpful for students’ professional development and will be directly related to the course content.
c. The paper will be a minimum of 5 pages of double-spaced typed text with APA style and formatting. Students must cite at least 3 articles published in referred journals.

Please refer to the following link for a sample of students-work (N = 10):

COUN 241 Sample work for AAQEP Std 1a (Google Doc)

Process to Ensure Fairness/ Trustworthiness

To ensure fairness, faculty provide a rubric along with specific directions for more detailed directions and expectations. Lit Review Rubric 2020.pdf

In future, within the assignment description, we plan to add that students must use the knowledge gained from reading peer-reviewed journal articles and engaging in dialogues with professional school counselors to suggest two evidence-based practices that can promote a positive change within school settings – both at individual and systemic levels. We will also make a comprehensive rubric that focus on applicability of pedagogical content. 

COUN 242 Final Grades

Quantitative Data Measure:COUN 242 Final Grades 2018-2020
Description of Measure In COUN 242: Parent Education, Pupil Advocacy and Consulting, a course that students enroll in toward the end of their program, their final presentation consists of a collaborative project. The students are supposed to develop a comprehensive program/initiative that will tackle an issue in a K-12 school. 
Process to Ensure Validity The assignment aligns with the major learning objectives of COUN 242 to advance students’ understanding about international and global perspectives in counseling that can influence consultation processes within K-12 settings.
Therefore, the assignment is a valid measure of students’ competency in standard 2 d on enhancing students’ global and international perspectives. 
Process to Ensure Reliability Requirements in courses stay relatively consistent over time since they are aligned with the CCTC program standards
Process to Ensure Fairness/ Trustworthiness Course grades are required at the end of the course. With few exceptions, all program completers must complete and receive a grade for courses taken. For each course, specific details are provided within course syllabi about requirements for course assignments and to earn a passing grade.

COUN 249 Assignment Lesson Plan

Quantitative Data Measure: Please refer to the following link on the template that students use to participate and submit the assignment lesson plan: Lesson Plan
Description of Measure

Students are required to create a 20-minute lesson plan based on the ASCA model and present it to the class (in PowerPoint) at the assigned dates. Objective: Develop, present, and evaluate a classroom lesson on school counseling core curriculum, including formative and summative assessments. The assignment includes the following 4 components:

a. Summary of data/information: Students gather data from their school site to support the need for their lesson. Data collection can be any of the following—i. available data from the school, district, and/or state; ii. Observe and gather information from the school Administration/faculty/staff to see what the needs of students are; ii. Or they may collect anonymous surveys from students/parents/teachers/counselors. 
b. Complete a plan using the Lesson Plan template from ASCA and submit to instructor.
c. Create a pre- and post- assessment for the lesson plan.
d. PowerPoint presentation should focus on the lesson and pre/post test.

Process to Ensure Validity The assignment is in direct line with American School Counseling Association’s focus on engaging in data-driven practices through the lesson plan. The content of assignment demonstrates validity in terms of assessing std. 1e objective of creating positive learning and work environments. 
Process to Ensure Reliability We engaged in qualitative analysis of lesson plan assignments of randomly selected students to demonstrate the trustworthiness of the assignment. Our goal was to see the students provide materials and ideas that would engage the audience, are relevant to the skills/age group, and are collaborative in nature to ensure a positive learning and working environment. Please refer to the following link to access the complete dataset on lesson plans.
Process to Ensure Fairness/ Trustworthiness We used COUN 249 assignment lesson plan because it complemented the case study (individual intervention) through engaging in systemic level efforts to promote positive change for a group of students or staff (group intervention).  The lesson plan is a way for students to brainstorm how to create a curriculum that teaches and engages the audience (to create a positive learning environment) and to use the ASCA Mindsets and Behavior Standards relevant to a positive learning and working environment. Since the assignment is same for all students, it is fair. Also, since the assignment is in line with ASCA national model, it will help students prepare for theur future school counseling roles. 

COUN 249 Case Study Assignment

Key Data Measure: COUN 249 Case Study Presentation
Description of Measure Students complete a Case Study report on a K12 student-client who they’ve been working with at their internship site. Apart from presenting about their work with a given student-client, students are also required to submit a written assignment based on the following format that explains the elements of this data source: Case Study
Process to Ensure Validity

Assignment aligns with student learning outcomes and CTC standards 23, 26, and 29:

Standard 23: Advocacy
Candidates demonstrate skills and attitudes essential for advocating for the learning and academic success of all pupils. Candidates are able to identify institutional, systemic, interpersonal and intrapersonal barriers to learning, and are able to plan and implement strategies to eliminate those barriers and effectively support positive learning and achievement outcomes for all pupils.

Standard 26: Group Counseling and Facilitation
Candidates understand group dynamics and possess skill in group work including counseling, psychoeducational task and peer helping groups and facilitation of teams to enable pupils to overcome barriers and independence to learning.

Standard 29: Prevention, Education and Training
Candidates know and have skills in the planning, organizing and implementing educational programs designed to promote pupil learning and high academic achievement. Candidates also have knowledge in preventing problems that pose barriers to learning and achievement. Candidates develop knowledge and skills in working with school staff, parents, and family members to enable them to eliminate barriers to learning and achievement.

Please refer to the following link on complete thematic analysis on data that further demonstrates the assignment’s validity. 

Process to Ensure Reliability

Assignment has been constantly revised as per latest school counseling standards and faculty members own growing expertise in school counseling research and school counselor education. For instance, in last two years, faculty members have created a rubric to evaluate the assignment based on extensive discussions and then also added the component on ASCA national model mindsets to further enrich this assignment and ensure its actual reliability in helping school counseling interns translate what they have learned to promote school students academic success, socio-emotional wellbeing, and career development. Specifically, in the case-study, students have to also explain how their case conceptualization and interventions align with American School Counseling Association’s National Model of School Counseling mindsets from ASCA (4th edition):

M 1. Every student can learn, and every student can succeed.

M 2. Every student should have access to and opportunity for a high-quality education.

M 3. Every student should graduate from high school prepared for postsecondary opportunities. M 4. Every student should have access to a school counseling program.

M 5. Effective school counseling is a collaborative process involving school counselors, students, families, teachers, administrators and education stakeholders.

M 6. School counselors are leaders in the school, district, state and nation.

M 7. Comprehensive school counseling programs promote and enhance student academic, career and social/emotional outcomes.

Please refer to the following link on complete thematic analysis on data that also includes links to students’ individual assignments that demonstrate the reliability of this assignment in ensuring all students’ active engagement in the process of case conceptualization, implementing interventions, and evaluating the effectiveness of their interventions in accord with ASCA and CCTC standards.  

Process to Ensure Fairness/ Trustworthiness All students present their case study in class discussions that enable peers to engage in case consultation and ensure that case study is actually used to resolve the real concerns of K-12 students. Doing so enhances the utility of this assignment in promoting K-12 students’ wellbeing and success. Since all students use the provided format to wrote case study, the assignment is fair and trustworthy for all. Stakeholders such as site supervisors are also aware of this assignment and support students to successfully pursue their case study, further ensuring support for students. Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.

Undergraduate GPA

Key Data Measure: Undergraduate GPA
Description of Measure M.S. in School Counseling Graduate program applicants must have a minimum GPA of 3.0 in the last 60 semester (90 quarter) units attempted and a minimum of 3.0 on any post baccalaureate work at Fresno State.
Graduate program applicants who do not meet this requirement but meet the University minimum GPA requirement for graduate admissions (2.5) may be considered for admission at the discretion of the program admission committee based on compelling evidence of academic and professional potential (i.e. recent academic performance, extensive experiential background) as described in the Statement of Purpose.
Process to Ensure Validity Undergraduate GPA is a commonly used metric in admission as an indicator of expected academic success of candidates in counselor education programs. 
Process to Ensure Reliability Undergraduate GPA is used by admission evaluators to assess the academic ability of candidates and is commonly used throughout the field. GPA requirements are established by the California Commission on Teacher Credentialing and by Executive Order of the California State University Chancellor.
Process to Ensure Fairness/ Trustworthiness All candidates have completed the same requirements. Potential students who are not admitted can file a petition for Special Consideration which is considered by a committee. The good faith of a provider of the data as well as the provider’s reputation assure that the data represents what it is supposed to represent and that there is no intent to misrepresent. Regional accreditation is required for all transcripts. We have also stated the clear expectation about GPA on our application to program website.

Counselor Disposition Assessment

Key Data Measure: Counselor Disposition Assessment
Description of Measure Students were assessed using the department’s 3-point likert-type Counselor Dispositional Assessment tool during internship (i.e., COUN 249). The likert scale descriptors are listed as evaluation guidelines for each of the dispositions. Included with each disposition are examples of behavioral indicators to help guide the evaluation process. Each indicator is offered as a suggested behavior, and not as a conclusive determining factor. Faculty are able to review data and draw inferences in order to assist with program evaluation methods. On a 3-point likert scale (1 = inadequate 2 = meet expectations 3 =exceeds expectations), supervisors evaluate students in six areas: (a) reflection, (b) critical thinking, (c) professional ethics, (d) valuing diversity, (e) collaboration, and (f) life-long learning.
Process to Ensure Validity The current Candidate Dispositions measure plays an important role in evaluating multiple areas of students’ personal and professional capacities in serving clients from diverse backgrounds. For example, this tool assesses candidates on the following areas: a) Demonstrates the ability to work creatively and collaboratively with colleagues, clients, families, and the community; (b) Values clients as full partners in the counseling/educational process; (c) Collaborates with community partners and agencies in all phases of intervention when possible; (d) Works well with others to develop opportunities for peer and student learning; and (e) Plans and collaborates to ensure that appropriate supports for smooth transitions are in place. Though the terminology used in this tool ensures content and face validity, as faculty members committed to constantly reviewing and revising our assessment tools, we again critically examined the tool in Spring 2021 using a Candidate Disposition Assessment form that shows faculty members active engagement in critiquing this tool based on their area of specialization.
As a result of this critique and consistent discussions about validity of this tool in assessing students’ dispositions as counselors-in-training, we have decided to replace this tool with other well-established valid and reliable measure of counselor dispositions that is explained in the article.
We are in the process of seeking authors permission to start using this tool starting Spring 2022. 
Process to Ensure Reliability Measuring student dispositions is an important aspect of program assessment and, when combined with systematic progress reviews, constitutes an essential step for continuous counselor education program improvement. The school counseling programs require assessment of student dispositions as a necessary component of the program and student evaluation. Specifically, dispositions are the core values, attitudes, behaviors, and beliefs needed to become an effective and competent professional. Because student dispositions are an integral aspect of counseling skills and suitability for field performance, the measurement of dispositions is a natural component of the school counseling program assessment planning. One way in which we ensure reliability of this assessment is by using the same tool to assess students’ dispositions during practicum class as observed by their COUN 208 instructor and to assess students’ dispositions during COUN 249 as assessed by their internship site-supervisor. 
Process to Ensure Fairness/ Trustworthiness Since school counselors personal development and professional dispositions play a major role on their work ethics and ability to advance a positive change by working with students at individual and systemic levels, seeking practicum and internship supervisors feedback for students using a candidates dispositions tool is a fair and trustworthy process. This process also ensures that students are assessed comprehensively in terms of not only their knowledge content but also other critical areas such as work ethics, cultural competence, ability to work collaboratively with others.  

Evaluation of Field Placement Student 

Key Data Measure:Evaluation of Field Placement Student (Google form)
Description of Measure As discussed in Standard 1 and Standard 2, students are supervised and evaluated during the 600 hour internship experience. School counselor complete the evaluation form for each student using the following evaluation Google form and providing open-feedback. The evaluations are submitted and the instructor carefully reviews each evaluation. If the supervisor identifies any concerns, or the student receives average ratings or lower, the instructor will contact the supervisor and the student to  discuss the evaluation.
Process to Ensure Validity Practicum and internship supervisor evaluations, which are carefully revised by program faculty and coordinator, provide an opportunity to assess students in skill development and program outcomes. The data from these experiential courses is collected and analyzed to not only evaluate current students but also to identify areas of improvement and innovation. Additionally, program faculty and coordinator reviews these evaluations for any potential concerns that need to be addressed accordingly. If there are any concerns, the program faculty consults with the site supervisor to develop any potential remediation plan. Site supervisors also attend the semesterly Field Placement Orientation to receive information on our program, expectations for field placement so they can identify opportunities for interns at their sites, as well share information on their sites. Since site supervisors and faculty consistently review and discuss students’ actual progress based on these surveys and expectations laid down by CACREP and ASCA National School Counseling model, this survey undergoes regular content validity check by different stakeholders, ensuring its validity and revising it with changing standards and expectations. 
Process to Ensure Reliability The reliability of the instrument comes from two principal sources, the number of responders and the extent that their responses concur with one other. Each year the data set yields the percent of respondents who gave specified answers to each item and includes reliability estimates in the form of confidence intervals based on the number of respondents and the concurrence or homogeneity of responses.
All our surveys until last year were in the form of hard-copies collected at the end of semester, which we could not access due to the pandemic. In response to the COVID-19 pandemic and to make the process easier for site-supervisors, we transferred the questions of this survey to a google form, which we started collecting from Spring 2020 onwards. 
In standard 1, we demonstrated internship supervisor evaluation data for each targeted year,  demonstrating a high quality performance and reliability of this measure in ensuring students professional growth as a result of their  internship experience. In standard 1 documentation, we have used different items of this survey to demonstrate candidates' strong knowledge in learning theories, cultural competence, and data literacy. On average the analysis of all these items showed our students receiving the rating of “very satisfactory” which aligns with overall feedback that our students also get from their K-12 clients, administration, site supervisors, and program faculty - demonstrating the reliability of these survey results. In future, we want to analyse the data collected over at least 3 academic years to calculate its reliability estimates. 
Process to Ensure Fairness/ Trustworthiness Faculty and site supervisors work closely to assess students’ performance throughout the program. These collaborative evaluation processes can support students’ plan to effectively work with students in multiple education settings. Given the close collaboration of site supervisors and school counseling faculty in evaluating students’ professional strengths and skill-set as counselors-in-training, the process of using this survey ensures fairness and trustworthiness in evaluating all students’ strengths and areas that need improvement. 

Clinical Review

Key Data Measure: Clinical Review
Description of Measure Counselor educators play an important role as gate-keepers to ensure that only competent counselors graduate given their critical influence on the lives of the clients they will serve. We use Clinical Review evaluation Google Form to engage in a systematic progress review. ​​Clinical Review evaluations are dichotomous with two results: Pass and Fail. Students were assessed on 10 areas: (a) attention to ethical and legal concerns, (b) cooperativeness of others, (c) awareness of own impact on others, (d) ability to deal with conflict, (e) openness to new ideas, (f) tolerates ambiguity, (g) willingness to accept and use feedback, (h) ability to accept personal responsibility, (i) ability to express feelings effectively and appropriately, and (j) initiative and motivation using a 5-point likert-type scale with 1 being lowest rating and 5 being the highest.
Process to Ensure Validity Practicum and internship course instructors as well as all faculty members who teach counseling courses to school counseling students can engage in Clinical Review to evaluate and discuss any concerns that students might demonstrate. Since faculty consistently review and discuss students’ actual progress based on the clinical review survey in line with expectations laid down by CACREP, this survey undergoes regular content validity check by different stakeholders, ensuring its validity and revising it with changing standards and expectations. 
Process to Ensure Reliability The reliability of the instrument comes from two principal sources, the number of responders and the consistency of using this measure across all courses (especially practicum and internship courses). Each year, we hold clinical review committee meeting in which we discuss any student that scored low on clinical review assessment. If a student gets low score, we meet the student to discuss concerns and develop a remediation plan in collaboration with concerned faculty, program coordinator and department chair. This process ensures reliability of the clinical review assessment. Analysis of last three semesters has shown that all students passed clinical review. In future, we want to analyse the data collected over at least 3 academic years to further ensure its reliability. 
Process to Ensure Fairness/ Trustworthiness Faculty teaching internship and practicum courses play a critical role in assessing students’ performance and clinical skills and especially in ensuring that they have the personal and professional dispositions needed to serve future K-12 students as school counselors. Given the consistency of this measure across years and for all students from different counseling specializations, the process of using this survey ensures fairness and trustworthiness in evaluating all students’ potential as future school counselors. 

Site Supervisor Program Evaluation Survey

Key Data Measure: Site Supervisor Program Evaluation Survey
Description of Measure The Program Evaluation Feedback Google form was created and reviewed by the School Counseling faculty to seek consistent feedback from our most important stakeholders, which are site supervisors. 
Process to Ensure Validity We have not tested the instrument’s validity. However, if any aspect of the survey is scored lower, the faculty discusses that with site-supervisor to gain deeper insights into site supervisors’ concerns and engage in the process of improving the program based on their feedback. 
Process to Ensure Reliability Engaging in open-dialogue about the scores that program receives based on site supervisors observations of students, helps to ensure the instrument’s reliability. However, we have not tested the reliability of this instrument empirically. In future, we need to make greater efforts to analyze the data we collect from this instrument to ensure its reliability and use it for continuous improvement. 
Process to Ensure Fairness/ Trustworthiness Faculty consistently seeks site supervisors feedback on students’ personal and professional dispositions as well as the quality of school counseling program in training future school counselors. Since site supervisors are most abreast with the latest trends and needs of K-12 settings, the process of seeking their feedback for program is fair and ensures trustworthiness for the program. 

Advancement to Candidacy

Key Data Measure: Advancement to Candidacy
Description of Measure The Petition of Advancement to Candidacy form serves as a guideline for progress toward and completion of the degree. Students must complete this form by the sixth week of the semester prior to the term in which a student registers for the culminating experience, and/or applies for the graduate degree to be granted. The form can be accessed on this website
Process to Ensure Validity The courses mentioned in Advancement to Candidacy form are in line with the course catalog, which has been approved by the university. Students complete the Advancement to Candidacy form by filling in the courses they have taken and will take by the end of the program. Each completed form is submitted to the office of Graduate Studies for approval. Signatures are obtained from the student, the program director and the student’s advisor. Once the Advancement to Candidacy is completed, our program is under contract with the student that we will not change any course requirements during their time in the program. Since the form is in line with courses mentioned in course catalog and is signed at various department, school, and university levels to ensure students’ steady progress towards degree completion, and the content on Advancement to Candidacy form reflect the important courses that are needed for professional development of counselors-in-training, it is a valid instrument to support students’ progress. 
Process to Ensure Reliability All students referred to the key role of Advancement to Candidacy form during the internal audit process. Students explained that receiving signature from their advisor on Advancement to candidacy form ensured that they met the key requirements for degree completion. The Advancement to Candidacy form also ensures that our program keeps with the requirements of the Graduate studies and State of California. 
Process to Ensure Fairness/ Trustworthiness Program faculty work closely with students to support their successful completion of the program and transition into professional careers. Students have an assigned advisor that helps planning their degree plan for graduation within the five years after admission. Advisors complete an advising sheet with students at the beginning of their program. One key role of advisor is to sign students’ Advancement to Candidacy form, which is in alignment with the courses listed in advising form. This ensures that all students receive the same information about courses needed and receive a mid-program check-in to ensure their successful progress towards degree completion. 

Comprehensive Exam

Key Data Measure: Comprehensive Exam
Description of Measure A comprehensive examination is an assessment of the student’s ability to integrate the knowledge, show critical thinking and demonstrate mastery of school counseling. Our students have successfully passed the Comprehensive exam suggesting that they have demonstrated high levels of proficiency in the content, pedagogical, and professional knowledge necessary to function as a school counselor. 
Process to Ensure Validity Comprehensive exam questions are consistently developed, reviewed, and revised by the faculty within the program. Faculty hired to teach within the programs are considered experts in their specific field and all have relevant experience for the content of the courses which they are teaching. The questions developed are in line with counseling standards and well-established texts, thus demonstrating strong content validity. 
Process to Ensure Reliability We use a standard rubric to analyze students’ responses to comprehensive exam essay. This ensures reliability among the faculty evaluating the surveys. Moreover, two full-time school counseling faculty members evaluate students’ essays and grade them as per rubric. If one faculty member passes a student but other does not, the student’s essay is shared with third faculty member in the program who then evaluates the student’s essay independent and helps in making a reliable decision on passing or failing the student.
Process to Ensure Fairness/ Trustworthiness Every year, for a culminating experience, students in the MS in School Counseling may elect to take a Comprehensive Exam during the final stage of their program. For this standard, we are choosing students’ comprehensive exam essay as a data source because the essay requires them to analyze a student’s individual and systemic level concerns based on a vignette and further develop counseling goals and an ethically sound as well as multiculturally competent action plan based on their assessment(s). Hence, these essays can directly demonstrate students’ ability to engage in assessment and use the given data to inform their practice within K-12 educational settings. They also provide the program faculty a trustworthy means to guage students proficiency as counselors-in-training and serve the role as gatekeepers of counseling profession to ensure that only the students with strong knowledge and skill-set graduate and serve future clients. To ensure objectivity in evaluating essays and fairness for all students, the comprehensive exam coordinator removes any identifying information on students’ names or year in program from the responses. 

2021 Survey of Recent School Counseling Graduates

Quantitative Data Measure: 2021 Survey of Recent School Counseling Graduates
Description of Measure We use the exit survey to assess completer’s perceptions of various aspects of the preparation program including their field placement experience as well as future educational/career plans. 
Process to Ensure Validity Exit Survey serves as a valid measure of program completers’ perceptions of the school counselors’ preparation program because it asks questions directly aligned with the school counseling program’s learning objectives and CCTC standards of PPS credential. All program completers respond to items asking about their preparation of general pedagogical skills, such as their perception of how well the program prepared them to engage in individual and group counseling skills with specific group of students. In this way, the survey is a valid measure of completers’ perceptions of the program.
Process to Ensure Reliability Reliability about evaluation findings comes from two principal sources: the number of evaluation participants and the extent of their concurrence with each other. The evaluation findings become increasingly certain to the extent that the questions are answered by increasing numbers of program completers. We started using this survey during the academic year 2019-2020 limiting our capacity to assess the survey’s reliability. We plan to continue to use the survey every year and engage in analyzing survey’s reliability and then further revising the survey to make it more reliable. 
Process to Ensure Fairness/ Trustworthiness Given the content of the questions asked, we believe the survey has face validity and content validity. However, in future we plan to triangulate the findings of this survey through engaging in focus group sessions with completers and students near graduation and then analyzing the results of this survey in the context of these discussions. Doing so would help us to ensure this survey’s validity and fairness. 

School Nursing

Candidate GPA at Completion of Course/Program

Key Data Measure: GPA of assignment and course
Description of Measure Earned grades of a course or assignment
Process to Ensure Validity Aligned to course objectives and CCTC guidelines of mandated course content for expected SN outcomes.
Aligned to course and assignment rubrics.
Process to Ensure Reliability Criteria for grade determination aligned to course rubrics used by all course instructors.
Course expectations and rubrics published in the syllabus.
Process to Ensure Trustworthiness Grade measures should align with scores from the preceptor skill checklists and other evaluation tools.
Process to Ensure Fairness All students have access to the grading rubric ahead of time and are based on criteria stated in the syllabus.  Any exceptions that differ from syllabus are offered to all students.  Students can access grades over the semester and can access instructor if questions occur.

Preceptor Checklist of Skills & Competencies and Evaluation of Student Performance

Key Data Measure: Preceptor Checklist of Skills (Elementary: Form 105/205 and Secondary: Form 106/206) 
Description of Measure Preceptor checklist of skills and dispositions determined by direct supervision of candidate in the field used by preceptors to evaluate candidates in their field work knowledge at two points.  Candidates are rated on a 5-point Likert scale from 5 “Excellent” to 1 “Poor” on 40 items clustered into three areas: Providing Health and Wellness Services (Primary Intervention), Providing Direct Client Care Services (Secondary and Tertiary Intervention), and Professional Management Skills.
Process to Ensure Validity Criteria scored is clearly defined and stated in multiple areas.  Criteria aligns with CCTC requirements of necessary skills for school nurse outcomes.
Process to Ensure Reliability Preceptors rate candidates and sign off skills during direct supervision of the candidate.  Any discrepancies are discussed and reviewed with instructors of course
Process to Ensure Trustworthiness Coordinators individually judge candidate applications and meet to discuss discrepancies.
Preceptor scoring should align with GPA/Instructor assessments.
Process to Ensure Fairness Preceptor skills check off evaluations are posted in the syllabus and available for student review.
Students receive feedback and review. Instructors review preceptor skill checkoff for relevance and scoring.

Employer/Supervisor Survey of Program Effectiveness

Key Data Measure: Employer/Supervisor Survey of Program Effectiveness
Description of Measure The Employer/Supervisor Survey of Program Effectiveness is administered to District Health Services Directors or Administrators who hire/supervise program candidates one year after candidates have graduated. Candidates are rated on a 4-point Likert scale in 10 areas that align with the CCTC School Nursing Program standards.
Process to Ensure Validity Criteria based on SNSC program objectives and goals.
Process to Ensure Reliability  Coordinators review scoring and comments and look for patterns of results to improve the program.
Process to Ensure Trustworthiness Coordinators individually review employee/supervisor evaluations and meet to discuss discrepancies.
Process to Ensure Fairness Feedback from employers/supervisors solicited and discussed at SNSC Advisory meetings to determine needed program changes.

Program Start and Completion of Knowledge Based Questionnaire

Key Data Measure: Pre-Knowledge-Based Questionnaire & Post-Knowledge-Based Questionnaire
Description of Measure The Knowledge-Based student self-rated questionnaire is administered at the beginning and end of the program as a way to measure candidates’ perceived knowledge at program entrance and after program completion. Items on the survey are clustered into 22 core areas: Coordinated School Health Programs, Standards of School Nursing Practice, Nursing Models/Theories, Families at Risk, Neurological Screening Tests, Legal/Ethical Issues, and Child Abuse. 
Results provide the School Nurse Services Credential (SNSC) Program faculty an idea of the candidates’ knowledge base at entry and areas needing additional assistance to meet the SNSC program competencies. Following the questionnaire, students develop individual goals and objectives based on their learning needs.
Process to Ensure Validity The items included on the questionnaire include key items to be learned in the School Nursing Credential program and are aligned with the CCCTC Program Standards.
Process to Ensure Reliability  Results from the questionnaire, particularly from the completers, are triangulated with student performance on course assignments and preceptor evaluations. Because we find alignment between these different measures, we believe the questionnaire is a reliable instrument.
Process to Ensure Fairness All candidates have the same access to the questionnaire. Their results are not counted against them in any way. Rather, the beginning-of-program results provide a way for candidates and program faculty to focus the learning experiences. At the end of the program, the questionnaire provides an opportunity for candidates to reflect on their growth and for faculty to learn candidates’ perceptions of their learning.

CCTC Survey of Program Effectiveness

Key Data Measure: CCTC Completer Survey
Description of Measure Beginning in 2018, the CTC has administered a survey that is completed by completers of credential programs other than pre-service teaching. The survey, administered between September 1 and December 31, examines the effectiveness of individual educator preparation programs approved to operate in California. In 2019-2020, 95.6% of  the 225 individuals who completed a School Nursing Credential program in California responded to the survey. Of those who responded, 96.7% completed their credential at a California State University.
Process to Ensure Validity This instrument has been used to evaluate the effectiveness of School Nursing Credential Programs across the State of California. Items are specifically tailored to completers of School Nursing Credential Programs and are clustered into five general areas: Background Information, Admission Process, Program Faculty and Coursework, Field Experience, and Assessment and Feedback/Program Guidance and Support.
Process to Ensure Reliability  The reliability of the instrument comes from two principal sources, the number of responders and the extent that their responses concur with one other. The instrument becomes increasingly reliable to the extent that the items are answered in similar ways by increasing numbers of completers of School Nursing Credential Programs. Each year the data set yields the percent of respondents who gave specified answers to each item and includes reliability estimates in the form of confidence intervals based on the number of respondents and the concurrence or homogeneity of responses.
Process to Ensure Fairness Data were not constructed with bias. The existence of this statewide instrument allows us to examine trends in perceptions of individuals who were prepared as School Nurses across the system and compare them with perceptions of individuals who completed our program. Because the survey is administered statewide, we believe it is a fair and trustworthy instrument.

Professional Dispositions

Key Data Measure: Professional Dispositions
Description of Measure This instrument is used by preceptors to assess the candidate’s ability to competently carry out required skills and competencies in clinical practice.  Items address behavior needed for successful practice, including ability to engage in reflection, critical thinking, ethical judgements, valuing diversity, collaboration, and lifelong learning. Candidates are evaluated in each area on a 5 point Likert scale as observed by the preceptor in clinical practice.
Process to Ensure Validity The dispositions listed on the rubric align with disposition critical for a successful school nurse.
Moving forward, we will continue to review the instrument to ensure that the disposition rubric definitions are current and clearly stated.
Process to Ensure Reliability  Moving forward, we will review the assessment forms with preceptors to ensure that all are in agreement with the rubric and how candidates are to  be scored.
Process to Ensure Fairness Although this is an observational tool, the data is compared to additional assessment tools by which a candidate is scored for a composite picture of a candidate.  Data can be compared between different preceptors for discrepancies to ensure fairness.

Key Course Assignments

Key Data Measure: Key Course Assignments

Across Standards 1 and 2, we utilized grades and candidate performance on key assignments. While we believe these assignments do reflect valid, reliable, and fair practices, moving forward, as a faculty, we will engage in inquiry to ensure this is indeed the case.

Mean Course Grades: The average GPA candidates achieved on a given assignment.

N186: Special Education Cross Cultural Case Study: The purpose of the assignment is to gain a broader perspective and more in-depth understanding of the Student Study Team and Individual Education Plan assessment team function, while exploring various cultural health beliefs and practices and how it affects the students ability to learn.

N136: Health Assessment:  A review of body systems, conducting health assessments, and how impairments can affect learning and function.

N184: Individual Health Care Plans: Purpose of the assignment is the development and implementation of an individualized health care plan by a school nurse of all the medical care needed to allow a student to attend school safely.  Used for the documentation of care/education of staff and monitoring of student’s health status and the safety of the school environment.

N184: Cultural Book Report: Books are assigned to enhance the cultural competence of school nurse candidates and to gain insight into the diversity of cultural practices and health care beliefs of the school community.

N185: Orientation/Staff Training Design Project: Purpose is to develop the school nurse manager role.  Assignment is to research, design and develop an orientation for health services staff taking into account federal, state, school practice, and law that impacts a school community.  As part of the orientation, the school nurse must design a program to implement and monitor training in health areas.

N186/N187: Leadership Role or Experienced Nurse Project: The purpose of these options is to give candidates additional choices for gaining clinical hours while encouraging them to stretch themselves outside their daily roles. 

N186/N187: Personal Learning Goals and Objectives: The candidate is able to personalize their learning experience by focusing on areas they are least comfortable in or have limited experience in order to have a well-rounded experience in the school.

Process to Ensure Validity In order to establish content validity, we will analyze the key assignments to ensure they focus on content that aligns with the intended outcome of the course. Additionally, we will review rubrics to ensure that they in fact measure the key content areas the assignments are intended to assess.
Process to Ensure Reliability  In order to establish reliability, as a program faculty, we will engage in joint scoring of student work using the established rubric to ensure that we are all in agreement on how the assignments should be scored. Where disagreements arise, we will work to come to a common understanding. We will repeat this process on (an annual, biannual, regular) cycle to ensure the ongoing reliability of the measure.
Process to Ensure Fairness To ensure fairness, we will analyze assignment instructions to make sure expectations are clear. One way to engage in this process is to have candidates articulate their understanding of what the assignment asks. Where discrepancies exist between what we intend candidates to do and what they understand, we will revise instructions to ensure they are clear to all. We will also be sure that clear details are provided about how the assignment will be assessed, including specific rubrics and, whenever possible, samples of previous students' successful work.