AAQEP Accreditation 2022
Appendix E: Evidence of Data Quality
School Counseling
| Quantitative Data Measure: Reflective assignments in COUN 201 that require research-based information to understand students’ experiences with intersectionality. | |
| Description of Measure | In order to prepare culturally sensitive and responsive candidates, students are required to understand their own social experience in our societal context. Students assess the interaction of race, ethnicity, class, gender identity and expression, sexual identity, language acquisition, disability, learning style among other variables that impact behaviors and roles. |
| Process to Ensure Validity | For these assignments, students were expected to evaluate their own identity as well as the intersectionality of multiple factors affecting their own experience and learning. Additionally, students explored the impact of privilege and oppression on their own development and future practice. Since the assignment’s expectations are in line with enhancing students’ cultural competence, it is a valid data source for std. 1 C. |
| Process to Ensure Reliability | To assess the reliability of this assignment, we analyzed students’ assignments and found that students scored above satisfactory in all areas with scores higher than 12 points total (M=13.39). Overtime, we need to continue to analyze the data to ensure the assignment’s reliability. Please refer to the following link to access full dataset: COUN201Co.pdf |
| Process to Ensure Fairness/ Trustworthiness | We believe this assignment demonstrates students’ understanding about intersectionality and cultural competence. There is also a clear rubric that assesses students’ cultural competence based on this assignment, contributing to the assignment’s fairness. Please see the following link for rubric: Rubric COUN201 |
| Quantitative Data Measure: Research based projects in COUN 201 to analyze and understand the experience of marginalized groups and how experiences are shaped by environmental forces. | |
| Description of Measure | Through engaging in this assignment, students were expected to explore and understand the experience of marginalized groups to understand their lived experiences. Additionally, students researched the impact of systematic forces on this group as well as ways to enhance the life of these marginalized groups. |
| Process to Ensure Validity | In order to prepare culturally sensitive and responsive candidates, students are required to increase their awareness about the struggles and challenges of socially marginalized groups. Since this assignment encourages students to explore, understand, research, and articulate the lived experiences of people belonging to marginzalized groups, it is a valid source for analyzing std. 1 c. |
| Process to Ensure Reliability |
Students were evaluated in their ability to follow instructions, write their ideas clearly, organize material, present content effectively, and engage in self-work. Students were required to incorporate research based resources to improve their understanding of the experience of marginalized groups and to learn about interventions that could improve the lived experiences of these groups. Providing students these clear directions supported with ensuring this assignment’s reliability in supporting students to engage in the specific requirements of this assignment. To analyze the effectiveness of this assignment, we examined the following dataset: COUN201Co.pdf Overall, students scored above satisfactory in all areas with scores higher than 12 points total (M= 13.32). It will be important to keep analyzing students’ responses to this assignment to assess its reliability overtime in expanding students’ cultural competence. |
| Process to Ensure Fairness/ Trustworthiness |
We strive to ensure that the assignment is fair for all students by using the following rubric: Rubric COUN201 Through engaging students in this assignment, our hope was to evaluate students’ understanding of the impact of external and internal forces on marginalized groups, which will be important for strengthening all candidates’ ability to provide counseling to a diverse group of K-12 students, hence making this assignment a fair choice. |
|
Quantitative Data Measure: Mid-Term Exam in COUN 206. Please refer to the following link for syllabus: COUN 206 Syllabus for AAQEP Standard 1b |
|
| Description of Measure | The Midterm exam consists of a clinical vignette component. The clinical vignette requires students to respond to one clinical vignette in an essay format. Students are expected to demonstrate competency with the relevant material taught to that point in the course. Specifically, students need to be able to connect developmental theory to a case example as well as identify normative and non-normative developmental factors that impact the individual(s) behavior. Students are also asked to discuss how developmental models across lifespan development explain the behavior seen in the vignette. |
| Process to Ensure Validity | As indicated in COUN 206 syllabus, the purpose of this course is to provide an overview of various developmental concepts and principles as they relate to the practice of counseling. This course explores topics related to different aspects of development (i.e. social, emotional, moral, cognitive, physical, and other forces of development). This course also covers the elements of the life cycle process and supports students to understand how individual, familial, and social developmental forces interact to create both health and dysfunction. Since this exam assesses students’ ability to apply learning and developmental theories to meet the developmental needs of all individuals including K-12 students, it is a valid measure. |
| Process to Ensure Reliability | Requirements to pass in the mid-term exam in COUN 206 stays relatively consistent over time since they are aligned with the CCTC program standards and CACREP standards given that COUN 206 is a core-counseling class in CACREP accredited specializations. In addition, the courses are staffed by the same faculty, on the whole to ensure reliability in content that is covered and assessed. |
| Process to Ensure Fairness/ Trustworthiness | To examine our progress in preparing candidates to learn and apply theory--specifically, learning theory including social, emotional, and academic dimensions-- we use a midterm exam in COUN 206. Since the exam is same for all students, it is fair. |
| Quantitative Data Measure: COUN 241 (Seminar in Organization of Counseling Services) Assignment titled “Research Paper and School Counselor Interview | |
| Description of Measure | “Research Paper and School Counselor Interview” assignment provides students an opportunity to write and submit a research paper on a topic related to school counseling. |
| Process to Ensure Validity | This paper directly supports students pursuing their Masters in School Counseling with PPS credential to actively read and reflect upon the research-based, peer-reviewed articles that can help them to gain pedagogical content. In addition, students are required to interview a professional school counselor on the topic of their interest in relation to school counseling and further reflect upon how they can apply the knowledge gained from this interview. Therefore, this paper is helpful for students’ professional development and directly related to the course content and std. 1 objective of the pedagogical, and/or professional knowledge relevant to the credential/degree sought by students, hence it a valid assessment for std. 1 A. |
| Process to Ensure Reliability |
Faculty are able to ensure the assignment’s reliability across different semesters by providing clear and consistent directions: a. Papers will include two parts: A school counselor interview and literature review.
Students are required to critically analyze, synthesize, and evaluate research articles
on the topic of their interest. Students should be able to suggest their own conclusions
and implications for school counseling based on an interview with school counselor
and literature review. Please refer to the following link for a sample of students-work (N = 10): |
| Process to Ensure Fairness/ Trustworthiness |
To ensure fairness, faculty provide a rubric along with specific directions for more detailed directions and expectations. Lit Review Rubric 2020.pdf In future, within the assignment description, we plan to add that students must use the knowledge gained from reading peer-reviewed journal articles and engaging in dialogues with professional school counselors to suggest two evidence-based practices that can promote a positive change within school settings – both at individual and systemic levels. We will also make a comprehensive rubric that focus on applicability of pedagogical content. |
| Quantitative Data Measure: COUN 242 Final Grades 2018-2020 | |
| Description of Measure | In COUN 242: Parent Education, Pupil Advocacy and Consulting, a course that students enroll in toward the end of their program, their final presentation consists of a collaborative project. The students are supposed to develop a comprehensive program/initiative that will tackle an issue in a K-12 school. |
| Process to Ensure Validity | The assignment aligns with the major learning objectives of COUN 242 to advance students’
understanding about international and global perspectives in counseling that can influence
consultation processes within K-12 settings. Therefore, the assignment is a valid measure of students’ competency in standard 2 d on enhancing students’ global and international perspectives. |
| Process to Ensure Reliability | Requirements in courses stay relatively consistent over time since they are aligned with the CCTC program standards |
| Process to Ensure Fairness/ Trustworthiness | Course grades are required at the end of the course. With few exceptions, all program completers must complete and receive a grade for courses taken. For each course, specific details are provided within course syllabi about requirements for course assignments and to earn a passing grade. |
| Quantitative Data Measure: Please refer to the following link on the template that students use to participate and submit the assignment lesson plan: Lesson Plan | |
| Description of Measure |
Students are required to create a 20-minute lesson plan based on the ASCA model and present it to the class (in PowerPoint) at the assigned dates. Objective: Develop, present, and evaluate a classroom lesson on school counseling core curriculum, including formative and summative assessments. The assignment includes the following 4 components: a. Summary of data/information: Students gather data from their school site to support
the need for their lesson. Data collection can be any of the following—i. available
data from the school, district, and/or state; ii. Observe and gather information from
the school Administration/faculty/staff to see what the needs of students are; ii.
Or they may collect anonymous surveys from students/parents/teachers/counselors. |
| Process to Ensure Validity | The assignment is in direct line with American School Counseling Association’s focus on engaging in data-driven practices through the lesson plan. The content of assignment demonstrates validity in terms of assessing std. 1e objective of creating positive learning and work environments. |
| Process to Ensure Reliability | We engaged in qualitative analysis of lesson plan assignments of randomly selected students to demonstrate the trustworthiness of the assignment. Our goal was to see the students provide materials and ideas that would engage the audience, are relevant to the skills/age group, and are collaborative in nature to ensure a positive learning and working environment. Please refer to the following link to access the complete dataset on lesson plans. |
| Process to Ensure Fairness/ Trustworthiness | We used COUN 249 assignment lesson plan because it complemented the case study (individual intervention) through engaging in systemic level efforts to promote positive change for a group of students or staff (group intervention). The lesson plan is a way for students to brainstorm how to create a curriculum that teaches and engages the audience (to create a positive learning environment) and to use the ASCA Mindsets and Behavior Standards relevant to a positive learning and working environment. Since the assignment is same for all students, it is fair. Also, since the assignment is in line with ASCA national model, it will help students prepare for theur future school counseling roles. |
| Key Data Measure: COUN 249 Case Study Presentation | |
| Description of Measure | Students complete a Case Study report on a K12 student-client who they’ve been working with at their internship site. Apart from presenting about their work with a given student-client, students are also required to submit a written assignment based on the following format that explains the elements of this data source: Case Study |
| Process to Ensure Validity |
Assignment aligns with student learning outcomes and CTC standards 23, 26, and 29: Standard 23: Advocacy Standard 26: Group Counseling and Facilitation Standard 29: Prevention, Education and Training Please refer to the following link on complete thematic analysis on data that further demonstrates the assignment’s validity. |
| Process to Ensure Reliability |
Assignment has been constantly revised as per latest school counseling standards and faculty members own growing expertise in school counseling research and school counselor education. For instance, in last two years, faculty members have created a rubric to evaluate the assignment based on extensive discussions and then also added the component on ASCA national model mindsets to further enrich this assignment and ensure its actual reliability in helping school counseling interns translate what they have learned to promote school students academic success, socio-emotional wellbeing, and career development. Specifically, in the case-study, students have to also explain how their case conceptualization and interventions align with American School Counseling Association’s National Model of School Counseling mindsets from ASCA (4th edition): M 1. Every student can learn, and every student can succeed. M 2. Every student should have access to and opportunity for a high-quality education. M 3. Every student should graduate from high school prepared for postsecondary opportunities. M 4. Every student should have access to a school counseling program. M 5. Effective school counseling is a collaborative process involving school counselors, students, families, teachers, administrators and education stakeholders. M 6. School counselors are leaders in the school, district, state and nation. M 7. Comprehensive school counseling programs promote and enhance student academic, career and social/emotional outcomes. Please refer to the following link on complete thematic analysis on data that also includes links to students’ individual assignments that demonstrate the reliability of this assignment in ensuring all students’ active engagement in the process of case conceptualization, implementing interventions, and evaluating the effectiveness of their interventions in accord with ASCA and CCTC standards. |
| Process to Ensure Fairness/ Trustworthiness | All students present their case study in class discussions that enable peers to engage in case consultation and ensure that case study is actually used to resolve the real concerns of K-12 students. Doing so enhances the utility of this assignment in promoting K-12 students’ wellbeing and success. Since all students use the provided format to wrote case study, the assignment is fair and trustworthy for all. Stakeholders such as site supervisors are also aware of this assignment and support students to successfully pursue their case study, further ensuring support for students. Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way. |
| Key Data Measure: Counselor Disposition Assessment | |
| Description of Measure | Students were assessed using the department’s 3-point likert-type Counselor Dispositional Assessment tool during internship (i.e., COUN 249). The likert scale descriptors are listed as evaluation guidelines for each of the dispositions. Included with each disposition are examples of behavioral indicators to help guide the evaluation process. Each indicator is offered as a suggested behavior, and not as a conclusive determining factor. Faculty are able to review data and draw inferences in order to assist with program evaluation methods. On a 3-point likert scale (1 = inadequate 2 = meet expectations 3 =exceeds expectations), supervisors evaluate students in six areas: (a) reflection, (b) critical thinking, (c) professional ethics, (d) valuing diversity, (e) collaboration, and (f) life-long learning. |
| Process to Ensure Validity | The current Candidate Dispositions measure plays an important role in evaluating multiple
areas of students’ personal and professional capacities in serving clients from diverse
backgrounds. For example, this tool assesses candidates on the following areas: a)
Demonstrates the ability to work creatively and collaboratively with colleagues, clients,
families, and the community; (b) Values clients as full partners in the counseling/educational
process; (c) Collaborates with community partners and agencies in all phases of intervention
when possible; (d) Works well with others to develop opportunities for peer and student
learning; and (e) Plans and collaborates to ensure that appropriate supports for smooth
transitions are in place. Though the terminology used in this tool ensures content
and face validity, as faculty members committed to constantly reviewing and revising
our assessment tools, we again critically examined the tool in Spring 2021 using a
Candidate Disposition Assessment form that shows faculty members active engagement in critiquing this tool based on their
area of specialization. As a result of this critique and consistent discussions about validity of this tool in assessing students’ dispositions as counselors-in-training, we have decided to replace this tool with other well-established valid and reliable measure of counselor dispositions that is explained in the article. We are in the process of seeking authors permission to start using this tool starting Spring 2022. |
| Process to Ensure Reliability | Measuring student dispositions is an important aspect of program assessment and, when combined with systematic progress reviews, constitutes an essential step for continuous counselor education program improvement. The school counseling programs require assessment of student dispositions as a necessary component of the program and student evaluation. Specifically, dispositions are the core values, attitudes, behaviors, and beliefs needed to become an effective and competent professional. Because student dispositions are an integral aspect of counseling skills and suitability for field performance, the measurement of dispositions is a natural component of the school counseling program assessment planning. One way in which we ensure reliability of this assessment is by using the same tool to assess students’ dispositions during practicum class as observed by their COUN 208 instructor and to assess students’ dispositions during COUN 249 as assessed by their internship site-supervisor. |
| Process to Ensure Fairness/ Trustworthiness | Since school counselors personal development and professional dispositions play a major role on their work ethics and ability to advance a positive change by working with students at individual and systemic levels, seeking practicum and internship supervisors feedback for students using a candidates dispositions tool is a fair and trustworthy process. This process also ensures that students are assessed comprehensively in terms of not only their knowledge content but also other critical areas such as work ethics, cultural competence, ability to work collaboratively with others. |
| Key Data Measure: Evaluation of Field Placement Student (Google form) | |
| Description of Measure | As discussed in Standard 1 and Standard 2, students are supervised and evaluated during the 600 hour internship experience. School counselor complete the evaluation form for each student using the following evaluation Google form and providing open-feedback. The evaluations are submitted and the instructor carefully reviews each evaluation. If the supervisor identifies any concerns, or the student receives average ratings or lower, the instructor will contact the supervisor and the student to discuss the evaluation. |
| Process to Ensure Validity | Practicum and internship supervisor evaluations, which are carefully revised by program faculty and coordinator, provide an opportunity to assess students in skill development and program outcomes. The data from these experiential courses is collected and analyzed to not only evaluate current students but also to identify areas of improvement and innovation. Additionally, program faculty and coordinator reviews these evaluations for any potential concerns that need to be addressed accordingly. If there are any concerns, the program faculty consults with the site supervisor to develop any potential remediation plan. Site supervisors also attend the semesterly Field Placement Orientation to receive information on our program, expectations for field placement so they can identify opportunities for interns at their sites, as well share information on their sites. Since site supervisors and faculty consistently review and discuss students’ actual progress based on these surveys and expectations laid down by CACREP and ASCA National School Counseling model, this survey undergoes regular content validity check by different stakeholders, ensuring its validity and revising it with changing standards and expectations. |
| Process to Ensure Reliability | The reliability of the instrument comes from two principal sources, the number of
responders and the extent that their responses concur with one other. Each year the
data set yields the percent of respondents who gave specified answers to each item
and includes reliability estimates in the form of confidence intervals based on the
number of respondents and the concurrence or homogeneity of responses. All our surveys until last year were in the form of hard-copies collected at the end of semester, which we could not access due to the pandemic. In response to the COVID-19 pandemic and to make the process easier for site-supervisors, we transferred the questions of this survey to a google form, which we started collecting from Spring 2020 onwards. In standard 1, we demonstrated internship supervisor evaluation data for each targeted year, demonstrating a high quality performance and reliability of this measure in ensuring students professional growth as a result of their internship experience. In standard 1 documentation, we have used different items of this survey to demonstrate candidates' strong knowledge in learning theories, cultural competence, and data literacy. On average the analysis of all these items showed our students receiving the rating of “very satisfactory” which aligns with overall feedback that our students also get from their K-12 clients, administration, site supervisors, and program faculty - demonstrating the reliability of these survey results. In future, we want to analyse the data collected over at least 3 academic years to calculate its reliability estimates. |
| Process to Ensure Fairness/ Trustworthiness | Faculty and site supervisors work closely to assess students’ performance throughout the program. These collaborative evaluation processes can support students’ plan to effectively work with students in multiple education settings. Given the close collaboration of site supervisors and school counseling faculty in evaluating students’ professional strengths and skill-set as counselors-in-training, the process of using this survey ensures fairness and trustworthiness in evaluating all students’ strengths and areas that need improvement. |
| Key Data Measure: Clinical Review | |
| Description of Measure | Counselor educators play an important role as gate-keepers to ensure that only competent counselors graduate given their critical influence on the lives of the clients they will serve. We use Clinical Review evaluation Google Form to engage in a systematic progress review. Clinical Review evaluations are dichotomous with two results: Pass and Fail. Students were assessed on 10 areas: (a) attention to ethical and legal concerns, (b) cooperativeness of others, (c) awareness of own impact on others, (d) ability to deal with conflict, (e) openness to new ideas, (f) tolerates ambiguity, (g) willingness to accept and use feedback, (h) ability to accept personal responsibility, (i) ability to express feelings effectively and appropriately, and (j) initiative and motivation using a 5-point likert-type scale with 1 being lowest rating and 5 being the highest. |
| Process to Ensure Validity | Practicum and internship course instructors as well as all faculty members who teach counseling courses to school counseling students can engage in Clinical Review to evaluate and discuss any concerns that students might demonstrate. Since faculty consistently review and discuss students’ actual progress based on the clinical review survey in line with expectations laid down by CACREP, this survey undergoes regular content validity check by different stakeholders, ensuring its validity and revising it with changing standards and expectations. |
| Process to Ensure Reliability | The reliability of the instrument comes from two principal sources, the number of responders and the consistency of using this measure across all courses (especially practicum and internship courses). Each year, we hold clinical review committee meeting in which we discuss any student that scored low on clinical review assessment. If a student gets low score, we meet the student to discuss concerns and develop a remediation plan in collaboration with concerned faculty, program coordinator and department chair. This process ensures reliability of the clinical review assessment. Analysis of last three semesters has shown that all students passed clinical review. In future, we want to analyse the data collected over at least 3 academic years to further ensure its reliability. |
| Process to Ensure Fairness/ Trustworthiness | Faculty teaching internship and practicum courses play a critical role in assessing students’ performance and clinical skills and especially in ensuring that they have the personal and professional dispositions needed to serve future K-12 students as school counselors. Given the consistency of this measure across years and for all students from different counseling specializations, the process of using this survey ensures fairness and trustworthiness in evaluating all students’ potential as future school counselors. |
| Key Data Measure: Site Supervisor Program Evaluation Survey | |
| Description of Measure | The Program Evaluation Feedback Google form was created and reviewed by the School Counseling faculty to seek consistent feedback from our most important stakeholders, which are site supervisors. |
| Process to Ensure Validity | We have not tested the instrument’s validity. However, if any aspect of the survey is scored lower, the faculty discusses that with site-supervisor to gain deeper insights into site supervisors’ concerns and engage in the process of improving the program based on their feedback. |
| Process to Ensure Reliability | Engaging in open-dialogue about the scores that program receives based on site supervisors observations of students, helps to ensure the instrument’s reliability. However, we have not tested the reliability of this instrument empirically. In future, we need to make greater efforts to analyze the data we collect from this instrument to ensure its reliability and use it for continuous improvement. |
| Process to Ensure Fairness/ Trustworthiness | Faculty consistently seeks site supervisors feedback on students’ personal and professional dispositions as well as the quality of school counseling program in training future school counselors. Since site supervisors are most abreast with the latest trends and needs of K-12 settings, the process of seeking their feedback for program is fair and ensures trustworthiness for the program. |
| Key Data Measure: Advancement to Candidacy | |
| Description of Measure | The Petition of Advancement to Candidacy form serves as a guideline for progress toward and completion of the degree. Students must complete this form by the sixth week of the semester prior to the term in which a student registers for the culminating experience, and/or applies for the graduate degree to be granted. The form can be accessed on this website: |
| Process to Ensure Validity | The courses mentioned in Advancement to Candidacy form are in line with the course catalog, which has been approved by the university. Students complete the Advancement to Candidacy form by filling in the courses they have taken and will take by the end of the program. Each completed form is submitted to the office of Graduate Studies for approval. Signatures are obtained from the student, the program director and the student’s advisor. Once the Advancement to Candidacy is completed, our program is under contract with the student that we will not change any course requirements during their time in the program. Since the form is in line with courses mentioned in course catalog and is signed at various department, school, and university levels to ensure students’ steady progress towards degree completion, and the content on Advancement to Candidacy form reflect the important courses that are needed for professional development of counselors-in-training, it is a valid instrument to support students’ progress. |
| Process to Ensure Reliability | All students referred to the key role of Advancement to Candidacy form during the internal audit process. Students explained that receiving signature from their advisor on Advancement to candidacy form ensured that they met the key requirements for degree completion. The Advancement to Candidacy form also ensures that our program keeps with the requirements of the Graduate studies and State of California. |
| Process to Ensure Fairness/ Trustworthiness | Program faculty work closely with students to support their successful completion of the program and transition into professional careers. Students have an assigned advisor that helps planning their degree plan for graduation within the five years after admission. Advisors complete an advising sheet with students at the beginning of their program. One key role of advisor is to sign students’ Advancement to Candidacy form, which is in alignment with the courses listed in advising form. This ensures that all students receive the same information about courses needed and receive a mid-program check-in to ensure their successful progress towards degree completion. |
| Key Data Measure: Comprehensive Exam | |
| Description of Measure | A comprehensive examination is an assessment of the student’s ability to integrate the knowledge, show critical thinking and demonstrate mastery of school counseling. Our students have successfully passed the Comprehensive exam suggesting that they have demonstrated high levels of proficiency in the content, pedagogical, and professional knowledge necessary to function as a school counselor. |
| Process to Ensure Validity | Comprehensive exam questions are consistently developed, reviewed, and revised by the faculty within the program. Faculty hired to teach within the programs are considered experts in their specific field and all have relevant experience for the content of the courses which they are teaching. The questions developed are in line with counseling standards and well-established texts, thus demonstrating strong content validity. |
| Process to Ensure Reliability | We use a standard rubric to analyze students’ responses to comprehensive exam essay. This ensures reliability among the faculty evaluating the surveys. Moreover, two full-time school counseling faculty members evaluate students’ essays and grade them as per rubric. If one faculty member passes a student but other does not, the student’s essay is shared with third faculty member in the program who then evaluates the student’s essay independent and helps in making a reliable decision on passing or failing the student. |
| Process to Ensure Fairness/ Trustworthiness | Every year, for a culminating experience, students in the MS in School Counseling may elect to take a Comprehensive Exam during the final stage of their program. For this standard, we are choosing students’ comprehensive exam essay as a data source because the essay requires them to analyze a student’s individual and systemic level concerns based on a vignette and further develop counseling goals and an ethically sound as well as multiculturally competent action plan based on their assessment(s). Hence, these essays can directly demonstrate students’ ability to engage in assessment and use the given data to inform their practice within K-12 educational settings. They also provide the program faculty a trustworthy means to guage students proficiency as counselors-in-training and serve the role as gatekeepers of counseling profession to ensure that only the students with strong knowledge and skill-set graduate and serve future clients. To ensure objectivity in evaluating essays and fairness for all students, the comprehensive exam coordinator removes any identifying information on students’ names or year in program from the responses. |
| Quantitative Data Measure: 2021 Survey of Recent School Counseling Graduates | |
| Description of Measure | We use the exit survey to assess completer’s perceptions of various aspects of the preparation program including their field placement experience as well as future educational/career plans. |
| Process to Ensure Validity | Exit Survey serves as a valid measure of program completers’ perceptions of the school counselors’ preparation program because it asks questions directly aligned with the school counseling program’s learning objectives and CCTC standards of PPS credential. All program completers respond to items asking about their preparation of general pedagogical skills, such as their perception of how well the program prepared them to engage in individual and group counseling skills with specific group of students. In this way, the survey is a valid measure of completers’ perceptions of the program. |
| Process to Ensure Reliability | Reliability about evaluation findings comes from two principal sources: the number of evaluation participants and the extent of their concurrence with each other. The evaluation findings become increasingly certain to the extent that the questions are answered by increasing numbers of program completers. We started using this survey during the academic year 2019-2020 limiting our capacity to assess the survey’s reliability. We plan to continue to use the survey every year and engage in analyzing survey’s reliability and then further revising the survey to make it more reliable. |
| Process to Ensure Fairness/ Trustworthiness | Given the content of the questions asked, we believe the survey has face validity and content validity. However, in future we plan to triangulate the findings of this survey through engaging in focus group sessions with completers and students near graduation and then analyzing the results of this survey in the context of these discussions. Doing so would help us to ensure this survey’s validity and fairness. |