Standard 3 Aspect E

Standard 3e: Preparation programs ensure that candidates, upon completion, are ready to engage in professional practice, to adapt to a variety of professional settings, and to grow throughout their careers. Effective program practices include: consistent offering of coherent curricula; high-quality quality, diverse clinical experiences; dynamic, mutually beneficial partnerships with stakeholders; and comprehensive and transparent quality assurance processes informed by trustworthy evidence. Each aspect of the program is appropriate to its context and to the credential or degree sought.

---

The changes included in the following table came about through effective use of the quality assurance system. Specifically, awareness of innovations in school counseling through attendance at national and state-level conferences; examination of the California requirements, CACREP model, and ASCA training standards led to a review of program curriculum. Discussion with students, faculty, district leadership, practicing school counselors (stakeholders) also supported these changes. 

Internal Audit Process:
In order to engage in continuous improvement, program faculty work closely with stakeholders. In addition to the attention to the field of school counseling, students’ performance are assessed with an internal audit. The program has collected data from a group of students for quality assurance. Apart from reviewing the documentation of several randomly selected students, 9 students were contacted by a faculty member to seek their feedback. Four students were available to provide responses to the following open-ended questions (See Appendix D): 

1. Please share how you learned about our program and what influenced your decision to pursue M.S. in School Counseling at Fresno State? 
2. Please share what steps and processes you had to follow to gain admission in the M.S. in School Counseling program. 
3. What helped you or prevented you from staying on track towards graduation? 
4. Can you share about your experiences associated with advising to meet your educational goals? 
5. Please share about your final clinical placement with regard to its requirements and criteria. 
6. What process did you follow to complete your M.S. in School Counseling program with PPS credential? 
7. After graduation, did you connect with your advisor for any reason and what was your experience like? 

Our findings from the audit showed that all students had a clear pathway that they followed from the time of recruitment to completion. Regarding recruitment, students talked about the utility of comprehensive information and clear directions that are provided on the program's website. Students also mentioned the important role of their undergraduate faculty advisors in sharing about the School Counseling program and connecting the student with the School Counseling Program Coordinator, who then helped the students learn more about and apply for Masters in School Counseling degree. Students’ responses further showed consistency in the admissions process that required all students to submit the same documents including university application, program application, transcripts, statement of purpose, resume, and letters of recommendation. 

Two students mentioned submitting a certificate of clearance that the school counseling program faculty added as a “required” document while applying for the school counseling program. The decision to include the Certificate of Clearance as a required document was the result of conversations with stakeholders such as Fresno Unified liaison and COUN 249 instructors who shared that, without having a certificate of clearance, students cannot work in K-12 settings during their internship. If students’ certificate of clearance was not submitted on time then students could not begin their internship and hence obtain their degree. To avoid these later conflicts and disappointments for students, the school counseling program added the requirement of certificate of clearance submission in the admissions process. 

With regard to advising and monitoring, students appreciated consistent and quality advising provided by their assigned faculty advisors, program faculty with whom they developed positive mentoring relationships, and the school counseling program coordinator. A student expressed that their assigned faculty advisor “was always quick to respond to my questions and advocate for all students.” Students explained that their advising sessions helped them gain information about program roadmap (what courses to take in which sequence), practicum consultation, culminating experience options (e.g., thesis, project, or comprehensive exam), completing the Advancement to Candidacy form, applying for internships, and applying for jobs. One student also mentioned the role of being a student ambassador and being in consistent communication with the school counseling program coordinator as the means to gain whatever information they needed to make a steady progress towards their degree completion. Students’ insights showed that apart from the consistent advising that faculty advisors provide, it is critical to ensure active involvement of school counseling program coordinators in providing group and individual information sessions to students. 

With regard to field placement, all students completed the required number of 600 hours, even amidst the challenges posed by COVID-19 pandemic, by engaging in effective zoom counseling sessions with K-12 students. 

Finally, regarding the experiences associated with program completion, students highlighted a clear and consistent pattern of steps taken to apply for graduation and PPS credential. Students’ responses showed the positive role of advisors and school counseling program coordinator reaching out to students to explain the documents that needed to be submitted to successfully graduate from the program. One student also talked about the role of advising in helping them search, apply, and prepare for job opportunities as school counselors. 

Even if all faculty support students with resume-building skills and writing letters of recommendation, starting from Fall 2021, we plan to integrate the job application process and career resources within the COUN 249 internship class, which is the last class that students take before applying for full-time positions as school counseling graduates. We plan to do so to ensure that all students get required job-related resources, advising, and encouragement apart from those who themselves take initiative to reach out to program faculty, advisors, and coordinator to seek support with job application.

Beginning in Fall 2022, the program aims to implement a biannual audit to scrutinize our program in a deeper, more detailed way than we did when developing this self-study. The AAQEP framework provides excellent guidelines to assess our program and identify ways to improve our training of future school counselors. The audit process will allow us to identify strengths and weaknesses, implement interventions to address weaknesses and ensure ongoing improvement and innovation. We will report findings and recommendations based on our careful inspection of our internal review system. During the Internal Review System, we will specifically assess recruitment, admission, advising, monitoring and placement. A random group of students’ files will be assessed for performance in all areas of recruitment process, admissions, advising, monitoring, final clinical placement, and completion. We will track 5 students across 3 different years to have a better understanding of their performance and experience in our program. We will trace their progress through our program from recruitment/application to completion. Data for each selected student will be compiled and analyzed to improve our performance, create opportunities for innovation, and ensure adherence to high quality standards. 

University Audit Process:
In addition to the department-level audit process, we engaged in continuous improvement through the University-level external program review for the M.S program in School Counseling that required a review process and a series of meetings for completion of this work. We followed the sequence mentioned below to engage in university audit process:

• Program Review - Academics 

Based on the review results, an Action plan was developed for the program. In addition, the program is expected to engage in annual reports to the dean to demonstrate progress toward accomplishing goals and tasks outlined in our action plan. As mentioned in the action plan document, as a result of the M.S. in Counseling program review, several actions will be taken that center on assessment, field placement experiences, new faculty hires, and course development. The review was overall satisfactory with clear goals and direction for the programs to continue training well prepared school counselors. 

Process for Ensuring Data Quality through Validity, Reliability, Trustworthiness, & Fairness:

Evaluating Instruments
We use several instruments--including assignments, course grades, field placement evaluations, and surveys--to evaluate the success of our program in supporting students to meet the standards described by AAQEP. Following are the descriptions of instruments used during three different phases of the program: admissions, gaining knowledge and skills in counseling, and completing the program with successful field experience and culminating requirements (e.g., comprehensive exam) (See Appendix E):

ADMISSIONS

1. We have listed clear criteria for admissions on our website including undergraduate GPA. M.S. in School Counseling Graduate program applicants must have a minimum GPA of 3.0 in the last 60 semester (90 quarter) units attempted and a minimum of 3.0 on any post baccalaureate work at Fresno State. Undergraduate GPA is a commonly used metric in admission as an indicator of expected academic success of candidates in counselor education programs, which ensures the validity of using this as a criteria for admission. It is also a reliable indicator because Undergraduate GPA is used by admission evaluators to assess the academic ability of candidates and is commonly used throughout the field. GPA requirements are established by the California Commission on Teacher Credentialing and by Executive Order of the California State
   University Chancellor. Further, it is a fair criteria All candidates have completed the same requirements. Potential students who are not admitted can file a petition for Special Consideration which is considered by a committee. The good faith of a provider of the data as well as the provider’s reputation assure that the data represents what it is supposed to represent and that there is no intent to misrepresent. Regional accreditation is required for all transcripts. We have also stated the clear expectation about GPA on our application.
2. Candidates progress towards degree completion is assessed through Advancement to Candidacy, which serves as a key data as a guideline for progress toward and completion of the degree. Students must complete this form by the sixth week of the semester prior to the term in which a student registers for the culminating experience, and/or applies for the graduate degree to be granted. The form can be accessed on the Graduate Studies website. The courses mentioned in Advancement to Candidacy form are in line with course catalog, which has been approved by university. Students complete the Advancement to Candidacy form by filling in the courses they have taken and will take by the end of the program. Each completed form is submitted to the office of Graduate Studies for approval. Signatures are obtained from the student, the program director and the student’s advisor. This ensures this form’s validity. Since, advancement to candidacy form ensures that all students meet the key requirements for degree completion and is submitted by all students listing specific courses they have taken and plan to take along with their chosen culminating experience, it is a reliable measure. To ensure fairness, program faculty work closely with students to support their successful completion of the program and transition into professional careers. Students have an assigned advisor that helps planning their degree plan for graduation within the five years after admission. Advisors complete an advising sheet with students at the beginning of their program. One key role of advisor is to sign students’ Advancement to Candidacy form, which is in alignment with the courses listed in advising form. This ensures that all students receive the same information about courses needed and receive a mid-program check-in to ensure their successful progress towards degree completion

KNOWLEDGE AND SKILLS IN COUNSELING 

1. COUN 201 Reflective Assignment: Reflective assignments in COUN 201 that require research-based information to understand students’ experiences with intersectionality serve as a key data measure for standard 1c. For these assignments, students were expected to evaluate their own identity as well as the intersectionality of multiple factors affecting their own experience and learning. Additionally, students explored the impact of privilege and oppression on their own development and future practice. Since the assignment’s expectations are in line with enhancing students’ cultural competence, it is a valid data source for Standard 1C. 
   To assess the reliability of this assignment, we analyzed students’ assignments and found that students scored above satisfactory in all areas with scores higher than 12 points total (M=13.39). Overtime, we need to continue to analyze the data to ensure the assignment’s reliability. We believe this assignment demonstrates students’ understanding about intersectionality and cultural competence. There is also a clear rubric that assesses students’ cultural competence based on this assignment, contributing to the assignment’s fairness. 
2. COUN 201 Research Based Assignment: Research based projects in COUN 201 invite candidates to analyze and understand the experiences of marginalized groups and how those experiences are shaped by environmental forces serve as an important data source for Standard 1C. Through engaging in this assignment, candidates were expected to explore and understand the experience of marginalized groups to understand their lived experiences. Additionally, candidates researched the impact of systematic forces on this group as well as ways to enhance the life of these marginalized groups. Since this assignment encourages candidates to explore, understand, research, and articulate the lived experiences of people belonging to marginalized groups, it is a valid source for analyzing Standard 1C. 
   Candidates were evaluated in their ability to follow instructions, write their ideas clearly, organize material, present content effectively, and engage in self-work. Candidates were required to incorporate research-based resources to improve their understanding of the experience of marginalized groups and to learn about interventions that could improve the lived experiences of these groups. Providing candidates these clear directions helped to ensure this assignment’s reliability in supporting students to engage in the specific requirements of this assignment. It will be important to keep analyzing candidates’’ responses to this assignment to assess its reliability overtime in expanding candidates’ cultural competence. We strive to ensure that the assignment is fair for all candidates by using the following rubric, which helps to ensure that all candidates are assessed fairly. Through engaging candidates in this assignment, our hope was to evaluate their understanding of the impact of external and internal forces on marginalized groups, which will be important for strengthening all candidates’ ability to provide counseling to a diverse group of K-12 students, hence making this assignment a fair choice. 
3. COUN 206 Exam: Mid-Term Exam in COUN 206 Syllabus for AAQEP Standard 1 b serves as an important data source to evaluate candidates’ key knowledge and pedagogical understanding of learning theories in accord with Standard 1A. The Midterm exam consists of a clinical vignette component. The clinical vignette requires students to respond to one clinical vignette in an essay format.  Candidates are expected to demonstrate competency with the relevant material taught to that point in the course. Specifically, candidates need to be able to connect developmental theory to a case example as well as identify normative and non-normative developmental factors that impact the individual(s) behavior. Candidates are also asked to discuss how developmental models across lifespan development explain the behavior seen in the vignette. 
   As indicated in COUN 206 syllabus, the purpose of this course is to provide an overview of various developmental concepts and principles as they relate to the practice of counseling. This course explores topics related to different aspects of development (i.e. social, emotional, moral, cognitive, physical, and other forces of development). This course also covers the elements of the life cycle process and supports students to understand how individual, familial, and social developmental forces interact to create both health and dysfunction. Since this exam assesses candidates’ ability to apply learning and developmental theories to meet the developmental needs of all individuals including K-12 students, it is a valid measure.
   Requirements to pass in the mid-term exam in COUN 206 stay relatively consistent over time since they are aligned with the CCTC program standards and CACREP standards, given that COUN 206 is a core-counseling class in CACREP accredited specializations. In addition, the courses are staffed by the same faculty, on the whole, to ensure reliability in content that is covered and assessed. 
4. COUN 241 Assignment: Research Paper and School Counselor Interview: This paper directly supports candidates pursuing their Masters in School Counseling with a PPS credential to actively read and reflect upon the research-based, peer-reviewed articles that can help them to gain pedagogical content. In addition, candidates are required to interview a professional school counselor on the topic of their interest in relation to school counseling and further reflect upon how they can apply the knowledge gained from this interview. Therefore, this paper is helpful for candidates’ professional development and directly related to the course content and Standard 1 objective of the pedagogical, and/or professional knowledge relevant to the credential/degree sought by students, hence it a valid assessment for Standard1 A. 
   To ensure fairness, faculty provide a rubric along with specific instruction and expectations. In the future, within the assignment description, we plan to add that students must use the knowledge gained from reading peer-reviewed journal articles and engaging in dialogues with professional school counselors to suggest two evidence-based practices that can promote a positive change within school settings – both at individual and systemic levels. We will also make a comprehensive rubric that focuses on applicability of pedagogical content. 
5. COUN 242 GPA: In COUN 242: Parent Education, Pupil Advocacy and Consulting, a course that candidates enroll in toward the end of their program, the final presentation consists of a collaborative project. Together with a group of colleagues, candidates develop a comprehensive program/initiative that tackles an issue in a K-12 school. The assignment aligns with the major learning objectives of COUN 242 of advancing candidates’ understanding about international and global perspectives in counseling that can influence consultation processes within K-12 settings. Therefore, the assignment is a valid measure of students’ competency in standard 2 d on enhancing students’ global and international perspectives. To ensure this assignment’s reliability, course requirements stay relatively consistent over time in order to ensure they are aligned with the CCTC program standards. 
   In the future, the faculty teaching COUN 242 will develop a clear rubric to assess students’ performance and make the process even more fair and reliable. 
6. COUN 249 Case Study: Candidates complete a Case Study report on a K12 student-client with whom they have been working at their field placement site. Apart from presenting about their work with a given student-client, candidates are also required to submit a written assignment based on the specifics provided in the assignment details. 
   The assignment is a valid measure because it directly aligns with CCTC Standards 23, 26, and 29, which focus on advocacy, group counseling and facilitation, and prevention, education, and training to support K-12 students as school counselors.
   To demonstrate the case study assignment’s validity and reliability, we engaged in thematic analysis of data of randomly selected student assignments. The assignment has been constantly revised, as per the latest school counseling standards and faculty members’ own growing expertise in school counseling research and school counselor education. For instance, in last two years, faculty members have created a rubric to evaluate the assignment based on extensive discussions and then also added the component on ASCA national model mindsets to further enrich this assignment and ensure its actual reliability in helping school counseling interns translate what they have learned to promote school students academic success, socio-emotional wellbeing, and career development. Specifically, in the case-study, candidates have to also explain how their case conceptualization and interventions align with American School Counseling Association’s National Model of School Counseling mindsets. All candidates present their case study in class discussions that enable peers to engage in case consultation and ensure that case study is actually used to resolve the real concerns of K-12 students. Doing so enhances the utility of this assignment in promoting K-12 students’ wellbeing and success. Since all candidates use the provided format to write the case study, the assignment is fair and trustworthy for all. Stakeholders such as site supervisors are also aware of this assignment and support students to successfully pursue their case study, further ensuring support for students. Moving forward, program faculty will meet to calibrate the rubric to ensure all faculty are using the rubric to assess this assignment in the same way.
7. COUN 249 Lesson Plan: Candidates are required to create a 20-minute Lesson Plan based on the ASCA model and present it to the class. The assignment is in direct line with American School Counseling Association’s focus on engaging in data-driven practices through the lesson plan. The content of the assignment demonstrates validity in terms of assessing Standard 1e’s objective of creating positive learning and work environments. 
   We engaged in a qualitative analysis of lesson plan assignments of randomly selected students to demonstrate the trustworthiness of the assignment. Our goal was to see whether candidates provided materials and ideas that would engage the audience, are relevant to the skills/age group, and are collaborative in nature to ensure a positive learning and working environment.
   We used the COUN 249 lesson plan assignment because it complemented the case study (individual intervention) through engaging in systemic level efforts to promote positive change for a group of students or staff (group intervention).  The lesson plan is a way for candidates to brainstorm how to create a curriculum that teaches and engages the audience (to create a positive learning environment) and to use the ASCA Mindsets and Behavior Standards relevant to a positive learning and working environment. Since the assignment is the same for all candidates and expectations are made clear, we believe the assessment is fair. Also, since the assignment is in line with ASCA national model, it will help candidates prepare for their future school counseling roles.
8. The Comprehensive Exam serves as a key measure of assessing candidates’ pedagogical understanding, knowledge, and ability to apply counseling theories to resolve K-12 students’ concerns. As such, the comprehensive examination is an assessment of the candidate’s ability to integrate the knowledge, show critical thinking and demonstrate mastery of school counseling. Our candidates have successfully passed the comprehensive exam suggesting they have demonstrated high levels of proficiency in the content, pedagogical, and professional knowledge necessary to function as a school counselor. Comprehensive exam questions are consistently developed, reviewed, and revised by the faculty within the program. Faculty hired to teach within the programs are considered experts in their specific field and all have relevant experience for the content of the courses which they are teaching. The questions developed are in line with counseling standards and well-established texts, thus demonstrating strong content validity. 
   We use a standard rubric to analyze candidates’ responses to the comprehensive exam essay. This ensures reliability among the faculty evaluating the surveys. Moreover, two full-time school counseling faculty members evaluate students’ essays and grade them as per the rubric. If one faculty member passes a student but the other does not, the candidate’s essay is shared with a third faculty member in the program who then evaluates the student’s essay independently and helps in making a reliable decision on passing or failing the student.
   To ensure objectivity in evaluating essays and fairness for all candidate, the comprehensive exam coordinator removes any identifying information on candidates’ names or year in program from the responses and holds regular advising sessions through Chi Sigma Iota to help candidates prepare for comprehensive exams. Coordinators also revise all course and school counseling syllabi consistently to ensure that content assessed in the comprehensive exam is covered in all sections of classes. 

PROGRAM COMPLETION

1. Internship Evaluations: At the end of each semester, site supervisors of field practice are asked to complete three assessments on the field practice students: Evaluation of Field Placement Student, Program Evaluation Feedback, Student Disposition.  
   The Evaluation of Field Placement Student is based on California Commission on Teacher Credentialing (CCTC) program standards, using wordings from the standards.  Four tenured/tenure-track faculty in the School Counseling Program reviewed and edited the instrument before use. 
   Directions included asking the site supervisors to indicate whether the student's performance was: 1 = very unsatisfactory; 2 = moderately unsatisfactory; 3 = moderately satisfactory; 4 = Very Satisfactory in each category provided.  
   The Program Evaluation Feedback and Student Disposition Survey were also created and reviewed by the School Counseling faculty. We have not tested these instruments for reliability or validity of the point scale assessments that we use.  However, if any candidate is rated as underperforming at any time of the semester or at the end of the semester, the instructor contacts the site supervisor to see if remediation steps are needed to help the candidate improve.  If the site supervisor and instructor believe the candidate is in need of interventions to improve before moving forward in his/her program, the instructor will refer the candidate to the Department Clinical Review Committee, consisting of tenured track and adjunct faculty.  Click here to see the Student Handbook, p.12 for information on the Clinical Review Process.  The Clinical Review Committee will determine if remediation steps are needed or if dismissal from the program is warranted.
   Note that the Student Disposition Survey is an instrument which we used in all our counseling programs during practicum and field practice courses.   These instruments have been used to achieve Council for Accreditation of Counseling and Related Educational Programs (CACREP) accreditation for the Clinical Rehabilitation and Mental Health and the Marriage, Family, and Child Counseling master programs.  This instrument uses a 0-3 point scale with 0 being not observed, 1 being not met expectations, 2 being met expectations, and 3 being exceeding expectations.  As mentioned, the first time all candidates are evaluated for their professional and personal disposition occurs in their practicum course, Coun208.  Candidates who do not meet expectations will meet with the Clinical Review Committee for remediation to assist the candidates in their progress.
   As a department, we are currently in the process of looking for more reliable instruments that are also valid. The faculty started editing current instruments but decided to look for instruments that have been studied for validity and reliability.  In Tate et al. (2014), 41 instruments for counselor competence, performance assessment, and program evaluation were reviewed for reliability and validity.  The School Counseling Field Practice Coordinator, Dr. Song Lee, with the help of other programs coordinators in the department, will review these 41 instruments and other more recently developed instruments to determine if any are appropriate for use. Before making final selections, stakeholders such as our advisory board and partnership district personnels will be consulted.  If none of the instruments reviewed fit the needs of the department, school counseling program, or the current school counseling field, the faculty will then move to evaluate the current instruments for reliability and validity.
   To ensure reliable administration, the school counseling program is providing training to all site supervisors, per requirements of our accreditation body, CCTC.  The Field Practice Coordinator provides those trainings and will ensure that all site supervisors and practicum instructors are also being trained on how to evaluate the students, using the assessments provided and the directions provided by the instrument developers.
   These evaluation measures also help us ensure fairness for candidates and the future school sites that they will serve. Using the point scale assessments helped us to identify candidates who are struggling.  We then use the Clinical Review process to help underperforming candidates to improve.  If candidates do not meet goals set in conjunction with the Clinical Review Committee, they will be referred to Student Affairs for dismissal from the program. We have consulted and will continue to consult accreditation standards and our partnerships for improvements.
2. Exit Survey: We use an exit survey to assess completer’s perceptions of various aspects of the preparation program including their field placement experience as well as future educational/career plans. The Exit Survey serves as a valid measure of program completers’ perceptions of the school counselors’ preparation program because it asks questions directly aligned with the school counseling program’s learning objectives and CCTC standards of PPS credential. 
   All program completers respond to items asking about their preparation of general pedagogical skills, such as their perception of how well the program prepared them to engage in individual and group counseling skills with specific group of students. In this way, the survey is a valid measure of completers’ perceptions of the program. 
   Reliability about evaluation findings comes from two principal sources: the number of evaluation participants and the extent of their concurrence with each other. The evaluation findings become increasingly certain to the extent that the questions are answered by increasing numbers of program completers. We started using this survey during the academic year 2019-2020 limiting our capacity to assess the survey’s reliability. We plan to continue to use the survey every year and engage in analyzing the survey's reliability and then further revising the survey to make it more reliable. Given the content of the questions asked, we believe the survey has face validity and content validity. However, in the future we plan to triangulate the findings of this survey through engaging in focus group sessions with completers and students near graduation and then analyzing the results of this survey in the context of these discussions. Doing so would help us to ensure this survey’s validity and fairness.

Aspect F →