MS in Clinical Rehabilitation and Mental Health Counseling and MS in Counseling-Marriage, Family, And Child Counseling Joint Soap
Program Coordinators: Alicia Becton and Chris Lucey
Department and Degree: Department of Counselor Education and Rehabilitation
Degrees: Clinical Rehabilitation and Mental Health Counseling (CRMHC), and Marriage
and Family Child Counseling (MFCC)
Assessment Coordinators: Alicia Becton and Chris Lucey
1. What learning outcome(s) did you assess this year? List all program outcomes you assessed (if you assessed an outcome not listed on your department SOAP please indicate explain). Do not describe the measures or benchmarks in this section Also please only describe major assessment activities in this report. No GE assessment was required for the 2017-2018 academic year.
Major Assessment Report
The CRMHC and MFCC programs assessed one goal across two objectives encompassing effective practice among counseling students in collaborative and interdisciplinary environments. According to the Student Outcomes Assessment Plan (SOAP) listed on the university website, the goal and objectives were as follows:
PLO (Goal): Prepare professional counselors to practice effectively in collaborative
and interdisciplinary environments.
SLO 2.5: Apply professional counseling expertise under direct supervision
SLO 2.6: Demonstrate essential interviewing, counseling, including group work, and
case conceptualization skills as well as evaluation of interventions
2. What assignment or survey did you use to assess the outcomes and what method (criteria or rubric) did you use to evaluate the assignment? If the assignment (activity, survey, etc.) does not correspond to the activities indicated in the timeline on the SOAP, please indicate why. Please clearly indicate how the assignment/survey is able to measure a specific outcome. If after evaluating the assessment you concluded that the measure was not clearly aligned or did not adequately measure the outcome, please discuss this in your report. Please include the benchmark or standard for student performance in your assessment report (if it is stated in your SOAP then this information can just be copied into the report). An example of an expectation or standard would be “On outcome 2.3 we expected at least 80% of students to achieve a score of 3 or above on the rubric.”
The CRMHC and MFCC programs used a variety of instruments including data from (a)
CRMHC SOAP rubrics (i.e., Rehab 211-Case Conceptualization Rubric, REHAB 237-Case
Study Rubric, Rehab 268C-Case Note Rubric, REHAB 265- Community Resource Project,
Rehab 238 and Rehab 239- Counselor Trainee Evaluation,
(b) MFCC SOAP Rubrics (i.e., COUN 200, 201, 202, COUN 208- Evaluation, COUN 232, COUN
233, COUN 234a-c, COUN 238, COUN 239),
(c) the comprehensive examination,
(d) the clinical review assessment form,
(e) counselor dispositions (MFCC) assessment, and
(f) supervisor/employer evaluations (MFCC).
For reference, all rubrics and surveys are attached below in Section 6 under additional
guidelines.
3. What did you discover from the data? Discuss the student performance in relation to your standards or expectations. Be sure to clearly indicate how many students did (or did not) meet the standard for each outcome measured. Where possible, indicate the relative strengths and weaknesses in student performance on the outcome(s).
CRMHC
SLO 2.5 In the area of applying “professional counseling expertise under direct supervision” students scored “Above Average” in REHAB 238-Practicum and 239-Internship, and “competent” in REHAB 268. As such, data for eleven students and nine students represent completion of the REHAB 238-Practicum and REHAB 239-Internship experience, respectively. Specific items were assessed from the Counselor Trainee Evaluation form for REHAB 238-Practicum and REHAB 239-Internship.
Using item A6 “use of background information” on the Counselor Trainee Evaluation as an assessment, the average rating was above average in Practicum (X=4.45) and Internship (X=4.78). Item A11 “counseling theory and techniques”, reflected scores above average in Practicum (X=4.18) and Internship (X=4.89). After assessing item E2 “interpersonal relations with agency staff and supervisors” on the counselor trainee evaluation form as an assessment,
The CRMHC and MFCC programs used a variety of instruments including data from (a)
CRMHC SOAP rubrics (i.e., Rehab 211-Case Conceptualization Rubric, REHAB 237-Case
Study Rubric, Rehab 268C-Case Note Rubric, REHAB 265- Community Resource Project,
Rehab 238 and Rehab 239- Counselor Trainee Evaluation,
(b) MFCC SOAP Rubrics (i.e., COUN 200, 201, 202, COUN 208- Evaluation, COUN 232, COUN
233, COUN 234a-c, COUN 238, COUN 239),
(c) the comprehensive examination,
(d) the clinical review assessment form,
(e) counselor dispositions (MFCC) assessment, and
(f) supervisor/employer evaluations (MFCC).
For reference, all rubrics and surveys are attached below in Section 6 under additional
guidelines.
SLO 2.6 In the area of essential interviewing, counseling, including group work, and case conceptualization skills as well as evaluation of interventions data from REHAB 204 and 211 utilizing the Case Study Rubric suggest students are “developing” and “achieving,” data from REHAB 237-Progress Report-Case Note and 268-Client Case Report suggest students are scoring in the “competent” range or above average, data from REHAB 265- Community Resource Project students are scoring in the “superior” range or above average, and students continue to score above average in REHAB 238-Practicum and REHAB 239-Internship according to items A9, A14, A18, and A19 on the Counselor Trainee Evaluation form.
Specifically, in REHAB 204, mean scores for content of presentation (X=33.7), professional approach including reflexive practice (X=42.3) and logistics of practice (X=8.6) denotes that students (N=20) are scoring mostly in the “developing” range. For this course, students were provided recommendations and sample intake and evaluation documents. Instructors sought consultation in order to improve scores of “developing” to “achieving” range.
In REHAB 211, mean scores in content of presentation, professional approach including reflexive practice, and logistics of practice denotes students (N=13) scored in the “achieving” range during Fall 2017. The one student who scored in the developing range was given another assignment and individual work with the professor to help achieve the achieved range. The standard was met by the end of the Fall 2017 semester. During Spring 2018 semester, the data revealed students (N=24) scored on the achieved range. For this course, students met with faculty for feedback and watch counseling techniques performed by counselors (i.e., individual counseling, group counseling).
In REHAB 237 (N=28), mean scores in counseling, referral, monitoring and follow-up were X=4.01); assessment of current medical conditions (X=4.01), and problem identification and prioritization (X=4.01). It should be noted that six students in REHAB 237 were provided recommendations and offered suggestions to improve original score of assignment. Students were also encouraged to attend the writing studio prior to resubmission.
In REHAB 265 (N=14), mean scores in depth of reflection were “superior” or above average (X=28.6), and “sufficient” in evidence and practice (X=11.25). Approximately 30% of students were unable to provide strong evidence of synthesis of ideas related to assignment. It should be noted that after assessment of the SOAPs and outcomes based on data the faculty plan to remove REHAB 265 from the identified SLOs since it does not explicitly assess the specific areas identified.
In REHAB 268, students (N=8) in Fall 2017 in interviewing, counseling, including group work, and case conceptualization (X=2.69) scored “proficient” and above. It should be noted that five students were provided recommendations and offered suggestions to improve original scores assigned. In Spring 2018, students (N=9) in interviewing, counseling, including group work, and case conceptualization did exceptionally well (N=5)
In REHAB 238 (N=11) and 239 (N=9), using item A9- use of intake information including biopsychosocial history on the counselor trainee evaluation form as an assessment, the average rating was above average for Practicum (X=4.52) and Internship (X= 4.44). It should be noted that five of the respondents (site supervisors) in REHAB 238 marked non-applicable “N/A.” The N/A rating could result from the site supervisor having limited information to make an assessment and/or result from the type of setting in which the practicum is completed (e.g., case management, job development, clinical practice). By using item A14- use of multicultural counseling competencies as tied to case conceptualization, the average rating was above average for Practicum (X=4.91) and Internship (X=4.89). After assessing item A18- use of evidence based culturally sensitive practices, the average rating was above average for Practicum (X=4.16) and Internship (X= 4.33). Item A19- engaging in appropriate use of diagnosis during crisis and trauma yielded an average rating for Practicum (X=3.56) and Internship (3.88). It is important to note that site supervisors marked N/A for Item A19 in Practicum (n=8) and Internship (n=2).
Comprehensive Exam
Using data from the comprehensive examination, during Fall 2017, six students in CRMHC took the examination and passed resulting in a 100% pass rate, and 24 students in MFCC took the examination during the Spring 2018. In Spring 2018, two students in CRMHC took the examination, in which two passed resulting in 100% pass rate. The results from the MFCC Comprehensive Exam of Spring 2018 yielded a 75% pass rate (6 out of 24 students who took the exam did not pass). While those who passed, the exam did well in the most important issues (crisis, legal and ethical issues), a notable weakness was the student’s responses across the board was their treatment planning abilities.
During the Fall and Spring examinations, students in CRMHC were presented with five vignettes in which students are required to write on three out of the five. Vignettes include case conceptualization relative to rehabilitation counseling, diagnosis and assessment measures, and theories and techniques overview. Additionally, ethical dilemmas that involved diversity issues including but not limited to race, sexuality, religion/spirituality, age, personal/professional values, and education in which they were responsible for writing about the cultural, legal, ethical, and clinical factors associated with the case. Students enrolled in MFCC also respond to a clinical vignette and respond in essay form to address the following: family's strengths, diversity issues, ability to properly diagnose, crisis issues, legal issues, ethical issues, systemic assessment and treatment planning.
Students are strongly encouraged to meet with their advisors prior to registering for and taking the examination in order to review necessary accommodations that might be needed through Services to Students with Disabilities (SSD), assess strengths and weaknesses (e.g., knowledge of content, but being able to succinctly demonstrate knowledge and skills within a specific time frame for the exam), discuss test-taking strategies (e.g., practice engaging in timed writing responses), etc.
For any failed attempt, students must meet with program faculty prior to re-registering for the examination. Students are also provided with a packet of material including scholarly resources, sample questions, and student responses-all identifying information is removed (questions from previous years are not in rotation to appear on the examination; prior student responses are offered, so that students obtain an idea for the type of depth and citing of resources needed to in the allotted amount of time). Students are strongly encouraged to attend a review session in order to gain familiarity with the structure of the examination and knowledge domains, which aligns with the national accreditation through the Council of Accreditation for Counseling and Related Educational Programs (CACREP). The knowledge domains/core content areas for national accreditation can be accessed via the CACREP website https://www.cacrep.org/.
Clinical Review
Using data from the clinical review tool to assess SLO 2.5-professional counseling expertise under direct supervision, and SLO 2.6-demonstrate essential interviewing, counseling, including group work, and case conceptualization skills as well as evaluation of interventions, 97% of students (N=11) scored above average at level “2.0” on item 23 (ethical/professional; X=2.72) and item 24 (cultural diversity/sensitivity; X=2.91). Additionally, students scored above average on item 5 (rehabilitation counseling; X=2.55). This data was extracted from REHAB 238- Practicum as students receive an “Advanced Clinical Review” during this time in the program. The program had an opportunity to assess the application of knowledge and skills in the professional counseling arena (e.g., ethics, case conceptualization) while students work directly with clients from various community agencies.
From Fall 2017 to spring 2018, all students enrolled in the COUN 208: Individual Counseling and COUN 238: Advanced Practicum courses were evaluated by the Counselor Education Programs’ Clinical Review Committee. Using data from the clinical review tool to assess SLO 2.5-professional counseling expertise under direct supervision, and SLO 2.6-demonstrate
Employer/Supervisor Surveys
A review of Employer/Supervisor surveys during the internship course found high reported satisfaction with the programs training of student counselors. A rating of 3.5 on a 5-point Employer’s Evaluation Forms in educational training and clinical expertise is the program’s benchmark.
Using data from the Employer/Supervisor surveys to assess SLO 2.5-professional counseling expertise under direct supervision, and SLO 2.6-demonstrate essential interviewing, counseling, including group work, and case conceptualization skills as well as evaluation of interventions. Data revealed students assessed scored above the 3.5 benchmark on the following items; #2 ability to counsel individuals (X=4.52); item # 3 ability to counsel in groups (X=4.41); item #3 ability to counsel families (X=4.03; item # 10 ability to utilize effective clinical judgement in the assessment of client needs (X=4.65); and item # 14 ability to accurately diagnose and develop treatment plans (X=4.30) (n = 27). One Student was rated as 1 on all five items and the program removed the student from the field-site.
Counselor Dispositional Assessment
Students were assessed using the department’s 3-point likert-type Counselor Dispositional Assessment device during Practicum (i.e., COUN 208, REHAB 238) and COUN 238 Advance Practicum. Items used to assess SLO 2.5 and SLO 2.6 included Diagnose clients’ needs by interpreting data from diverse sources (e.g., formal/informal assessments, student/client behavior and feedback, and collateral responses (X= 2.73) and Is open to corrective feedback (X=2.78). SLO 2.6 also included items Develop intervention plans compatible with diverse needs of clients (X=2.54) and Applies assess, reflects upon, and adjusts counseling strategies to different needs of clients (x = 2.54). All students with the exception of three scored in the competent range and were reviewed during Clinical Review.
Information from the assessment activities were brought to program(s) and department meetings for review, discussions, and planning. Additionally, the CRMHC and MFCC advisory board was consulted. Based on assessments, the program faculty is improving individual mentoring and advising, documenting student progress, making changes to the curriculum or communicating to all instructors on areas that need to be included in each course, specifically based on national accreditation (i.e., CACREP) standards. Increasing students’ knowledge in and application of theory, group counseling and case conceptualization in addition to other information continues as both programs are currently accredited by CACREP, worked to develop a joint SOAPs assessment, and modified/restructured several courses (N=13). The CER department is enhancing communication with part-time instructors and site supervisors (both internship and practicum) in order to develop critical thinking skills and be able to evaluate different worldviews, perspectives, and theoretical orientations.
Full and part-time faculty are working hard to emphasize content and foundational skills (e.g., ethics in counseling, professionalism). As a department, we have decided to continue highlighting this emphasis at our program orientation, in our respective advising meetings with students, and in our individual courses. The programs have a very active advisory board, which meets twice throughout the academic year. The programs present comprehensive examination results, and also engage in lively discussion regarding foundational skills (e.g., writing, problem-solving, conflict resolution, oral communication, overall professionalism, etc.) in professional counseling and curricular content changes (e.g., Are there gaps in knowledge or skills set that you are seeing among students who are in engaging in practicum, internship, or as new employees?). We have found conversations between faculty and community partners/prospective employers are essential in assessing and further shaping our learning outcomes. Many of the instructors (part and full-time) integrate community partners into their class sessions on a regular basis. The programs plan to continue these efforts, as community partners (ones who work with practicum/internship students and hire students as new employees) are able to reinforce content and the importance of possessing and translating foundational skills (i.e., case conceptualization, case recording/documentation, ethics in counseling, etc.) in the real world.
In addition, based on the assessments, the program faculty is improving individual mentoring and advising, documenting student progress, making changes to the curriculum or communicating to all instructors on areas that need to be included in each course. Changes have been made to the Assessment Coordinator position and a more in-depth evaluation of the assessment process has begun. Forms have also been digitized to facilitate the organization, analysis and implications of the data received from the assessment tools. Finally, CER department is enhancing communication with part-time instructors and site supervisors (both internship and practicum) about treatment planning and the intentional use of interventions.
4. What changes did you make as a result of the data? Describe how the information from the assessment activity was reviewed and what action was taken based on the analysis of the assessment data.
The CRMHC and MFCC programs will continue assessing students’ skills, knowledge and dispositions during (practicum, and during internship courses. The programs will also continue to conduct ongoing Clinical Reviews to assess student concerns and provide support. Assessments will be re-evaluated to determine if changes need to be made on the actual items (questions being asked to assess students’ skills), and the remainder of the assessments that have not been digitized will be developed.
5. What assessment activities will you be conducting in the 2018-2019 AY? List the outcomes and measures or assessment activities you will use to evaluate them. These activities should be the same as those indicated on your current SOAP timeline; if they are not please explain.
While the programs have revised, combined the SOAP plans and consulted with university assessment personnel, at present, we are scheduled to assess SLO (4.1, 5.1) using data from Rehab 237 and COUN 233, in the areas of:
• SLO 4.1: Demonstrate skills in assessment, evaluation, and case management when
working with individuals, couples, and families from a systems perspective.
• SLO 5.1: Implement and maintain an active relationship to the broader rehabilitation
community and related professional associations through consultation, education, advocacy
and leadership activities.
We are also scheduled to review/revise our comprehensive examination, which we are already diligently working on; as such, our efforts will continue.
6. What progress have you made on items from your last program review action plan? Please provide a brief description of progress made on each item listed in the action plan. If no progress has been made on an action item, simply state “no progress.”
The programs submitted a joint self-study to CACREP and have worked closely on streamlining standards. The FTEF to FTES ratios have also been an ongoing issue between the Department of Counselor Education and Rehabilitation, and University administration. The CER department has reduced the number of applicants accepted into programs, recently hired three full-time tenured track faculty (as of Fall 2018) and the department currently has open searches for one new full-time faculty member to begin in Fall 2019. The hiring of new faculty as well as continued efforts to manage enrollments will continue to be made in order to decrease the FTE ratio.
Additionally, the department continues to use pass rates on the comprehensive examination as a key indicator of student learning and closing the loop. The majority of students enrolled in
Additional Guidelines: If you have not fully described the assignment then please
attach a copy of the questions or assignment guidelines. If you are using a rubric
and did not fully describe this rubric (or the criteria being used) than please attach
a copy of the rubric. If you administered a survey please consider attaching a copy
of the survey so that the Learning Assessment Team (LAT) can review the questions.
the program choose the examination as their culminating experience. Data from the
examination aids the program in highlighting areas of strength and weaknesses (e.g.,
where are students performing well; where is additional support needed; what are faculty
doing well; how can assignments be restructured; do we need to improve relative to
student-instructor interaction, content, etc.).
Another area of achievement is the pass rate on the national examination. During the
2017-2018 academic year, students enrolled in the CRMHC program experienced 75% pass
rate on the first attempt. While, the examination entity has changed its practices,
and the program now has to rely on students relaying information of whether they passed
or did not pass with supplemental documentation, the program’s in-house data collection
system reflects that the pass rate has increased by 7%. Preparation sessions being
conducted by a faculty member, and the joint emphasis on professional identity by
all faculty members is leading to an increase in pass rates. More students are also
being invited to work on manuscripts being submitted for publication, grants being
submitted for funding, and involvement in student leadership and community-based activities,
which the programs think is strengthening efforts in the classroom, and impacting
student knowledge and foundational learning outcomes.
Please see attached rubrics and surveys.
Counselor Trainee Evaluation: http://bit.ly/CRMHCCounEval
CRMHC Clinical Review: http://bit.ly/CRMHCClinicalReview
REHAB 204- Case Study Rubric: http://bit.ly/REHAB204CasestudyRubric
REHAB 237- Case Note Rubric: http://bit.ly/REHAB237ProgressReport
REHAB 211-Case Study Rubric: http://bit.ly/REHAB211rubric
REHAB 268-Case Note Rubric (note adjustment): http://bit.ly/REHAB268casesreport
REHAB 265- Community Resource Project: http://bit.ly/REHAB265CRP
Counselor Disposition: http://bit.ly/CounselorDisposition
Ethics Policy: http://bit.ly/ETHICSPOLICY
MFCC Clinical Review: http://bit.ly/MFCCCLINICALREVIEW
Graduate Writing Rubric: http://bit.ly/MFCCGWR
COUN 232- Vignette Rubric: http://bit.ly/COUN232VIGNETTERUBRIC
COUN 202- Group Leader Rubric: http://bit.ly/202GROUPLEADERRUBRIC
COUN 208- Practicum Evaluation: http://bit.ly/COUN208practicumevaluation
COUN 238- Case Presentation Rubric: http://bit.ly/COUN238CASERUBRIC
COUN 238- Advanced Practicum Evaluation: http://bit.ly/AdvancedPracticumEvaluation
COUN 239- Counselor Evaluation- http://bit.ly/COUN239evaluation
COUN 239- Case Presentation Rubric http://bit.ly/COUN239ORALPRESENTATIONRUBRIC
This HTML version was generated with AI assistance and may contain errors or omissions. The original approved PDF remains the official document of record. An alternative accessible format is available upon request from the document creator in accordance with Title II of the ADA.