Standard 1 Aspect B

Standard 1b: Evidence shows that, by the time of program completion, candidates exhibit knowledge, skills, and abilities of professional educators appropriate to their target credential or degree, including: Learners; learning theory, including social, emotional, and academic dimensions; and application of learning theory

---

Data Sources & Analysis:

Data Source 1

LEE 254 Coaching Presentation

Perspective Captured from Data Source:
Instructor

Rationale for using Data Source:
LEE 254, Supervised Field Experiences in Reading, is a course our candidates take during their final semester in the Reading/Literacy program. This course uses field-based experiences to examine the multiple roles, duties, and expectations of reading professionals across K-12 settings.

Within the course, candidates conduct a series of literacy coaching sessions. Candidates select a professional colleague to collaborate with in a coaching cycle, which consists of initial consultation, observation and/or modeling, and debriefing consultation. As part of the assignment, candidates present video examples of their work and an analysis of their coaching processes. 

The overall assignment is scored using a rubric. For the purposes of determining whether or not our candidates have the necessary abilities to support adult learning, we chose to focus on the rubric dimension “Reflective Analysis.” This component of the rubric assesses candidates’ ability to critically analyze how they supported teachers through collaborative, job-embedded professional development to improve literacy assessment and instruction.

Specific Elements of Data Source Using:
LEE 254 Coaching Videos Rubric Item: Reflective Analysis

Definition of Success for Each Element:
As a program, our goal is for candidates to score at least at the Fair level, which equates to a score of 15. This demonstrates that candidates exhibit an ability to at least identify general qualities of effective coaching.

Displays of Analyzed Data:

Link to Full Dataset:

• 2018-19 SOAP Data
• 2019-20 SOAP Data
• 2020-2021 SOAP Data

Interpretation of Data:
Findings from 2018- 2020 demonstrate that, overall, program candidates scored well above the programmatic target of a 15 on this rubric component. This suggests that candidates leave the program with a clear understanding of and ability to apply effective coaching techniques to support adult learners.

Data Source 2

Reading, Language, Literacy Specialist Credential Comprehensive Exam Item: LEE 278

Perspective Captured from Data Source: Program Faculty

Rationale for using Data Source:
LEE 278 is designed as the initial reading education course in the sequence of courses leading to the Reading/Language Arts Specialist credential and/or the Master of Arts in Education with a concentration in Reading/Language Arts. The major focus of the course is the investigation and understanding of the reading process through various reading theories, and the instructional implications of those theories for K-12 language and literacy development.

The comprehensive exam is a culminating experience option for students who are pursuing the Master's Degree. Prior to the exam, students receive a list of potential questions they then prepare to answer. During the exam, students are given five of these questions and must select and respond to three. Due to changes in program leadership, we currently only have data from Summer and Fall 2020 comprehensive exams. 

Important to note is that not all students who pursue the Reading Added Authorization or Reading Specialist Credential pursue the Master's Degree. Still, annually, 90-95% of students enrolled do pursue the Master's Degree, and so we believe that, for now, this is an appropriate data source to use.

Specific Elements of Data Source Using:
Comprehensive exam scores for the following questions:             
LEE 278a – Identify two similarities, two differences, two benefits, and two limitations between Text-based and Reader-based theories, or Interactive and Critical Theory (choose one set). Be sure to reference published theoretical authors. 
              
LEE 278b – List and describe 4 classroom practices predominately taught by one particular theoretical paradigm (text, reader, interactive, or critical theory), and explain why it supports the theoretical foundation of that paradigm. Be sure to include an author, a theorist, and/or research for each practice. 

Though these two items both focus more on candidates’ knowledge of reading theory rather than learning theory more generally, given the focus of this credential, we believe this is an appropriate data source to review. In particular, Item B asks candidates to apply knowledge of the theory in their work with students.

Exam responses are scored using the Reading/Language Arts Comprehensive Exam Rubric 

Definition of Success for Each Element:
An average score of 2 is considered passing. The students’ exam scores are also submitted to the university as part of the program’s Student Outcomes Assessment Plan (SOAP). For the SOAP, a mean score of ≥3 is considered to have met the learning outcome. Annually, 75% of students are expected to meet the learning outcome.

Displays of Analyzed Data:

Tables of overall scores

 

Tables of individual scores

 

Overall, data demonstrate that candidates are successfully able to discuss and apply the main reading theories discussed within the course. As the table above shows, all candidates passed these items on the exam. However, only 80% of the candidates demonstrated meeting the learning outcome for item a, and only 71% demonstrated meeting the learning outcome for item b. This is an area of concern that we will need to address in the future.

It is additionally problematic that we only have data from two iterations of the exam, and so we will need to continue to analyze the data moving forward.

Next Steps:
While the data we analyzed for this aspect demonstrates that our candidates do develop knowledge of the relevant reading theories and are able to apply their knowledge of those theories in their work with students, we realize that both data sources come from the perspective of the program faculty. We see this as a flaw in our ability to get a full picture of completers’ preparation. 

To address the low percentage of students meeting our program goal for the 278B exam question (only 71% of students who answered this question) it is clear there needs to be a clearer connection between the theoretical approaches discussed in the LEE 278 course ((text, reader, interactive, or critical theory) and classroom literacy practices. This connection an be introduced in LEE 278 through the students’ discussion assignments and reinforced in courses like LEE 224 when learning about literacy strategies, and in LEE 230 when implementing literacy strategies in tutoring contexts. By creating a stronger connection between theory and practice in these courses, students should be able to provide a more detailed response to the comprehensive exam question. 

Moving forward, we intend to revisit our completer survey, with the hope of adding at least one item that addresses learning theory. To help inform our work, we will review the surveys administered by the CCTC to completers of other advanced credential programs to determine if there are items that may be applicable to this standard. 

Aspect C →