AAQEP Accreditation 2022

Standard 2 Aspect E

Standard 2e: Program completers engage in professional practice in educational settings and show that they have the skills and abilities to do so in a variety of additional settings and community/cultural contexts. For example, candidates must have broad and general knowledge of the impact of culture and language on learning, yet they cannot, within the context of any given program, experience working with the entire diversity of student identities, or in all types of school environments.

Candidate preparation includes first-hand professional experience accompanied by reflection that prepares candidates to engage effectively in different contexts they may encounter throughout their careers.

Data Sources & Analysis:

Data Source 1

LEE 254 Coaching Presentation

Perspective Captured from Data Source: Instructor

Rationale for using Data Source:
LEE 254, Supervised Field Experiences in Reading, is a course our candidates take during their final semester in the Reading/Literacy program. This course uses field-based experiences to examine the multiple roles, duties, and expectations of reading professionals across K-12 settings.

Within the course, candidates conduct a series of literacy coaching sessions. Candidates select a professional colleague to collaborate with in a coaching cycle, which consists of initial consultation, observation and/or modeling, and debriefing consultation. As part of the assignment, candidates present video examples of their work and an analysis of their coaching processes.

The overall assignment is scored using a rubric. For the purposes of determining whether or not our candidates have the necessary abilities to support adult learning, we chose to focus on the rubric dimension “Reflective Analysis.” This component of the rubric assesses candidates’ ability to critically analyze how they supported teachers through collaborative, job-embedded professional development to improve literacy assessment and instruction and identify insights about their future goals as a literacy coach.

Specific Elements of Data Source:

Reflective Analysis

Clearly highlights key events to share with audience. Analysis includes lessons learned about coaching and identifies critical insights about growth and future goals.

Highlights several events to share with audience. Analysis includes lessons learned about coaching but needs elaboration about growth and future goals.

Includes few events to share with audience. Analysis does not include lessons learned about coaching or insights about growth and future goals.

Definition of Success for Each Element:
As a program, our goal is for candidates to score at least at the Fair level, which equates to a score of 15. This demonstrates that candidates exhibit an ability to at least identify general qualities of effective coaching.

Displays of Analyzed Data:

	Reflective Analysis Mean Score (out of 25)	% of students reaching goal of (score of 15 or higher)
2018 (22) Students	21.82	100%
2019 (12) Students	24.58	100%
2020 (15 students)	23.57	100%

Link to Full Dataset:

Interpretation of Data:
Findings from 2018- 2020 demonstrate that, overall, program candidates scored well above the programmatic target of a 15 on this rubric component. This suggests that candidates leave the program with a clear understanding of and ability to reflect and goal set for their future development.

Data Source 2

LEE 230 Reflection on TORP Assignment

Perspective Captured from Data Source: Candidate

Rationale for using Data Source:
In this class, candidates complete the DeFord Theoretical Orientation to Reading Profile (TORP) survey which rates the candidates’ theoretical orientation toward reading (decoding, skills, or whole language perspectives). The discussion assignment highlights candidates’ ability to reflect on the ideological/philosophical underpinnings of teaching reading, to assess their own stances, and to critically reflect on the implications of that assessment.

Specific Elements of Data Source Using:
LEE 230 Discussion Prompt:
Assignment: Write a Discussion Board post after you have taken the DeFord Theoretical Orientation to Reading Profile (TORP) survey and done the readings (Vacca et al., 2006; Spencer, 2011). Now that you have 'found' your theoretical orientation toward reading (according to the TORP survey), please give your critical response to your placement. Do you agree? Disagree? Why? What might your score indicate about the way(s) you teach reading and stage interventions? (Note: Please bring 1 or both of the readings into this part of your response.)
Key: The TORP score ranges map onto the following theoretical orientations:

0-65 Decoding (also referred to as Phonics, Bottom Up, and Behaviorist)
66-110 Skills (also referred to as Blended, Bottom Up, and Cognitivist)
111-140 Whole Language (also referred to as Holistic, Top Down, and Constructivist)

Posts should be approx. 1/2 to 1 page single-spaced. Respond to at least 1 of your classmates' posts (approx. 1 paragraph).

Definition of Success for Each Element:
Candidate reflection assignments are deemed to be successful in terms of how they generate students’ continuing, iterative reflection processes. This definition of success pulls from Lather’s (1992) concept of catalytic validity, and applies to work designed to “reorient, focus, and energize participants toward knowing reality in order to transform it.” Success derives from catalyzation and engaging with other ideas, in this case two research articles and peer posts.

Dataset: Candidate Discussion Responses: Truncated and anonymized discussion board posts for five random candidates in 2019 and five random students in 2020, the two years for which data are available

Displays of Analyzed Data: Reflective Stance of Candidates

accordion1

Heading

Content

Fall 2019

Student 1	My score was not a surprise to me due to the reason I actually teach a phonic base reading program for my small groups.
Student 2	I agree with this description of the theory, but I have enough experience to know skill instruction is necessary. Some students thrive with structure, direct instruction lessons of I do, We do, and You do support. I wholeheartedly agree meaning is the essence of why we read.
Student 3	I was a little surprised by this because I identify most with the top-down and reader based theories when we have been reading these various papers during the master's program. I often think of myself as more of a constructivist.
Student 4	I definitely agree with my placement on the TORP assessment. I have taught all grades between K-5 (mostly 2nd, currently in 3rd) and have experience teaching lessons within all three theoretical orientations.
Student 5	I feel that my TORP score does not align with my beliefs. I don’t think I am a skills perspective teacher or have a bottom-up approach to literacy. I always considered myself an interactive approach teacher. I use multiple avenues to try to meet children’s needs. I think that the reason I scored in the skills perspective range is because our district requires us to pull from a basal reading curriculum.

Fall 2020

Student 1	I would overall agree that I am skills based, I would push and say my goal is to be more balanced. I think this balanced approach comes from much of the work I have done during my masters courses.
Student 2	I agree that I am a blended top-down and bottom-up teacher. I aim to make students feel a sense of community in the classroom. Both articles by Spencer and Vacca emphasize the importance of making reading experiences personal and meaningful with an interactive approach.
Student 3	My response to my placement by the TORP survey is that I would have to agree with my placement. I believe that due to my position as a reading teacher and given I am in my 3rd year in this position, I use the bottom-up approach regularly in my assessment, instructional decisions on intervention, and materials I use.
Student 4	My score for the TORP survey is 86. This places me in the skills perspective category. I do not fully agree with my skills-based scores. Although I do believe that I try and teach with a blended approach, I lean more towards using a top down approach rather than a bottom up.
Student 5	I agree with the score of the TORP. At this point in my career I think I am a balance of both decoding and whole language. I felt like I was able to relate to the small group instruction in Tamara Spencer’s article Learning to Read in the Wake of Reform: Young Children’s Experiences with Scientifically Based Reading Curriculum. Students can make jokes and have fun while learning to read.

Interpretation of Data:
Candidates in both 2019 and 2020 were enthusiastic about being invited to reflect upon the oft-unspoken philosophies undergirding their approaches to teaching reading. They responded to the writing prompt with critical depth and self-analysis, applying their TORP results to their daily practice as literacy educators. Rather than coming away with easy answers (indeed, there was no ‘right answer’ the instructor was looking for), inquiry begat inquiry as students considered their TORP results in relation to the theories of literacy and how their beliefs impacted their teaching practices.

Of the five random data points collected from 2019 and 2020, it seems students in 2019 wrote longer responses. This may not be representative of the larger dataset. If it is, we believe it is likely on account of 2020 students working during the COVID-19 pandemic.

Data Source 3

Reading, Language, Literacy Specialist Credential Year+ Completer Pilot Survey

Perspective Captured from Data Source:
Program completer at least one year or more out of program completion

Rationale for using Data Source:
Beginning in late Spring 2021, the RLLSC Program administered a pilot survey to program completers who completed the program at least one year or more prior to Spring 2021. This pilot survey was administered to learn the perceptions of how well the program prepared them to establish goals for professional growth.

Using the RLLSC Year+ Completer Pilot Survey allows us to capture candidates’ perceptions of how well the program prepared them with content and pedagogy necessary to carry out their role as Reading Specialists/ Literacy Leaders.

Specific Elements of Data Source:

Reading Elements

Definition of Success for Each Element:
Because this is a new pilot survey we have not set a programmatic goal and are hoping to learn areas of need for our program with this survey. Eventually, our goal is for candidates to rate the program at a 4 or a 5 within each area.

Displays of Analyzed Data:

Data from May/June 2021 Data Cycle

Analyzed Data
Link to Full Dataset:

RLLSC Year+ Completer Pilot Survey (Names Redacted)

Interpretation of Data:
Because the program only began administering the Year+ Completer Pilot survey in May 2021, we only have one (partial) cycle of data to analyze. The pilot survey was sent to 56 RLLSC program completers from years 2015-2020 in early May 2021 and again in June 2021. As of July 14, 2021, five responded to the survey.

The completers who responded indicated they somewhat agreed that the program prepared to establish goals for their own professional growth and engage in self-assessment, goal setting, and reflection.

Next Steps:
In order to address what we found, we will include the TORP discussion prompt as a routine discussion/reflection assignment in LEE 230. Additionally, as the last course in the program, LEE 254 can support an assignment focused on setting professional goals, monitoring progress, and reflection which will be created and piloted in the 2021-22 academic year.

We will continue to send both the completer and Year + completer surveys out each year and as a program, analyze the data for feedback and trends. We plan to conduct focus group interviews with year+ completers and develop an employer pilot survey, both of which will be created and piloted in the 2021-22 academic year.

Aspect F →