QAR 2: Conclusion

Overall Findings & Analysis form Self-Study
As documented throughout this Quality Assurance Report, we viewed this first AAQEP self-study as an opportunity to establish a baseline portrait of our advanced credential programs using the AAQEP Standards as a framework. Our belief is that establishing an accurate baseline is critical to our ability to engage in authentic continuous improvement work as we move forward. As can be expected, our findings from our analyses allowed us the opportunity to see areas where our programs--and we as an institution--excel, as well as to identify specific areas for growth.

Areas of Strength
In particular, our findings demonstrated to us that each of our advanced credential programs--the Reading/Literacy Specialist, Preliminary Administrative Services, School Counseling, and School Nursing--are providing the appropriate coursework and fieldwork experiences to prepare our program completers to excel in their future roles. We found our programs provide the necessary coursework to ensure our completers develop the necessary knowledge and skills to be able to support all learners in their development and to create positive and productive learning environments. In particular, we saw that our programs excel in preparing our candidates to engage in culturally responsive practices, which affirms our School’s mission to prepare education leaders to serve in diverse communities. In order to come to these conclusions, program faculty analyzed data from a variety of sources, including candidate performance on key assignments, performance-based assessments, field experience supervisor evaluations, and surveys of completers, alumni, and employers. 

We believe there are multiple reasons why our completers are well prepared, and our findings confirmed these beliefs for us. To begin, our advanced credential programs have a long history of being fully accredited by the California Commission on Teacher Credentialing, meaning programs’ practices align with specific program standards. These program standards are updated regularly to respond to shifts in the demographics of our state’s learners and new findings about best practices from educational research. In response, our programs engage in a regular cycle of revisions. Additionally, as demonstrated in our findings, our programs make an effort to maintain strong, collaborative partnerships with local school sites and districts that allow our candidates to have positive and meaningful field experiences in diverse contexts. These field experiences are critical to allowing our candidates to apply their learning from their coursework in local contexts, especially as the majority of our completers remain in the region. Finally, faculty members in all programs maintain active connections with educators throughout the region, which allows them to stay in touch with challenges faced by individuals in the field. They are then able to use this learning to inform their program practices, ensuring that our completers are prepared to meet these challenges.

Our findings also highlighted that candidates and completers of our programs come from a range of racial and cultural backgrounds that are largely representative of the learners in our context. As documented in responses to Standard 3, the majority of learners in our region identify as Hispanic/Latinx. Similarly, in our Preliminary Administrative Services, Reading/Literacy Specialist, and School Counseling programs, the majority of our candidates identify as Hispanic/Latinx. The demographics of candidates in our School Nursing program are different, but it is worth noting that this program draws candidates from throughout the state who then serve learners throughout the state. Once candidates enroll in our programs, our programs work to realize the notion of “high challenge, high support.” Our QAR responses highlight the fact that program faculty provide a number of supports to ensure their success, while still maintaining a high level of rigor. We take pride in the high percentages of diverse candidates that successfully complete our programs annually.

Another strength that arose out of our work in preparing this QAR was the opportunity to forge new relationships both with program faculty of the advanced credential programs houses in Kremen and with program faculty in the School Nursing Credential program, which is housed jointly in the College of Health and Human Services and the Division of Continuing and Global Education. As we have worked together to prepare responses to the AAQEP Standards, we have gained a deeper understanding and appreciation of the work each program does, which helps to build a strong foundation for ongoing inquiries into how to better support our candidates. In addition to the advanced credential programs included in this QAR, Kremen serves as the educational unit for additional programs that offer credentials accredited by the CCTC that receive national accreditation through other accrediting bodies. Even though those programs were not apart of this process, the discussions we had with their program faculty and college administrators led to fruitful discussions we believe will pave the way towards future collaborations and joint inquiries into new ways to improve our programs.

Finally, as discussed in the responses to Standard 4, in the work they do, our programs absolutely reflect the mission of our university and our school. The university mission is “to boldly educate and empower students for success,” while the vision is that students will be prepared to become our next generation of leaders. Similarly, the Kremen School of Education and Human Development's mission is the recruitment and development of ethically informed leaders for classroom teaching, education administration, counseling, and higher education. As documented throughout this QAR, our programs carry out both missions and the university’s vision as we actively recruit and prepare our advanced credential candidates to take on leadership roles in the P20 education system of the region and state, demonstrating the deep commitments of our program faculty. 

Areas for Growth
In addition to providing confirmation of the quality of our advanced credential programs, engaging in this self-study also demonstrated clear areas where our programs and our educational unit as a whole have room to grow and improve, areas that we can build on in our ongoing efforts towards continuous improvement.

New Systems to Collect and Analyze Data. One of our biggest take-aways from this process is our need to create new systems for the ways in which we collect and analyze data, in addition to our need to make changes to what data we actually collect. As documented in our responses to Standards 1 and 2, we realized that we do not have a unit-wide systematic approach to collecting data from any of our key stakeholder groups--completers, K-12 partners, employers. While together the CSU Educator Quality Center and the California Commission on Teacher Credentialing administer surveys to program completers, employers, and year-out professionals, we discovered that their measures do not always align with the analyses we were trying to do in response to the Standard 1 aspects. Another challenge to the data collected from these large-scale surveys is that, in many cases, we were unable to disaggregate the data in a way that made the findings meaningful to us. For many of our advanced credential programs, the number of completers was too small. As a result, for this QAR, we were not always able to capture the perspectives of each key stakeholder group. More often than not, we relied on the perspectives of our faculty and our candidates, primarily through the use of coursework.

Moving forward, we intend to develop unit-wide surveys that can be administered annually to each stakeholder group that will include both general items about the work our institution does as a whole and program-specific items. The hope is that this will allow us to collect data that will be useful at both levels but that will not lead to survey-fatigue from administering too many surveys, which is already a concern given the administration of both the CCTC and CSU Educator Quality Center surveys. During the 2021-2022 academic year, we intend to develop tools that we can pilot in Spring 2022, working with program faculty to develop items that are meaningful both unit-wide and programmatically and that will support our inquiries into program improvement. Once we have responses, we can begin the process of analyzing the data we collect to ensure the reliability of the tools and then making the necessary revisions. Our hope is that we will create tools that we can use on an ongoing basis to provide meaningful information about the work we do, information we can then use as we continue to engage in our ongoing cycles of continuous improvement.

More Meaningful Recruitment Efforts. Related to the ways in which we collect and analyze data, another area we realized we need to improve is in tracking our candidates--from the point when they are first interested in the program, through their application, to their eventual enrollment, through their ultimate completion of the program, and into their respective roles as educational leaders. While programs do hold a number of recruitment events, often attendance at those events is not recorded, so we have no way of knowing who attended. As an educational unit, we are currently working with our Communications Coordinator to create a system that will allow us to more strategically collect data at recruitment events and then track who from each event applies to, is admitted to, and enrolls in our programs. At the program level, we then want to analyze this dataset on an annual basis in order to be more strategic in our recruitment efforts. We also want to continue to ensure that all who are admitted to our programs successfully complete those programs. Related to the topic of recruitment, while our programs do pride themselves on the diversity of candidates who enroll, we realize we could be doing more, particularly in the area of recruiting candidates for our advanced credential programs who currently work in under-resourced areas. This will be an area of focus for us moving forward.

Engaging in Quality Assurance. Another key finding of all programs was the value of engaging in an internal audit as a way to evaluate the work of the program. Again, this was the first time programs had undertaken this process, and it led to authentic findings about where programs were excelling and where they might be able to do more to better support candidates. Moving forward, programs intend to formalize their processes and engage in this process on an annual basis, with a plan to review findings from the audit at program meetings. These findings can then be used to support ongoing continuous improvement into program practices.

As we begin to rethink the data sources we use to evaluate our program practices, we recognize that we also need to be investigating the reliability, validity, trustworthiness, and fairness of the instruments we use throughout programs on an ongoing basis. In future years, program faculty will continue to engage in these investigations, working together to evaluate tools such as surveys or focus group protocols; ensuring the validity of course assignments and assessments and field experience evaluations; and analyzing student work in response to key assignments across different sections of courses. As a unit, we intend to support these efforts by making data source evaluation a focus of future Data Summits.

Supporting Transitions. Our findings also demonstrated that, while our work preparing our candidates during their time in our programs is strong, we do little to inquire into and support their transition into their new roles and to provide ongoing professional development. Providing better support to our alumni would also allow us to have a broader reach within our service area. In order to begin to do this well, we need to extend our data collection beyond the point of program completion into our completers’ time of employment. As we move forward, we intend to create a system to collect data from our completers that allows us to learn more specifically about where they are placed when they leave our program and then to follow up with them to learn about successes and challenges they experience in those placements. We envision this beginning with our internal survey that we administer to completers at the time they leave our programs that will collect contact information along with details about their places of employment. Beyond that, we intend to follow up with them on an annual basis in order to continually update our database. Within the next year, we intend to pilot these instruments to see how effective they are in gathering the data we are seeking. Once we do so, we will evaluate that effectiveness and then make any necessary revisions in order to ensure that we gather the information that is most meaningful to us. That said, our hope is that this process will be mutually beneficial for both us and our completers. As we learn about particular challenges our advanced credential candidates are facing in their new roles, our intent is to be able to provide professional development opportunities to address these challenges, while also making changes to our program practices so that future completers will not face the same challenges.

Stronger P12 Collaborations. Moving forward, as highlighted in Standard 4, our programs also need to be more strategic in the creation and use of advisory boards to inform program practices. While many of the program faculty are actively engaged in community organizations related to their field and/or the local P12 community, most do not have their own advisory boards to help shape the work of the program. Bringing individuals from the community together to discuss program practices and to analyze program data is a critical next step to improving the work we do to prepare qualified candidates. We envision that these partnerships will be mutually beneficial in that we can share data from our programs to get input and guidance, and our P12 partners can do the same. Our ultimate goal is to work together to ensure that we truly are empowering our candidates to become the next generation of ethically informed educational leaders in our region and our state.

Top of Page