Standard 17: Implementation of the Teaching Performance Assessment (TPA):
Program Administration Processes
The TPA is implemented according to the requirements of the Commission-approved model
selected by the program. One or more individuals responsible for implementing the
TPA document the administration, scoring, and data reporting processes for all tasks/activities
of the applicable TPA model in accordance with the requirements of the selected model.
The program adopts a passing score standard and provides a rationale for establishing
that passing standard.
The California Commission on Teacher Credentialing approved the Fresno Assessment
of Student Teachers (FAST) as a locally designed alternative performance assessment.
The CCTC-approved FAST is in compliance with SB2042 Standards of Quality and Effectiveness
for Professional Teacher Preparation Programs. The assessment design includes four
complex pedagogical tasks, each accompanied by a multi-dimensional scoring scale,
to evaluate the thirteen Teacher Performance Expectations (TPEs). Each FAST task
is embedded into a specific field experience within a sequential credential program.This helps assure that TPE-related knowledge and skills have been taught and formatively
assessed in coursework, and applied with feedback in fieldwork situations prior to
being summatively assessed through FAST. [ See Multiple Subject Overview Matrix.] (See FAST Manual.)
The successful implementation of the FAST system is ultimately the responsibility
of the Dean of the Kremen School of Education and Human Development (KSOEHD) in his
role as Director of Teacher Education. However, the system’s administration, scoring,
and data reporting processes are assigned part-time to two FAST System Co-Coordinators
who are responsible for the day-to-day operations of the system. Both compensated,
one coordinator takes responsibility for administration and scoring while the other
provides expert data analysis and reporting. A staff member is assigned part-time
to provide logistical and data-entry support.
The activities of the Co-Coordinators are overseen by the Assessment Committee whose
members include the Associate Dean of the KSOEHD, Single and Multiple Subject Credential
Program Coordinators, the Directors of Field Placement, and faculty and university
supervisor representatives with expertise in data collection and analysis. The committee
meets at least annually to address implementation issues, to review the results of
data analysis, and to recommend a plan to disseminate candidate performance data,
including FAST data, for the purposes of program improvement.
The original FAST tasks, rubrics, and passing standards have been maintained since
their initial CCTC approval. Each of the four tasks has a project-specific, four-point
scoring rubric used to evaluate the project. (See FAST Manual.)
The rubric levels are:
1 = Does not meet expectations
2 = Meets expectations
3 = Meets expectations at a high level
4 = Exceeds expectations
In three of the four tasks, a score is generated for each separate TPE evaluated by that task.
In the fourth task, the Teaching Sample Project, a score is generated for each of
seven TPE-related sections. This discreet level of scoring enhances the specificity
of candidate evaluation and of program improvement efforts. (See FAST Manual.)
The FAST system is comprised of four tasks. Each task is designed to evaluate multiple Teacher Performance Expectations (TPEs) (SEE FAST Matrix, FAST Manual). For three of the four tasks, each TPE is scored independently on a task-specific rubric (See Comprehensive Lesson Plan Project Rubric, FAST Manual ; Site Visitation Project Rubric, FAST Manual; Holistic Proficiency Project Rubric, FAST Manual). The Teaching Sample Project is scored by section with each section representing one or more TPEs (See Teaching Sample Project Rubric, FAST Manual). All rubrics use a 1 to 4 point designation. As stated on the first row of each rubric, a score of “1” is consistently designated as “Does not meet expectations.” A score of “2” is consistently designated as “Meets expectations.” A score of “3” is consistently designated as “Meets expectations at a high level.” A score of “4” is consistently designated as “Exceeds expectations.” As stated in the FAST Manual “… you must earn a minimum score of “2” on each TPE evaluated in a project (or, in the case of the Teaching Sample Project, a minimum of score of “2” on each section) in order to receive credit for the fieldwork course in which the assessment is required.” (See FAST Manual).
FAST projects or tasks were specifically designed to provide scores by TPE while still meeting the requirement for task complexity. This discreet scoring method has clear advantages in planning for program improvement and in counseling students with regard to areas of strength and weakness. We only consider disaggregated TPE-specific scores at the student, program, and school levels.
The only instance in which holistic scores are used is for reporting annual scores to the CCTC (Their state-level data base design would not allow for 28 scores to be submitted for each student). Subsequently, for CCTC-reporting purposes, we report students’ average scores for each task with the caveat that in order to pass, no individual TPE score shall be less than “2.” We also provide a “total final score” to CCTC that is calculated by adding the average scores of the four tasks. The passing total equals “8,” again with the footnote that no individual TPE score generated on any individual task can be less than a “2.”
The rationale for the passing score of “2” on individual TPEs is reflective of the
CCTC’s qualitative expectations for mastery of the TPE by a beginning teacher. The
task score assumes a passing score of “2” on all TPEs (sections) evaluated by the
FAST task, and a total final score reflects the passing of all TPEs on all four tasks.
The program maintains both program level and candidate level TPA data, including but
not limited to individual and aggregated results of candidate performance, assessor
calibration status, and assessor performance over time. The program documents the
use of these data not only for Commission reporting and/or accreditation purposes,
but also for program improvement. The program assures that candidates understand the
appropriate use of their performance data as well as privacy considerations relating
to candidate data.
All teacher candidates are required to subscribe to TaskStream, an electronic portfolio
system, as part of their fieldwork.[ See FAST Manual and EHD174 syllabus.] Candidates’ scores on all FAST performance assessments are maintained on that system
allowing students access through their personal Directed Response Folio. Besides
being accessible by the candidate him/herself, access to this information is restricted
to the candidate’s current supervisor, the TaskStream Manager/FAST Co-Coordinator,
and the staff person responsible for data entry.
At the program level, an Excel database is maintained for Multiple & Single Subject Credential Programs by a designated staff person. This database includes:
- All CCTC-required demographic data on each teacher candidate,
- Both TPE- or section-specific scores as well as the holistic score earned on each assessment (including the scores for any re-takes),
- The name of the scorer(s),
- Any double-scores that were generated from that candidate’s performance assessments, and
- A total score at the point of completion of the four FAST tasks.
These data are used for the purposes of individual candidate evaluation, for program
improvement, to plot scorer performance over time, and to meet annual CCTC reporting
requirements.
The FAST Coordinator maintains a separate scorer database. The database contains
a list of scorers who have been trained to score each of the four tasks along with
the date of their most recent training and calibration. The FAST coordinator compares
the list each semester to those who are eligible or required to score each task based
on their University assignment to determine who must attend a formal scorer training
prior to scoring that particular task.
FAST is an important source of student performance data used by the KSOEHD for program
improvement. Coupled with exit and graduate survey data and anecdotal data from faculty
and supervisors, FAST data informs practice at the course level. Because each course
is responsible for certain TPEs and scores are aggregated by TPE, programs are able
to evaluate candidates’ performance relative to their designated TPE responsibilities
and to recommend improvement at the delivery level.
For example, several years ago FAST scores for TPE 7, Teaching English Learners, were
aggregately lower than desired. When results were triangulated with those Graduate
and Supervisor Surveys generated by the CSU Chancellor’s Office where many graduates
after one year of employment and their supervisors indicated that the graduate had
not been adequately prepared by Fresno State to teach English Learners, program improvement
efforts were clearly warranted. In response the Multiple Subject Credential Program
developed a program-specific plan to address the instruction of English learners;
English learner faculty amended the Lesson Observation tool used in student teaching
to reinforce candidates’ use of the Sheltered Instruction Observation Protocol (SIOP),
and a program-wide faculty commitment was made to emphasize EL teaching strategies
in all classes and in all fieldwork experiences. As was reported in our Biennial
Report to the CCTC in September 2010 teacher candidates’ performances on TPE 7 has
improved in the past year. Confirmation or the stability of improvement will not
be available until those same candidates and their professional supervisors are surveyed
as graduates at the end of the 2011-2012 academic year and reported in late fall,
2012.
Teacher candidates are informed of the use of their performance data as well as privacy
considerations relating to candidate data through the Intended Use Policy printed
in the FAST Manual that is available to every candidate electronically through TaskStream.
The policy, along with all others dealing with FAST is reviewed by the FAST Coordinator
or by the Director of Field Placement at program orientations at the onset of candidates’
teacher preparation program. (See FAST Manual.)
The program establishes and consistently uses appropriate measures to ensure the security
of all TPA materials, including all print, online, video candidate, and assessor materials.
The program also consistently uses appropriate measures and maintains documentation
to assure the privacy of the candidate, the K-12 students, the school site and school
district, and other adults involved in the TPA process.
The FAST system was designed with security in mind. In the Multiple Subject Program
two of the four tasks are submitted on TaskStream for scoring. Only the candidate
and the scorer see the candidate’s response to the task. The other two assessments
are scored in a controlled setting after which the scores are entered on TaskStream.
In one case the candidate’s work is attached to the TaskStream folio while in another
case, the work is returned to the candidate.
Assessor materials are held by the FAST Coordinator and are only used in scoring and in training and calibration sessions. All materials are collected after use. They are not available electronically. Admonitions for candidate confidentiality are built into every scorer-training module. Because all scorers are either employees of California State University, Fresno, or partnering K-12 school districts, such admonitions are recognized as serious and compelling.