TEAMS Evaluation 1992-1993 Executive Summary

by Carla Lane, Ed.D.

[This article has been divided into a number of separate web pages for browser-loading ease. You may view (and select) the contents by section title from the Contents below, or click on the "Next" button at the bottom of each page.]

Contents

Introduction

TEAMS Distance Learning is an Educational Telecommunications Network (ETN®) service of the Los Angeles County Office of Education (LACOE). The TEAMS Project was funded through the Star Schools Programs of the United States Department of Education, Office of Educational Research and Improvement (OERI) from 1990-92. TEAMS Distance Learning programs are currently supported by LACOE and the APOLLO 2000 Star Schools Project for 1992-94.

TEAMS provides live, interactive instructional telecasts for students in grades two through six, their teachers and parents across the country and in Canada. Programs are designed by specialists in curriculum, instruction and parent education, using input from students, parents and staffs of participating agencies. Programs are produced and telecast through a cooperative effort between the TEAMS staff and ETN®

The Three-Tier Distance Learning Staff Development Model

TEAMS focuses primarily on instructional programs for students and a Three-Tier Distance Learning Staff Development Model for teachers (Cassidy, 1990). The Three-Tier approach includes:

    1. Theoretical Training: information, theory, demonstration and two-way communication about the theoretical basis of the TEAMS instruction and training
    2. Implementation Training: theory, demonstration, practice and peer discussion of curriculum and instructional methods involved in the student programming, providing training to implement the student programs
    3. Simultaneous Teacher Training and Student Instruction: teacher training through in-class experience, practice and support from the studio team-teacher, through live, interactive student instructional programs.

Overall, this three-tiered approach answers many of the problems related to traditional staff development design. It:

    • is long term, sequential training
    • fosters immediate transfer of learning, with skills becoming a part of the teacher's repertoire of instructional methods
    • is conducted mostly in the teacher's own classroom during the school day
    • it creates immediate changes in the roles of the teacher and student
    • provides opportunities for teachers to see their own students being successful with a rich and challenging curriculum, allowing them to change their attitudes and behaviors related to instruction and expectations of their students
    • provides motivation for teachers to participate in other staff development after the regular school day because it is directly related to their classroom program

This model, developed by Sheila Cassidy for TEAMS Distance Learning, is based on research and practice in the fields of staff development and adult learning, as well as national and state standards and guidelines. The basis of the staff development research is formed by work by Joyce and Showers; Cassidy and Taira; and the Rand Corporation. The adult learning principles are summarized in work by Jones and Woodcock.

The staff development research (Joyce & Showers, 1988) provides compelling data on the relationship between training outcomes and specific training components. They analyzed the relationship between the training outcomes of knowledge, skill and transfer of training for participants engaged in training programs options providing:

    1. information
    2. theory
    3. demonstration
    4. theory and demonstration
    5. theory and practice
    6. theory, demonstration and practice
    7. theory, demonstration, practice and feedback
    8. theory, demonstration, practice, feedback and coaching

Their research clearly shows that training which provides only information and theory produces only increased knowledge in participants. That encompassing any of options numbers four through eight shows greater knowledge and skill outcomes.

Option eight provides the greatest outcomes in knowledge, skills, and transfer of training. Practice, feedback, and coaching can be considered an in-classroom, on the job, experiential and support component. With its three tiers, TEAMS provides a distance learning alternative to option eight. It clearly provides theory, demonstration and practice. Although distance learning cannot provide a full face-to-face feedback and coaching component, part of what feedback and coaching provides is an in-class support system. That is provided through the in-class team teaching with the studio instructor. In retraining of teachers, Cassidy and Taira (1988, 1989) found that teachers reported the factors which contributed to their success were: a sound theoretical basis; experience and practice with the particular curriculum and instruction being adopted/adapted; a support system designed specifically to their needs; convenience, with training during the school day and at their own site when possible; training with no expense to the teacher. The simultaneous in-class training component of TEAMS meets all of these criteria.

The Rand Corporation found that successful projects had these common characteristics for staff development (Berman and McLaughlin, 1978):

    1. training is concrete, continual, and tied to the world of the teacher
    2. local resource personnel provide direct follow-up assistance
    3. peer observation and discussion provide teachers with reinforcement and encouragement
    4. school leader participates in staff development
    5. regular meeting held with teachers for problem solving and adapting techniques and skills of the innovation
    6. released time used for teacher staff development
    7. staff development planned with teachers prior to and during the project

Cassidy (1985) reviewed programs with findings similar to the Rand study but with additional information.

    1. individualized staff development activities are more effective than large-group activities
    2. programs incorporating demonstrations, trials, and feedback of ideas are more effective than lecturing and reading of ideas
    3. staff development programs are more successful when teachers are active planners and help each other.

Jones and Woodcock (1984) describe these adult learning principles:

    1. the adult is a partner with the instructor in the learning process
    2. adults are capable of taking responsibility for their own learning
    3. adult learners gain through two-way communications
    4. adults learn through reflection on their and others' experience
    5. adults learn what they perceive to be useful in their life situations
    6. adults' attention spans are a function of their interest in the experience
    7. adults are most receptive to instruction that is clearly related to problems they face daily
    8. adults learn best when they are treated with respect
    9. adults do not typically se themselves as learners
    10. adults learn better in a climate that is informal and personal
    11. adult learners apply learnings that they have been influential in planning
    12. adults learn when they feel supported in experimenting with new ideas and skills
    13. adults are likely to have somewhat fixed points of view that make them closed to new ways of thinking and behaving
    14. adults learn to react to the differential status of members of the group
    15. adults are internally motivated to develop increased effectiveness
    16. adults filter their learning through their value system

TEAMS Evaluation

The TEAMS (APOLLO 2000) Evaluation was designed to be conducted over a two year period.The evaluation years are 1992, 1993, and 1994. The 1992-93 Year One evaluation collected broad-based data, and it provided input from TEAMS Regional Coordinators at the New Orleans Advisory Committee Meeting (January 1993); data from teachers and students obtained through school focus group sessions conducted at Arizona, Boston, Detroit, Los Angeles, Los Angeles County, Missouri, Orange County, Utah, and Washington, D.C. (March-May 1993); and data from surveys completed by TEAMS mathematics and science teachers, TEAMS school principals, site coordinators and parents, as well as TEAMS Regional Coordinators.

1992-1993 Research Design

The 1992-1993 research design is based on the PEP (Program Effective Panel) recommendations of OERI.

Claim Type 2 focuses on the change in teacher's attitudes and behaviors to improve the teaching process. This requires demonstration of changes in attitudes or behaviors, and presentation of a reasonable link between the results and an educationally important goal.

Claim Type 2 was used as there was access to this information through surveys of teachers and principals. The original evaluation design was to correlate responses from all respondents in order to substantiate the findings. However, the number of responses were too limited to allow a full correlation analysis. Instead, this material will be evaluated and compared with the data that is currently being gathered for the 1993-94 TEAMS year.

Claim Type 3 focuses on changes in students' attitudes and behaviors that in the long term lead to educationally desirable outcomes. Use of this claim requires data showing positive change in the target group and strong logical or empirical evidence that this change is large enough to be educationally meaningful.

Through surveys of teachers, principals, RTCs, student focus groups and observations

TEAMS had access to data supporting Claim Types 2 and 3. Timelines & Data Collection Procedures Survey Instruments were sent to six groups (math and science teachers, principals, TEAMS site coordinators and RTC coordinators, and parents of TEAMS students) from March - May 1993. Because some schools closed for the summer, a second round of surveys were sent to all schools in September 1993. Data collection was terminated on December 27, 1993.

Empirical data were collected and descriptive statistics were generated using Statview 4.1. The full report provides descriptive statistics for all empirically based questions as well as qualitative information. Focus group interviews tapes were transcribed and will be compared with the focus group interviews conducted in 1994 to detect patterns of behavior and use. Respondents

There were 334 respondents to the surveys. The mean age of those responding to this question was 44 (135 did not respond). There were 209 females and 65 males responding (61 did not respond to this question). The following (Table 1) is the breakdown of the 334 respondents:

Table 1.

Respondents
Principals

40
RTC Coordinators

33
Math Teachers 104
Science Teachers

99
Parents

58
Total 334

Of the educators responding, 13 respondents hold terminal degrees and 151 hold master's degree. Only 22 reported that this was their first year of teaching; 62 had been in education 2-5 years; 70 for 6-10 years; 49 for 11-19 years; and 38 reported being in teaching for over 20 years. The changes in the instructional methods used by teachers because of TEAMS is even more unusual when viewed against the number of years taught. Sixty-six percent (157) have taught over five years and 16 percent have taught over 20 years. (See Table 2)

Table 2.

Respondents'
Years of Teaching
1 year

22
2-5 years

62
6-10 years

70
11-19 years

49
over 20

38
Total 238

 


Next