of the Star Schools Projects
[This report has been divided into a number
of separate web pages for browser-loading ease. You may view
(and select) the contents by section title from the Contents
below, or click on the "Next" button at the bottom
of each page.]
The following is excerpted from the Star
Schools Projects Report which was carried out by the Southwest
Regional Laboratory (SWRL) and Abt Associates, Inc. (AAI). SWRL,
4665 Lampson Avenue, Los Alamitos, CA 90720 310-598-7661.
Carla Lane, Ed.D. Project Director
Naida C. Tushnet
Christina Bodinger-de Uriarte
David van Broekhuizen
for Abt Associates Inc.
Mary Ann Millsap
The Office of Educational Research and
Improvement (OERI) of the U.S. Department of Education has provided
funding for three successive cycles of two-year Star Schools
projects. At this stage in the development of the new communication-delivery
technology, it is appropriate to assess the early efforts supported
by federal funds.
The evaluation focuses on a series of questions
posed in the authorization of Star Schools. The following questions
address issues related to project organization and impact on
students, staff, and schools.
- How effective are telecommunication partnerships
programs and services after federal funding ceases?
- To what extend did projects use nonfederal
funds? What were the sources of such funds?
- What activities were supported by grantees?
- Student programming
- Staff development
- How many students participate in Star
Schools-sponsored activities? What are their demographic characteristics?
Are there differences in services offered to economically and
educationally disadvantaged and minority children? Are there
differences in effectiveness of programming and services?
- How effective were Star Schools courses?
Are differences in effectiveness related to differences in course
- How were studio teachers selected? What
support did they receive?
- How were the staff members responsible
for distance learning at the school site selected? What support
did they receive.
- What staff development programs were offered?
To how many teachers? With what effects? How much time was spent
in staff development?
- What were the socioeconomic and geographic
characteristics of participating schools? What types of programming
were offered to different schools?
- What were the effects of distance learning
on curriculum and staff patterns at participating schools?
The evaluation focused on two aspects of
the context for the Star Schools Program. First the federal context
influences the organization of the Star Schools projects as well
as the services they provide. The federal context includes not
only the legislation but changes in priorities For example, when
the authorizing legislation was passed in 1988, the National
Education Goals had not yet been formulated. They are now significant
objectives for Star Schools activities. One consequence is that
science and mathematics programs developed during the first cycle
of funding have a somewhat different content and focus from those
Legislative requirements form the second
aspect of the federal context. Some of these have not changed
from the inception of the program. For example, the law mandates
that projects be multistate consortia and have at least three
partners. These requirements had an impact on the way projects
were organized and the issues and problems they faced. In addition,
the requirements affected the functioning of distance learning
activities in the funded organizations after federal funding
ceased. The multistate consortium requirement meant that projects
confronted issues of teacher certification, as well as coursework
and testing requirements, across state jurisdictions. Funded
projects addressed these issues in a variety of ways that involved
operations, series delivery, and the targeting of activities.
In addition, the statute requires at least one half of the schools
served to be Chapter 1 schools. (Chapter 1 is the single largest
federal elementary and secondary education program. It is designed
to provide supplemental services to low-achieving students in
low income schools.)
The second set of contextual issues comprises
state and local characteristics. These issues had an impact on
the effectiveness of Star Schools projects in several ways. First,
some state regulations, particularly those that raised high school
graduation on state university entrance requirements, helped
create a market for distance learning projects. Isolated rural
schools employ distance learning technologies to ensure opportunities
for their students and to comply with state regulations. Second
state and local regulation created barriers to the effectiveness
of distance learning technologies. For example, different requirements
for certification in a subject area in different states and regulations
or contract agreements that require the presence of a teacher
certified in the subject being taught in the classroom on occasion
precluded participation in Star Schools activities. At the minimum,
certification issues required projects to develop agreements
with a number of states.
Local contextual issues include demographic
and geographic information. Perhaps more important, however,
the local context provides academic and other experiences for
students and teachers that influence the effectiveness of distance
learning programs. For example, the local context provides students
with prior opportunities to learn. Students who come to advanced
physics classes, for example, well- prepared in trigonometry
and calculus, are more likely to succeed than students who take
physics with weaker preparation. Local context also includes
the qualification of teachers who prepare students and those
who support distance learning activities.
The effectiveness of Star Schools activities
that are integrated into existing coursework or are designed
to improve teaching are particularly influenced by local context.
Student learning in courses that combine school-site and distance
learning opportunities,, for example, is likely to vary depending
on the quality of teacher implementation of curriculum and instruction.
Similarly, the effectiveness of activities aimed at instructional
improvement will be influenced by improvement efforts at the
local site as well as other staff development opportunities available
The contextual issues also influence the
organization and service offerings of the Star Schools projects.
Of particular interest within the conceptual framework are the
project-focused, staff-focused, student-focused, and school-focused
questions raised by Congress and OERI. The framework provides
a method for organizing descriptive information about such matters
as the number of students served, the types of services they
receive and demographic and geographic distribution of services
in a manner that facilitates analyzing relationships between
those matters and the context and condition under which services
are received. It also enhances analysis of the relationship of
the objectives of particular activities to the distribution of
services and to their effectiveness in achieving their own goals.
Various configurations of Star School services
affect students' access to learning opportunities. Indicators
of access are curse offerings (the number and content), student
enrollment, and student access to quality teaching.
Finally, the outcomes are student learning
and improved instruction.
The evaluation has two phases, each with
a distinctive methodology. Phase 1, from which this report results,
relies primarily on qualitative data collection procedures. Phase
2, which began in fall 1993, combines survey research methods
with case study research.
Phase 1 was designed to yield defensible
understandings of the existing Star Schools projects. It drew
on the following data sources:
- a literature review encompassing research
findings from distance learning related studies and of alternative
approaches to achieving the objectives addressed through Star
- site visits to Star Schools projects and
schools and to other distance learning projects; and
- project reports.
Project agendas and site protocols, adapted
from procedures described by Miles and Huberman (1984), provide
a means of integrating the data from the various sources. Project
reports were analyzed using a content analysis procedures that
provided quick retrieval of project-generated information about
each element in the conceptual framework (Marshall & Rossman,
1989). The information was then included in project agendas,
which, in turn were used to generate site protocols. The site
protocols identified the individuals to interview and events
to observe during the site visits to Star Schools projects and
schools. One use was to highlight "missing" information
that must be gathered on site. Another was to indicate the questions
that probed more deeply into project activities organization,
and impact than was possible from written documents. During the
analysis phase, the project agendas provided a means of integrating
qualitative data collected on site and quantitative data gleaned
from project documents.
The first phase of the study involved data
triangulation; that is, collecting information from multiple
sources using multiple methods. Staff collected descriptions
of activities from project-generated documents; interviews with
project staff, staff from participating institutions, and the
recipients of services; and observations of the activities.
Two person teams conducted the site visits.
They spent at least four days on site, two at project headquarters
and one at each school. The sites visits had three purposes:
- They provided the opportunity to confirm
and extend the information gleaned from project documents.
- They yielded information that addresses
the evaluation questions associated with project organization
perceived effects of federal requirements, and actual school-level
experiences with Star Schools The site visit information was
particularly important in addressing questions about the effectiveness
of telecommunications partnerships programs and services after
federal funding ceased.
- The site visits served to focus Phase
2 data collection and analysis plans.
In addition to interviews with project
staff and teachers and administrators at school sites, staff
observed Star Schools classes. The major purpose of the observations
was to gain insights into the degree to which students have the
opportunity to interact with one another and with the distance
learning teacher, student responses, and the role of the on-site
The information collected on site was synthesized
by the site-visit team following its visit. Team members reread
interview notes, notes from observations and project documents,
coding information according to the conceptual framework. They
prepared an interpretive summary of their findings, which served
two purposes. First, it provided a concise statement of the progress
and problems of particular Star Schools project. Second it contained
tentative analyses that explained the status of the project.
In analytic meetings site visitors reviewed the interpretive
summaries to develop what Yin (1981 calls "causal arguments"
both within and across cases. The causal arguments were used
to identify the existence of phenomena in more than one case
under predictable conditions. For example, a preliminary analysis
revealed that the use of taped, as contrasted with live, broadcasts
was related to the structured broadcast schedules, which met
project needs to provide regular service, but conflicted with
in-school schedules. The causal argument is that the combination
of rigid technologies and organizational inertia leads to creating
flexible use of programming. The analytic meetings followed procedures
recommended by Miles and Huberman (1984), which culminate in
conclusions about Star Schools and its activities. These conclusions
were then framed in terms of the literature in completing this
preliminary interim report.
Limitations of the Report
This report accurately reflects current
information about the Star Schools Program It provides descriptive
information about project operation the numbers and types of
students and schools reached, and the role of the on-site facilitator
and teacher. Equally important, the report contains information
about how Star Schools projects spent federal funds and how funds
from other sources were used to extend the implementation of
the activities. It also provides policy-relevant information
about how Star Schools distance learning is used at school sites,
the ways such programs are combined with others at some sits,
the perceived value to students and schools, how programs are
combined with others at some sites, the perceived value to students
and schools, how federal and state context influence the use
of distance learning at the school level, and reported effects
on the organization of curriculum and instruction. In addition,
the report includes recommendations for policy and future study
well-grounded in the experience of both providers and recipients
of Star Schools services. The report also includes an evaluation
of the relationship of federal requirements to the operation
of the program and recommendations for potential changes in how
the program is organized. In sum, this first year of a two-tear
study provides useful knowledge to policy makers as they consider
the value of the Star Schools Assistance Program and ways to
enhance its positive effects.
At the same time, the report has limitations,
all stemming from its timing. First although site visits were
made to all Cycle One and Two projects, at least two Cycle One
projects kept few records of the Star School years, and one was
no longer providing distance learning opportunities to schools.
Consequently, information about those projects is more limited
than information about others. Also, the remaining projects had
adjusted programming to accommodate the end of federal funding.
While such adjustment constitute important information included
in this report, they also limit the evaluators' ability to see
Star Schools as it originally operated. Second, the report relies
heavily on project-generated evaluations, which varied greatly
in quality and perceptions of Star Schools project staff and
teachers who received services. Consequently, although the report
provides information about probable impacts on teachers, schools
and students, it will be important to gather independent data
about those impacts. It is equally important to gather data relating
impacts to federal and state contexts, project organization and
processes, and the type of distance learning activities. These
will be pursued during Phase 2.
Finally the report includes some speculation
about how the impact of Star Schools distance learning can be
enhanced. Phase 2 will explore these speculations more systematically.
Despite its limitations, the report has
many values. It contains clear descriptive information, documented
associations, and grounded policy recommendations. Although an
interim report, it goes farther than previous research to document
and evaluate Star Schools distance learning, its possibilities