Evaluation of the Star Schools Projects
[This report has been divided into a number of separate web pages for browser-loading ease. You may view (and select) the contents by section title from the Contents, or click on the "Next" button at the bottom of each page.]
Conclusions and Recommendations
Findings Related to Congressional Questions
1. How effective are telecommunications partnerships and programs after federal funding ceases?
Three projects funded by the Star Schools Program currently receive no Star Schools funding. Two offer primarily whole-course programs and are dependent on enrollment for continuation. One has sufficient demand, although not for the courses developed with Star Schools funds, to continue a strong presence in the distance learning community; the other project has shrunk considerably. The third project, which relied on computer-based technologies and developed materials to supplement the curriculum, no longer provides distance learning activities. However the computer-based curriculum materials are used by an other project, and the organization developed a more advanced computer networking program a result of the problems experienced in Star Schools. In all three instances, the partnerships did not survive the loss of funding.
Currently, funded Star Schools projects are working toward developing a diversified funding base so they could continue if Star Schools funds were no longer available to them. The most common form of diversification is requiring dues from participating states and school districts. Projects also actively seek additional grants and contracts. Although one project encourages schools to use federal and state funds to support Star Schools activities, project staff members express concern about the longevity of such funding.
2. What non-federal funds are leveraged by Star Schools projects? How stable is such funding?
Star Schools projects derive funds from a variety of sources. The most stable sources seem to be dues paid at the state level. When projects rely on local school or district contributions, they are quite susceptible to changes in the fiscal environment. Most projects actively seek funds from federal agencies in addition to the OERI, and in fact, most non-Star Schools funding comes from other federal sources and is short term in nature.
3. How did Star Schools grantees spend funds generated by the Star Schools Assistance Program?
Across the projects from which information was obtained, the largest portion of funds (35 percent) was spent on equipment. Fiscal agents distributed an average of 28 percent of their grants to partners, including state, local, and regional educational agencies, institutions of higher education , not-for-profit organizations, and profit-making companies. On average, 21 percent of funds were spent to develop and disseminate programming, and 17 percent was spent on administration.
The investment in equipment helped build an infrastructure for distance learning activities. Consequently, it might be possible to devote more funds to programming than to equipment.
4. What subject matter was offered through Star Schools/ With what effectiveness? What student learning outcomes occurred?
Star Schools projects offered two types of programs. The first were whole courses, mainly in high school subject. The second were supplementary courses or modules offered at all levels of elementary and secondary school.
Foreign languages (including French, German, Japanese, Latin, Russian, and Spanish) comprise the largest number of full-course offerings (31), and 11,630 students currently are enrolled in distance learning foreign language courses. Science course serve 1,552 students; and mathematics courses, 939 students. Over 1,583 students currently are enrolled in AP courses, including math and science, through Star Schools distance learning.
Projects do not keep good data on enrollment in modules and supplementary courses. However, a conservative estimate is that a total of 140,000 students currently are involved in Star Schools distance learning activities. This report relies on data gathered by projects, which were not consistent in how they assessed effectiveness. What little evidence, from two projects, exists seems to indicate that Star Schools distance learning is as effective as other means of instruction. Further, it provides opportunities for students particularly in rural areas, that would otherwise not exist.
5. What general staff development was offered by Star Schools projects? How effective was it?
General staff development was perhaps the weakest component of Star Schools projects. During the 1992-93 academic year, Star Schools projects sponsored 130 in-service "events," ranging from one-half hour workshops to training programs lasting 10 sessions. However, a few projects collected data about participation, and none collected data on effects. Most projects believed the general staff development activities were severely under used.
Two Star Schools project identified staff development as the focal point of their activities. In them, programs and materials delivered to students were designed to change teachers'[ attitudes and behaviors in teaching science, and in one case, math as well. The purpose was to demonstrate hands-on, constructivist approaches to instruction and provide materials that supported such pedagogy. One project included extensive staff development, including in-person workshops, one-week intensive seminars, and teleconferences. The project also provided substantial on-site technical assistance. Teachers who participated in the project did not change their attitudes as many came with a philosophical commitment in line with project goals. They did, however, report changes in how they actually worked with students and the materials they used. The other project provided fewer supports. Nonetheless, teachers reported changes in their own attitudes and behavior.
6. How did issues related to teacher certification affect Star Schools projects?
In general, issues related to teacher certification were not salient in Star Schools projects. The issue only arose in full-course programs. Most projects used certified teachers as studio teachers and worked with participating states to gain certification for them. One project relied heavily on university professors for whole-course teaching but required that a receiving site classrooms be supervised by a certified teacher. It recommended, but did not require, that the teacher be certified in a related field. The project ran into problems in one participating state.
Star Schools-sponsored supplementary instruction was received in regular classrooms that were staffed by certified teachers. Therefore, there were no concerns about the certification status of studio teachers.
7. What were the effects of distance learning on curricula and staffing patterns at participating schools?
In general, Star Schools activities did not affect curricula and staffing patterns at participating schools. However, two outcomes occurred in a limited number of school. first, a few schools reported that math teachers who supervised distance learning calculus classes became confident enough to teach calculus at the site. Second d, one Star Schools project fostered interdisciplinary instruction. In its most successful sites, team teaching and other forms of faculty collaboration were reported to increase.
The major effect of Star Schools activities was to broaden opportunities for students to take a variety of courses, particularly in math, science, foreign language, and in all AP subjects.
8. How many students participated in Star Schools activities? What were their characteristics?
Based on project documents, supplemented by information gathered in site visits and follow-up telephone interviews, a conservative estimate is that 140,000 students currently are participating in Star Schools distance learning activities. The students live in 48 states.
Less clear is the degree to which disadvantaged and traditionally underserved populations are being reached through the Star Schools Program. Projects do not tend to collect data concerning students' gender, ethnicity, or poverty levels. From the limited information available, it appears that 58 percent of students served are minorities, 40 percent are educationally disadvantaged, and 77 percent are in Chapter 1-eligible schools. (However, there is little information about how many students participating in Star School are actually receiving Chapter 1 services.) The mail survey to schools that is planned for Phase 2 of the study will systematically collect information on these issues.
9. What are the socioeconomic and geographic characteristics of schools participating in Star Schools projects? What types of programs are provided to different schools?
Again, project-generated data are incomplete regarding the types of schools participating and the types of Star Schools-sponsored activities they receive. In general, it seems that urban schools are more likely to receive programs designed to supplement regular instruction than they are to receive whole courses. Rural schools seem equally likely to receive both types of programs. In absolute numbers, there are fewer rural than urban school, largely because two projects involve numerous schools in the largest school district in the nation.
Additional Findings
The Two-Year Funding Cycle
Of all policy decisions, the one that gave projects and receiving schools the most problems was the two-year funding cycles. For most projects, the first year of the grant was spent acquiring and installing equipment, recruiting sites, and developing programs. Consequently, schools did not begin active participation until the second year, and often experienced normal problems of implementation. The problem was most difficult for projects that used a variety of technologies or engaged in a great deal of curriculum development, but it affected all projects.
Even projects offering whole-course instruction that was fairly traditional (AP course, foreign language courses, and high school math and science courses) found they could not solidify their market with a two-year funding cycle. One reported that it as willing to use Star Schools funding as seed money, but it needed more time to convince potential "buyers" of the value of the product.
The two-year funding cycle had particularly negative effects on projects that viewed Star Schools as an R&D opportunity. The time limitation pushed projects to deliver services to participants as early as possible, without adequate time for field-testing or limited market tests. The short funding period provides a perverse incentive for projects to work with schools that need only equipment, not facility changes; courses that are readily developed and therefore fairly traditional; and with staff that accept technology. The incentive, in sum, works against the goal of equalizing educational opportunity.
Schools visited during the 1992-93 academic year were selected by projects as examples of "best- case" implementations of Star School activities. Staff within these sites, while praising Star Schools project staff, raised the question of ongoing support.
In part, the two-year funding cycle fits a view of Star School as a seed money program, particularly if the seed money is allocated to equipment. However, all projects engaged in some program development. Further, even funding on equipment required projects to train staff at the receiving site, and the two years were inadequate to the task.
