Bridgingthe Gap: Equity in Systemic Reform

 

 

Jane Butler Kahle; Miami University

FINAL REPORT FOR AWARD # 9602137

 

 

 

 

 

Co Principal Investigator(s):

Kenneth G Tobin;
Steven R Rogg;
Kathryn C Scantlebury
Judith L. Meece

 

 

Contents

 

Partner Organizations

Other Collaborators

Activities and Findings

Training and Development

Outreach Activities

Journal Publications

Book(s)or Other One-Time Publications

Major Presentations

Other Specific Products

Contributions

Partner Organizations:

 

Ohio Board of Regents

Ohio Department of Education

 

University of Wisconsin

Kent State University

Ohio State University

Curtin University

Indiana University

University of Pennsylvania

 

Cincinnati Public Schools

Cleveland Municipal SchoolDistrict

Columbus Public Schools

Hudson Public Schools

Zanesville City Schools

 

Grover Cleveland MiddleSchool

Hudson Middle School

Johnson Park Middle School

Joseph Gallagher MiddleSchool

Shroder Paideia MiddleSchool

A. B. Hart Middle school

Clark Middle School

Meigs Junior High School

North Junior High School

Parker Elementary School

Peoples Middle School

Schwab Middle School

Trotwood-Madison MiddleSchool

 

Other collaborators:

Within Miami University

Dr. Amin Wang, Department of Educational Psychology,Miami University

Dr. Jane Kaiser, Department of Mathematics, MiamiUniversity

Dr. James Poth, Department of Physics, MiamiUniversity

Dr. Arta Damnjanovic, Department of EducationalLeadership, Miami University

Dr. Bernard Badiali, Department of EducationalLeadership, Miami University

 

Outside Miami Universitybut within the U.S.A.

Dr. James Bishop, The OhioState University

Dr. Patricia Campbell,Campbell-Kibler Associates

Dr. Mary Ellen Harmon,Boston College, TIMSS International Study Center

Dr. Henry Heikkinen,Northern Colorado University

Dr. Joanne Goodell,Cleveland State University

Dr. Norman Webb, Universityof Wisconsin

Dr. Andrew Porter,University of Wisconsin

Dr. Iris Weiss, HorizonResearch Institute

Dr. Paul LeMahieu,University of Delaware

 

Outside the U.S.A.

Dr. Barry Fraser, NationalKey Center for School Science and

Mathematics, Curtin Universityof Technology, Perth, Australia

Dr. Leslie Parker, DeputyVice Chancellor, Curtin University of Technology, Perth, Australia

Dr. Leonie Rennie, NationalKey Center for School Science and Mathematics, Curtin University of Technology,Perth, Australia

Dr. Kopano Taole, Foundationof Research and Development, South Africa

Activities and findings:

Research and Education Activities

Bridging the Gap: Equityin Systemic Reform was proposed toevaluate all mathematics and science education reform efforts in Ohio,including the Statewide Systemic Initiative (Discovery), the Urban SystemicInitiatives in Cincinnati, Cleveland, and Columbus, and the Appalachian RuralSystemic Initiative in southern Ohio counties. Its primary focus was the SSI(funded in

1991), which focused onupgrading the content and instructional skills of middle school teachersthrough six-week summer institutes in physics, mathematics, and life sciences.Teachers received the equivalent of 6 weeks of graduate education(approximately 140 contact hours). In 1997, the SSI added institutes foreducational leaders to assist principals and other administrators infacilitating school-wide reform. Miami University (MU) and The Ohio StateUniversity (OSU) were the sites of the first SSI institutes, but eventuallythey were offered at 26 colleges and universities in Ohio. Currently,OSI-Discovery’s institutes are the primary source of professional developmentfor Ohio’s teachers and administrators. One of its institutes, Physics byInquiry, has been approved in multiple teacher education programs across thestate.

 

From1991 to 2001, Discovery has provided professional development for almost 13,000Ohio mathematics and science teachers and for approximately 350 principals andother administrators. Further, it has worked with 90 teachers andadministrators as part of its Model School initiative. Although Bridging did not provide the described institutes, Bridging’s capacity (access to teachers, administrators, andstudents; maintenance of its data base; availability of baseline data; etc.)was extended by its close connection with Discovery. Bridging extended the human resource capacity in the state bydeveloping and providing instruction in evaluation and research techniques.(See Sections II.3 and II.4).

 

Themain focus of Bridging was toaccess equity in systemic reforms. It was specifically designed to provideinformation on equal access to reform-based science and mathematics education,equity in student outcomes, and persistent barriers to achieving equity inreform. Research activities supported by Bridging included (a) refinement and development of researchinstruments; (b) data collection; (c) development of a linked database; (d)data analyses; and (e) dissemination of research findings through bothscholarly and popular venues.

 

Thestudy utilized both quantitative and qualitative research methodologies andemployed a three-tier design. It included a large random sample of 100-150schools, designated Level A, where principals and all mathematics and scienceteachers (Grades 6 to 9) completed questionnaires focusing on standards-basedteaching, parent involvement, and administrative support related to mathematicsand science education. From the statewide random sample, a subset, ranging from8-14 schools depending on the year, was selected based on student demographicinformation. Level B schools enrolled at least 25% African American students orserved a high number of students eligible for free or reduced lunch. At theseschools, questionnaire data from students, administrators, and teachers werecollected to assess teaching practices, attitudes, and support related tomathematics and science education. Students also completed achievement testsdesigned to measure their problem solving abilities and conceptualunderstandings in mathematics or science. In addition, three to four day site visits were made to Level B schoolsfrom 1994 through 1997 to collect observation and interview data.  Last, Level C of the research designincluded two-year case studies at five schools, involving extensiveobservations and interviews with students, teachers, administrators, parents,and community leaders as well as student achievement data.

 

Atall levels of data collection, information from classes taught by Discovery-trainedteachers was contrasted with information from teachers who did not receiveDiscovery training. From 1994 through 2000, data were collected from 420schools, 4,643 classrooms, and over 19,000 students from across Ohio.

 

Todate, Bridging researchactivities resulted in 9 international, 59 national, and 9 regionalpresentations at national meetings. A list of journal articles, periodicals, and book chapters can be foundin Section III.1.  A description ofthe instruments and the database is provided in Section III.3 of this report.

 

Findings

Bridging the Gap resulted in several important contributions to theknowledge base on systemic reform in mathematics and science education. Theresults indicated that sustained professional development can increase theteachers' use of teaching methods aligned with the standards of the NationalCouncil of Teachers of Mathematics (NCTM) or the National Research Council(NRC) in science and mathematics classrooms, including those located in urbansettings. The results further show that standards-based teaching practices arepositively linked to middle school students' achievement in mathematics andscience. Across all years, middle school students in Discovery classesoutperformed students in Non-Discovery classes on standardizedassessments.  Moreover, gender,socioeconomic, and racial achievement gaps are smaller in classes taught byteachers who used standards-based teaching methods. Finally, case study analysesidentified important facilitators and barriers to equitable reform inmathematics and science. Each of these important finding are briefly summarizedbelow.

 

First,teacher measures indicate that teachers who participated in Discovery'sprofessional development programs were more likely to use teaching practicesaligned with the NCTM or NRC.  Forexample, Discovery-trained teachers were more likely than Non-Discoveryteachers to report that they provided opportunities for students to solveproblems together, to work at their own pace, and to answer open-endedquestions. More importantly, teachers' reports of standards-based teachingpractices are supported by the student data. Students of Discovery-trainedteachers reported greater use of standards-based teaching methods in theirclasses than did students taught by non-Discovery teachers.  Significant differences betweenDiscovery and Non-Discovery teachers were found for both science andmathematics (see Kahle & Rogg, 1997, 1998; Kahle & Damnjanovic, 1998;Kahle, Meece, & Damnjanovic, 1999). Thus, the results support the efficacyof sustained professional development activities for increasing the use ofstandards-based teaching practices in mathematics and science education.

 

Second,assessments of mathematics and science achievement revealed positive benefitsfor middle school students taught by Discovery-trained teachers. The primarymeasure of achievement was the Discovery Inquiry Test, which used 1990 and 1992public release items from the National Assessment of Educational Progress(NAEP) to measure students' problem solving abilities and conceptualunderstandings in mathematics or science (for additional information, seeSection III.3 below).

 

Acrossthe years, students in the Discovery classes, regardless of race/ethnicity,gender, or disability, outperformed students in Non-Discovery classes onachievement measures. From 1995 to 1999, mathematics scores on the DiscoveryInquiry Test increased from 57% to 70% correct for eighth-grade students taughtby Discovery teachers, whereas the mathematics scores of their peers inNon-Discovery classes remained below 60% correct. In science from 1995 to 1999,achievement scores of eighth-grade students in Discovery classes increased from54% to 73% correct, whereas the science scores of students in Non-Discoveryclasses remained below 70% correct (see Kahle, Meece, & Damnjanovic, 1999).In general, effect sizes were small and varied across school settings; however,teacher differences in the African American sample, favoring Discovery-trainedteachers, remained significant when differences in students' home environmentwere controlled (parental encouragement for mathematics or science achievement,number of books in the home, parental educational level, etc.; see Meece &Kahle, in preparation).

 

In1998, we collected data from a sub-sample of Ohio's schools to assess relationsbetween the Discovery's Inquiry Tests and Ohio's Proficiency Tests (OPT) foreighth-grade students. Using a sample of Ohio's urban schools, results showedthat students' performance on the DIT was significantly correlated with theirachievement scores on the Ohio Proficiency Test (.97 for mathematics and .98for science). From these data, we estimated that a larger proportion ofDiscovery-taught students received passing scores in mathematics and science,when compared with students in Non-Discovery classes (see Kahle &Damnjanovic, 1998; Kahle, Meece, & Damnjanovic, 1999).  Moreover, results revealed thatachievement scores on state assessments improved significantly when a 'criticalmass' of the science and mathematics teachers within a school had participatedin Discovery professional development. Data from 1998 revealed that OPT scoresimproved 17.5% in mathematics and 9.2% in science in schools with more than 51%of Discovery teachers, whereas OPT scores in declined 11.3% and 3.3% inmathematics and science, respectively, in schools with fewer than 25% Discoveryteachers (see Kahle, 1999).

 

Third,there were positive associations between students' reports of standards-basedteaching practices and their performance on the Discovery Inquiry Tests inmathematics and science. Using Hierarchical Linear Modeling (HLM) proceduresthe unique contributions of standards-based teaching practices were examined inan analysis of African American students' achievement in Ohio's urban schools(see Kahle, Meece, & Scantlebury, 2000). The analyses focused oneighth-grade students in science classes, and results showed that 15% of thevariance in student achievement scores on Discovery Inquiry Test in Science wasdue to teacher differences. Results further showed that teachers' use ofstandards-based teaching practices, as reported by students, explained asignificant proportion of the variance attributed to teacher effects.

 

Animportant objective of Bridging the Gap was to assess equity in learning opportunities and student outcomes.Early analyses (1995-1997) indicated that both African-American and Whitestudents in Discovery mathematics classes outscored those in Non-Discoveryclasses. The results further showed that achievement gaps between AfricanAmerican and White students decreased in Discovery mathematics classes, whilegasps increased in Non-Discovery (Kahle & Rogg, 1997).  Looking within African Americansamples, Bridging the Gap datarevealed that standards-based teaching practices in science can promote genderequity in science achievement. In science classes taught by Discovery-trainedteachers, gender differences, favoring females in this sample, were smaller inmagnitude when compared with students in Non-Discovery science classes (seeKahle, Meece, & Scantlebury, 2000). Analyses of mathematics data of African American students in urbanschools revealed that teacher effects (Discovery vs. Non-Discovery) explainedvariance in achievement scores, not explained by differences in homeenvironment (e.g., number of books, parental education level, access tocomputer, etc.).  Results alsorevealed no differential impact of reform practices in mathematics for genderor home background variables (Meece & Kahle, in preparation).

 

Intensivecase studies were completed at five middle schools.  Within each school, observation and interview data werecollected and combined with quantitative data to assess progress towardequitable practices in mathematics and science education. The framework foranalyzing case reports was Kahle's (1998) Equity Metric that includes 28indicators of equitable mathematics and science education related to (1) Access(e.g., minutes per day of mathematic and science instruction, enrollmentpatterns in mathematics and science, home resources, etc.); (2) Retention(e.g., instructional quality, teacher expectations and behavior, studentmobility, etc.); and (3) Achievement (e.g., increases in achievement, decreasesin achievement gaps, increases in graduation rates, etc.).

 

Casestudy analyses revealed several important themes. First, the five schoolsshowed varying levels of progress toward equitable reform (Kahle & Kelly,2001). Three of the five schools showed minor progress in all categories ofequity, but did not excel in any category. Schools also differed in categoriesof progress. Some schools showed progress in Access, whereas others showedprogress in Retention or Achievement. In these schools several barriers toreform were identified including the limited number of teachers involved in thereform process, the mobility of school personnel, and the effectiveness ofschool leaders.

 

Onlyone of the five schools was successful in meeting indicators of progress acrosscategories, and it was particularly strong in retaining middle school studentsin quality science and mathematics courses. The underlying philosophy of thisschool discouraged academic tracking, empowered teachers, and increasedadult-to-student ratios. In addition, the teachers assumed an active role inreform processes (e.g., encouraging other teachers to participate, teachingyear-long professional development workshops, etc.). Also, the school'sstability led to low student, teacher, and administrative mobility. Takentogether, the case studies emphasized the important role of the following infacilitating reform in mathematics and science education at the school level:(1) A stable population of principals, teachers, and students; (2) Theleadership role of the principal and shared goals between teachers andprincipals; (3) Positive teacher expectations concerning the ability of allstudents to learn high quality mathematics and science; (4) Active parentalinvolvement; and (5) State, district, and school alignment in  reform initiatives.

Training and Development:

Bridging has contributed to the research and teaching skillsof those who have worked on the project by providing in-service and large-scaleresearch opportunities to K-12 teachers, administrators, graduate students, postdocs, and college faculty.

I. In-Service Opportunities

Through the Institute forthe Assessment of Evaluation in Science, Mathematics and Technology Education,40 individuals including eight elementary school teachers, 14 middle schoolteachers, 10 high school teachers, eight district board members, eight graduatestudents and faculty were able to develop the knowledge, skill, and techniquesto implement authentic, equitable assessment in classrooms and to evaluate theextent to which instruction met national standards in mathematics and science.

 

Thirty-nineindividuals, including four elementary and 14 middle school teachers from fivedistricts, eight Cincinnati Public School officials, two Cleveland PublicSchool officials, six university faculty, and five Discovery graduate studentswere provided training in administrating and scoring of the Third InternationalMathematics and Science Symposium (TIMSS) performance tasks. Students involunteer school districts also participated in the TIMSS performance testing.

State-levelresults were disseminated to concerned parents, administrators, teachers, andthe public.

 

Supplementalfunds were used to share Bridgingexpertise with a group of South African educators and researchers. Threescholars from the Rhodes University's Mathematics Education Project, one fromthe Curriculum 2005 Project at the University of the North, one from theUniversity of Port Elizabeth, and the Education Director, Foundation forResearch Development, Pretoria, South Africa, learned about standards-basedU.S. curricula and large-scale evaluation techniques. In addition, theresearchers and educators were introduced to classroom environment research.

II. Large Scale Research Opportunities

A total of 168 teachers(Level B) participated regularly in data collections, interviews andobservations at their schools.

 

Bridging worked in close partnership with seven teachers infive districts to develop an in-depth understanding of school-level issues andto assess individual teaching experiences (Level C). 

 

Sixgraduate students, two post-docs, and the fifteen university faculty (from 12different universities) were provided with direct opportunities to assessequity in systemic reform as well as to develop and use the database. Theseindividuals were involved in large-scale research methodologies such asHierarchical Linear Modeling (HLM) and Item Response Theory (IRT). Theydeveloped a good background in equity and a firm understanding of the researchmethodologies used in investigating educational trends. They acquired theexpertise to analyze the effectiveness of systemic reform efforts in a widerange of school settings.

Outreach Activities:

Bridging has reached out to community members through variousassociations, publications, and presentations.  Three invited presentations were provided for the OhioGeneral Assembly.  Other outreachesfor public understanding were facilitated through the Ohio Mathematics andScience Coalition (OMSC), the Council of Chief State School Officers (CCSSO),and the Ohio School Board.

 

Throughyearly brochures, Bridging hasshared key research findings with the state’s governor, state and federalsenators and legislators, as well as with school personnel including teachers,administrators, superintendents, and board members.  These concise and informative brochures also were sent tothe Ohio Department of Education, Ohio Board of Regents, and the StateUniversity Education Deans, as well as to Ohio’s network of Regional ProfessionalDevelopment Centers. Dissemination of information through these brochures hasproven very effective in reaching out to community members who may not havebeen familiar with the Bridgingactivities and findings.

 

Bridging’s findings have been presented at 9 international, 59national, and 9 regional professional meetings. Through these presentations, Bridging was able to reach out to members in international,national, and local learning communities. A complete listing of the presentations is found in the Major Findingssection (II.2) of this report. 

Journal Publications:

Boone, W. J. (1998). Assumptions,cautions, and solutions in the use of omitted test data to evaluate theachievement of underrepresented groups in science—Implications for long-termevaluation. Journal of Women and Minorities in Science and Engineering, 4, 183­194.

Boone, W. J., & Kahle, J. B.(1998). Student perceptions of instruction, peer interest, and adult supportfor middle school science: Differences by race and gender. Journal of Womenand Minorities in Science and Engineering, 4, 333­340.

Boone, W. J., & Kahle, J. B.(1997). Implementation of the standards: Lessons from a systemic initiative. SchoolScience and Mathematics, 97, 292­299.

Carnes, G. N. (1998). An assessmentof equitable instruction within three urban middle school classrooms. Journalof Women and Minorities in Science and Engineering, 4, 283­296.

Damnjanovic, A. (1998). OhioStatewide Systemic Initiative (SSI) factors associated with urban middle schoolscience achievement: Differences by student sex and race. Journal of Womenand Minorities in Science and Engineering, 4, 217­233.

Kahle, J. B. (1998). Equitablesystemic reform in science and mathematics: Assessing progress. Journal ofWomen and Minorities in Science and Engineering, 4, 91­112.

Kahle, J. B. (1998). Measuringprogress towards equity in science and mathematics education (Brief Vol. 2, No. 3). Madison:University of Wisconsin­Madison, National Institute for Science Education.

Kahle, J. B. (1998). Reachingequity in systemic reform: How do we assess progress and problems? (Research Monograph #5). Madison:University of Wisconsin­Madison, National Institute for Science Education.

Kahle, J. B. (1997). Systemicreform: Challenges and changes. Science Educator, 6, 1­6.

Kahle, J. B., & Boone, W. J.(2000). Strategies to improve student science learning: Implications forscience teacher education. Journal of Science Teacher Education, 11(2): 93-107.

Kahle, J. B., & Kelly, M. K.(2001). Equity in reform: Case studies of five middle schools involved insystemic reform. Journal of Women and Minorities in Science and Engineering,7, 79-96.

Kahle, J. B., & Rogg, S. R.(1997). Assessing systemic change: Ohio’s Statewide Systemic Initiative,Discovery. It’s difficult to measure change while changing the measure. SystemicInitiatives, 1 (3),10, 11.

Kahle, J. B., Meece, J., &Scantlebury, K. (2000). Urban, African American, middle school sciencestudents: Does standards-based teaching make a difference? In Journal of Researchand Science Teaching, 37, 1019-1041.

Roychoudhury, A., & Kahle, J. B.(1999). Science teaching in the middle grades: Policy implications for teachereducation and systemic reform. Journal of Teacher Education, 50, 278-289.

Scantlebury, K., Boone, W., Kahle,J. B., & Fraser, B. J. (2001) Design, validation and use of an evaluationinstrument for monitoring systemic reform. Journal of Research in ScienceTeaching, 38 (6):1-17.

Singham, M. (1998). The canary inthe mine—The achievement gap between black and white students. Phi DeltaKappa, 80 (1),9­15.

Supovitz, J., Kahle, J. B., &Mayer, D. (2000). The impact over time of Project Discovery on teachers’attitudes, preparation, and teaching practice. Educational Policy, 14, 331-356.

Book(s) or other one-time publications(s):

Book Chapters

Goodell, J. E., Parker, L., &Kahle, J. B. (2000). Assessing the impact of sustained professional developmenton middle-school mathematics teachers. In J. McIntyre (Ed.), Association ofTeacher Educators Teacher Education Yearbook VIII: Effective models of teachereducation. (pp27-43) Thousand Oaks, CA: Corwin Press.

Kahle, J. B., & Kelly, M. K.(2001). Science teacher professional development: A researcher's perspective.In J. Rhoton & P. Bowers (Eds.), Professional development in scienceteaching and learning. (pp. 101-113) Arlington, VA: National Science Education LeadershipAssociation and National Science Teachers Association.

 

Kahle, J. B. (1998). Assessing progress towardshigh-quality mathematics and science. In Infusing Equity in Systemic Reform:An Implementation Scheme (pp.40­53). Washington, DC: National Science Foundation Directorate for Educationand Human Resources.

Dissertations

Carnes, G. N. (1996). Aninvestigation of the effect of an intensive professional development program onthree urban middle school teachers: A case study. Dissertation AbstractsInternational, 57(07), A2798.

Damnjanovic, A. (1996). Ohio-SSIfactors associated with urban middle school science achievement: Differences bystudent sex and race. Dissertation Abstracts International, 57 (07), A2952.

Goodell, J. E. (1998). Equity andreform in mathematics education. Unpublished doctoral dissertation. Curtin University ofTechnology, Perth, Western Australia.

Herrera, T. (1995). Teacher changein the professional development setting: Case studies of three middle schoolteachers of mathematics. Dissertation Abstracts International, 56 (12), A4689.

 

Kelly, M. K. (2001). Moving Toward Equitable,Systemic Science Education Reform: The Synergy Among Science Education andSchool-Level Reforms in an Urban Middle School. Unpublished doctoral dissertation. MiamiUniversity, Oxford, Ohio

Technical Reports/Proceedings

Kahle, J. B. (1998). Pathways tosystemic reform: Case studies of Ohio schools.

1.            Battista, M., Mathematics education reform at Lafayette Middle School.

2.            Hewson, P., & Davies, D., Urban Middle School: How much is too much?         

3.            Kahle, J. B., Scantlebury, K., Damnjanovic, A., & Kelly, M. K., Steele Middle School: The best education for the best is the best education for all.

4.            Costner, K. M., & Wagner, S., Central Ohio case study.

5.            Hewson, P., & Kahle, J. B., Cross-site analysis of five schools involved in systemic reform.

Kahle, J. B. (1997). Impressionsof reform in Ohio schools.

1.            Battista, M., Lafayette Middle School.

2.            Brooks Hedstrom, M., Daniel Miller Junior High School.   

3.            Damnjanovic, A., Hutton Middle School.

4.            Damnjanovic, A., Turner Middle School.

5.            Hewson, P., Urban Middle School.

6.            Kahle, J. B., Steele Middle School.

7.            Keiser, J., J., Adams Middle School.

8.            Kelly, M. K., Anderson Junior High School.

9.            Kelly, M. K., Peterson Middle School.

10.       Rogg, S., Jackson Junior High School.

11.       Tobin, T., West Side Middle School.

12.       Wagner, S., Macon Junior High School.

13.       Wagner, S., Lantern Hill Middle School.

Kahle, J. B. (1997, December). Whathave we learned? What do we need to know? In W. H. Clune, S. B. Millar, S.A. Raizen, N. L. Webb, D. C. Bowcock, E. D. Britton, R. L. Gunter, &R. Mesquita (Eds.), Research on systemic reform: What have we learned? Whatdo we need to know?(Vol. 2: Proceedings, Workshop Report No. 4, Synthesis of the Second AnnualNISE Forum, pp. 11­16). National Institute of Science Education,University of Wisconsin­Madison.

Kahle, J. B., & Meece, J. L.(2000). Reform that works, December 2000, [Brochure]. Oxford, OH: Miami University.

Kahle, J. B., Meece, J. L., &Damnjanovic, A. (1999). A pocket panorama of Ohio’s systemic reform, 1999, [Brochure]. Oxford, OH: MiamiUniversity.

Kahle, J. B., & Damnjanovic, A.(1998). A pocket panorama of the Landscape Study, 1998, [Brochure]. Oxford, OH: MiamiUniversity.

Kahle, J. B., & Rogg, S. R.(1998). A pocket panorama of the Landscape Study, 1997, [Brochure]. Oxford, OH: MiamiUniversity.

Kahle, J. B., & Rogg, S. R.(1997). A pocket panorama of the Landscape Study, 1996, [Brochure]. Oxford, OH: MiamiUniversity. (ERIC Document Reproduction Service No. ED 419 687).

 

Kahle, J. B., & Rogg, S.R. (1996). A pocket panorama of the Landscape Study,

1995, [Brochure]. Oxford, OH: Miami University.

Major Presentations:

International

Hewson, P., & Kahle, J. B.(1999, August). Equity in systemic reform: Comparing two middle schools. Paper presented at the SecondInternational Conference of the European Science Education Research AssociationE.S.E.R.A., Kiel, Germany.

Kahle, J. B. (1999, October). Areview of science education research. Seminar, Swiss Institute of Technology, Zurich,Switzerland.

Kahle, J. B. (1999, October). Professionaldevelopment, systemic reform, and curriculum change. Keynote address, National ScienceFoundation and the National Research Foundation Conference, Pretoria, SouthAfrica.

Kahle, J. B. (1998, January). Twoexamples of systemic reform in science and mathematics. Invited plenary address presentedat the Foundation for Research Development, United States/South Africa Forum onSystemic Reform, Pretoria, South Africa.

Kahle, J. B. (1997, January). Assessmentand evaluation of systemic reform. Invited paper presented at the International Conference onScience, Mathematics & Technology, Hanoi, Vietnam.

Scantlebury, K., Tobin, K., Kahle,J. B., & Ague, J. (1999, July). The quiet achiever: Enacting culturallyrelevant science teaching in an American, urban middle school. Paper presented at the meeting ofthe Australasian Science Education Research Association, Rotorua, New Zealand.

Tobin, K. (1998, March). Studies of scienceeducation in urban schools: Research that makes a difference. Paper presented at the meeting of the 7thInternational Consultation for Research in Science and Mathematics Education,Port of Spain, Trinidad.

National

Arambula-Greenfield, T. (2000,April). Transforming middle school science: A case study. Symposium presented at the meetingof the National Association for Research in Science Teaching, New Orleans, LA.

1.            Arambula-Greenfield, T. (2000, April). Student views of science reform. In Arambula-Greenfield (Chair), Transforming middle school science: A case study.

2.            Carnes, G. N., & Damnjanovic, A. (2000, April). Teacher views of science reform. In Arambula-Greenfield (Chair), Transforming middle school science: A case study.

3.            Kelly, M. K., & Kahle, J. B. (2000, April). Science reform and school-level reform efforts in one urban school. In Arambula-Greenfield (Chair), Transforming middle school science: A case study.

4.            Poth, J., & Fraser, B. (2000, April). Constructivist nature of classroom environments in a middle school undergoing reform. In Arambula-Greenfield (Chair), Transforming middle school science: A case study.

Boone, W. J. (1998, April). Testitem measurement—Key steps to accurately calculating achievement measures. Paper presented at the meeting ofthe National Association for Research in Science Teaching, San Diego, CA.

Boone, W. J., Scantlebury, K., &Kahle, J. B. (2000, January). Science teacher education and reform: An instrumentto measure opportunities to learn. Paper presented at the meeting of the Association for theEducation of Teachers in Science, Akron, OH.

Boone, W. J., Scantlebury, K., &Kahle, J. B. (2000, April). Five years of statewide science measurement. Paper presented at the meeting ofthe National Association for Research in Science Teaching, New Orleans, LA.

Boone, W. J., Scantlebury, K.,Kahle, J. B., & Damnjanovic, A. (1999, January). An instrument to gaugeteachers’ attitudes towards and assessment of parental involvement in middleschool classrooms.Paper presented at the meeting of the Association for the Education of Teachersin Science, Austin, TX.

Carnes, G. N., & Damnjanovic, A.(2000, April). Teachers’ voices: An account of teaching and learning atWebster Middle School. Paper presented at the meeting of the National Association of Researchin Science Teaching, New Orleans, LA.

Chappel, J. L. (1997). SystemicReform: What not to do/What to do. Paper presented at the meeting of the School Science andMathematics Association, Milwaukee, WI.

Fraser, B. J., & Kahle, J. B.(1999, April). Building a Theory of Systemic Reform. Discussant, American EducationalResearch Association Annual Meeting, Montreal, Canada.

Fraser, B. J., Kahle, J. B.,Scantlebury, K., & Meece, J. (1999, April). Classroom, home, and peerenvironment influences on student outcomes: An analysis of systemic reform data. Paper presented at the meeting ofthe American Educational Research Association, Montreal, Canada.

Goodell, J. E., & Kahle, J. B.(2000, April). Effective schools and restructuring: The case of one urbanmiddle school.Paper presented at the meeting of the American Education Research Association,New Orleans, LA.

Goodell, J. E., Parker, L. H., & Kahle,J. B. (2000, April). Facilitators and barriers to achieving equity andreform in middle-school mathematics classrooms. Paper presented at the meeting ofthe American Education Research Association, New Orleans, LA.

Goodell, J. E., Parker, L. H., &Kahle, J. B. (1998, April). Assessing the impact of sustained professionaldevelopment on middle school mathematics teachers. Paper presented at the meeting ofthe American Education Research Association, San Diego, CA.

Hewson, P. (1999, April). Equityin systemic reform: The case of mathematics and science in middle schools. Symposium presented at the meetingof the American Educational Research Association, Montreal, Canada.

1.            Battista, M., (1999, April). Mathematics education reform at Lafayette Middle School. In P. Hewson (Chair), Equity in systemic reform: The case of mathematics and science in middle schools.

2.            Hewson, P., & Davies, D. (1999, April). Urban Middle School: “How much is too much?” In P. Hewson (Chair), Equity in systemic reform: The case of mathematics and science in middle schools.

3.            Kahle, J. B., & Hewson, P. (1999, April). Cross-site analysis of five schools involved in systemic reform. In P. Hewson (Chair), Equity in systemic reform: The case of mathematics and science in middle schools.

4.            Scantlebury, K., Kahle, J. B., Kelly, M. K., & Damnjanovic, A. (1999, April). Steele Middle School: “The best education for the best is the best education for all.” In P. Hewson (Chair), Equity in systemic reform: The case of mathematics and science in middle schools.

5.            Costner, K. M., & Wagner, S. (1999, April). Central Ohio Middle Schools. In P. Hewson (Chair), Equity in systemic reform: The case of mathematics and science in middle schools.

Kahle, J. B. (2001, February). Equityin outreach: Measuring the difference. Presented paper, Conference on K-12 Outreach fromUniversity Science Departments, Pittsboro, NC.

Kahle, J. B. (1999, December). Ohiosystemic initiative: Outcome based assessment, research, evaluation andcommunication.Presented paper, Division of Elementary, Secondary and Informal Education,National Science Foundation, Washington, D.C.

Kahle, J. B. (1999, January). Bridgingthe Gap: Equity in Systemic Reform. Invited address, Performance Effectiveness Review (PER),K-12 Student Assessment Activities, Directorate for Education and HumanResources, National Science Foundation, Washington, DC.

Kahle, J. B. (1998, July). Whatmatters in K­12 mathematics and science? Effective strategies for reform andstudent achievement.Testimony, National Science Board, Chicago, IL.

Kahle, J. B. (1998, April). Howcan we measure and monitor the progress of our efforts to infuse equity in asystem? InfusingEquity in Systemic Reform: An Implementation Scheme. Invited panelist for themeeting of the National Science Teachers Association, Las Vegas, NV.

Kahle, J. B. (1998, April). Lions,zebras, and education reform in South Africa. Invited address presented at the meeting of theAdvisory Committee, Directorate for Education and Human Resources, NationalScience Foundation, Washington, DC.

Kahle, J. B. (1997, October). Reflectingon Sputnik: Linking the past, present, and future of educational reform. Invited panelist, Center forScience, Mathematics, and Engineering Education, National Research Council,Washington, DC.

Kahle, J. B. (1997, September). Largescale assessments and equity. Invited plenary address, Programs for Women and Girls,National Science Foundation, Washington, DC.

Kahle, J. B. (1997, April). Equitymetrics—How we can measure equity. Invited paper presented at the meeting of the Council ofChief State School Officers (CCSSO) and the American Association for theAdvancement of Science, Collaboration for Equity: Fairness in Science andMathematics, Washington, DC.

Kahle, J. B. (1997, March). Reachingequity in systemic reform: How do we assess progress and problems? Invited paper presented at theEvaluation of Systemic Initiatives Conference, National Institute for ScienceEducation, University of Wisconsin­Madison.

Kahle, J. B. (1996, February). Assessingsystemic reform from national perspectives to students outcomes. Symposium presented at the meetingof the American Association for the Advancement of Science, Baltimore, MD.

1.      Kahle, J. B., & Damnjanovic, A. (1996, February). Bridging the Gap: An assessment of minority and majority student achievement levels and attitudes. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

2.      Lewis, J. (1996, February). Assessing an urban systemic initiative: Cincinnati. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

3.      Rogg, S. (1996, February). The affect of systemic reform initiatives on the public’s perception of mathematics and science. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

4.      Tobin, K. (1996, February). Assessing progress: Describing the landscape of science and mathematics education in Ohio. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

5.      Weiss, I. (1996, February). Approaches to evaluating systemic reform in states with varying strategies. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

6.      Zucker, A., & Shields, P. (1996, February). Beyond program evaluation: Assessing the impact of systemic change. In J. B. Kahle (Chair), Assessing systemic reform from national perspectives to students outcomes.

Kahle, J. B., & Kelly, M. K.(2000, April). Equity in reform: Case studies of five middle schoolsinvolved in systemic reform. Paper presented at the meeting of the National Association forResearch in Science Teaching, New Orleans, LA.

Kahle, J. B., Meece, J. L., &Scantlebury, K. (2000, April). Urban African American middle school sciencestudents: Does standards-based teaching make a difference? Paper presented at the meeting ofthe American Education Research Association, New Orleans, LA.

Kelly, M. K., & Kahle, J. B.(2000, April). The contributions of school-level reform efforts to theprogress of science reform: The case of one urban middle school. Poster presentation at the meetingof the American Education Research Association, New Orleans, LA.

Kelly, M. K., & Kahle, J. B.(1999, March). A comparison of student achievement on performance andpaper-and-pencil assessment tasks. Paper presented at the meeting of the National Associationfor Research in Science Teaching, Boston, MA.

Kelly, M. K., & Kahle, J. B.(1999, March). TIMSS Performance Assessment student interviews. Paper presented at the meeting ofthe National Science Teachers Association, Boston, MA.

Kelly, M. K., Kahle, J. B., &Scantlebury, K. (1999, January). Performance assessment as a tool to enhanceteacher understanding of student conceptions of science. Paper presented at the meeting ofthe Association for the Education of Teachers in Science, Austin, TX.

Meece, J. L., Scantlebury, K., &Kahle, J. B. (2001, March). “She Tricks You into Learning": The Role ofMotivation in Reform-Based Science Classrooms. Paper presented at the meeting ofthe American Educational Research Association Annual Meeting, Seattle,Washington.

Scantlebury, K. (1999, March). Aneducational mosaic: Using multiple measures to evaluate systemic reform. Symposium presented at the meetingof the National Association of Research in Science Teaching, Boston, MA.

1.               Boone, W., & Scantlebury, K. (1999, March). Challenges, false starts and successes: Using the Rausch Model to link science tests: A tool for the evaluation of reform. In K. Scantlebury (Chair), An educational mosaic: Using multiple measures to evaluate systemic reform.

2.               Fraser, B., Kahle, J. B., & Scantlebury, K. (1999, March). Classroom, home and peer environment influences on student outcomes: An analysis of systemic reform data. In K. Scantlebury (Chair), An educational mosaic: Using multiple measures to evaluate systemic reform.

3.               Kahle, J. B., & Hewson, P. (1999, March). Pieces of the mosaic: Cross-site analysis of five case studies. In K. Scantlebury (Chair), An educational mosaic: Using multiple measures to evaluate systemic reform.

4.               Scantlebury, K., Boone, W., Damnjanovic, A., & Kahle, J. B. (1999, March). Design of an evaluation tool to measure long-term systemic reform in science education. In K. Scantlebury (Chair), An educational mosaic: Using multiple measures to evaluate systemic reform.

Scantlebury, K., Meece, J. L., &Kahle, J. B. (2001, March). Students', Teachers and Researchers' Perceptionsof Standards-Based Teaching Practices in Science and Mathematics Middle SchoolClasses. Paper presentedat the meeting of the American Educational Research Association Annual Meeting,Seattle, Washington.

 

Scantlebury, K., Tobin, K., Kahle, J. B., & Ague,J. (1999, April). Culturally-relevant science teaching. Paper presented at the meeting of the AmericanEducational Research Association, Montreal, Canada.

Regional/State

Boone, W. J., & Kahle, J. B.(1998, May). Equitable assessments in K­12 classrooms. Paper presented at the 1998 SpringConference: Implementing Mathematics, Science and Technology AssessmentStandards into K­12 Classrooms, Purdue University, West Lafayette, IN.

Boone, W. J., Kahle, J. B., &Rogg, S. R. (1998, February). Bridging the equity gap in Ohio. Paper presented at the HoosierAssociation of Science Teachers, Inc. Convention, Indianapolis, IN.

Boone, W. J., Kahle, J. B., &Rogg, S. R. (1997, October). Investigating science equity gaps in Ohio. Paper presented at the IndianaAcademy of Science Annual Conference, St. Joseph’s College, Rensselaer, IN.

Kahle, J. B. (1999, November). Researchon teacher professional development and teaching practices. Keynote address at the HawaiiEnvironmental Education Association and Hawaii Science Teachers Association,Honolulu, HI.

Kahle, J. B. (1997, March). Measuringsystemic reform: The development of an equity metric. Invited paper, Curriculum Researchand Development Group, University of Hawaii­Manoa, Honolulu, HI.

Kahle, J. B. (1997, January). Assessingsystemic reform: What works. Seminar, Curriculum Research and Development Group, Universityof Hawaii-Manoa, Honolulu, HI.

Other Specific Products:

Data or databases

Principal, teacher, andstudent attitudinal and achievement data collected using the instrumentsdescribed later in this section, were stored in an ACCESS database system.  Principal, teacher, and student datawere stored separately by year. Student attitude and achievement information were linked. Entries weremade for 420 principals, 4,643 teachers, and 19,850 students. Teacher andstudent data in ACCESS were exported and linked to form an SPSS data set.  In this dataset, student informationfor 16,250 students was linked to information collected for their teachers.

 

Miami University’s humansubject regulations prohibit the direct sharing of the database with non-Bridging personnel. However, state-level results generated from the database have beenshared through numerous publications and presentations.  The information presented in thePublication section of this report lists publications that were generated usingthe Bridging database.  A complete list of presentationsgenerated using the Bridgingdatabase is attached as a file in the Major Findings section (II.2) of thisreport.

Physical collection (samples, etc.)

Artifacts were collectedfrom the Level C case study.  Theyincluded school brochures (or manuals) with rules and regulations, districtpolicies, course outlines, examples of lesson plans, assessments, and studentwork.

 

Bridging researchers used the artifacts to prepare reports,publications and presentations. The information presented in the Publication section of this reportlists publications generated using Bridging research, including the artifacts collected from theLevel C case studies.  A completelist of presentations generated using Bridging research, including the artifacts collected fromLevel C case studies is attached as a file in the Major Findings section (II.2)of this report.

Instruments or equipment developed

Valid and reliableprincipal, teacher, and student questionnaires and student mathematics andscience tests were developed.  Thequestionnaires assessed participants’ attitudes and perceptions ofstandards-based instructions. Tests evaluated student problem-solving, synthesis and applicationabilities.

 

Bridging has shared Principal, Teacher, and StudentQuestionnaires with university and school personnel throughout Ohio and withresearchers in Maine and Oregon. The student tests are secure, because of Discovery continues to usethose instruments.

Instruments or equipment developed

Performance-based TIMSS kitswere prepared for 8th grade mathematics and science students.  The kits were used at an in-serviceopportunity provided by Bridging(see the Training and Development section II 2).

 

Kahle's (1998) was developedto assess a school's progress toward equitable practices in mathematics andscience education. This instrument includes 28 indicators of equitablemathematics and science education related to (1) Access (e.g., minutes per dayof mathematic and science instruction, enrollment patterns in mathematics andscience, home resources, etc.); (2) Retention (e.g., instructional quality,teacher expectations and behavior, student mobility, etc.); and (3) Achievement(e.g., increases in achievement, decreases in achievement gaps, increases ingraduation rates, etc.).

 

The kits were given toparticipating teachers who use them with 8th grade mathematics andscience students in their schools.

 

Kahle's (1998) Equity Metrichas been published in the Journal of Women and Minorities in Science andEngineering (Vol. 4, pages 57-95),and it is available to other researchers.

Internet Dissemination

http://www.sistudyforum.org/studydetail.cfm?study_ID=11.MeeceSSI

Contributions:

Contributions within Discipline

Just as NSF took a bold leapin the design and implementation of the systemic initiatives, equally boldsteps were needed in the design and methodologies used to evaluate the reforms.Because a major goal of systemic reform was to narrow any achievement gapsbetween subgroups of students, the issue of assessing student achievement was ahigh priority. Bridging the Gap: Equity in Systemic Reform was designed specifically to assess any narrowing ofidentifiable achievement gaps between boys and girls, between African Americanand White students, and/or between students from different economic levels. Itscontributions to the base of knowledge, theory, and research are focused inthree areas: 1) methodology for collecting data across large numbers ofschools; 2) methodology for analyzing and reporting findings across schools andyears; 3) extension of the theory underpinning systemic reform of the theoriesconcerning achievement differentials among subgroups of students, and of thetheories concerning effective teaching and learning.

 

Bridging’s first addition to the existing knowledge base wasits adaptation of basic research methods to an applied research project. Forexample, simply obtaining reliable and valid data across a state, particularlyin its largest urban districts, was complex. The quality of data had to becarefully weighed against the cost of obtaining the data. Therefore, data onopportunities to learn, attitudes, and support of science and mathematicseducation were collected by questionnaires. However, those data were confirmedby visits to selected schools. In addition, because data collected withquestionnaires are subject to self-report bias, principals, teachers, andstudents responded to similar items. Their responses were compared, increasingconfidence in the findings. Last, a deep analysis (case studies) of diverseschools was conducted in order to try to fully understand barriers to orsupport mechanisms for systemic reform. Kahle's (1998) Equity Metric proved tobe a valuable instrument for assessing the progress of schools toward equitablereform in mathematics and science education. Further, Bridging contributed to the knowledge base through itsanalysis of achievement data in multiple ways in order to establish patterns ina situation so complex that direct attribution could not be established. Bridging added also to the knowledge base concerning how toeffectively communicate complex research results to the lay public. Bridgingthe Gap and Discovery jointlydisseminate the research findings through a variety of means, including thePocket Panorama (see Section III.1).

 

Anothercontribution is Bridging’s uniquedatabase. At both Levels B and C large numbers of urban, African Americanstudents, high proportions of whom are eligible for free or reduced-pricelunches, are represented. Second, important events such as the institution ofthe Ohio Proficiency Test in science occurred during the project period,allowing the assessment of the effect of policies on both practice andachievement.

 

Althoughthe project focused on research and evaluation of math and science educationreform, its methodologies and findings contribute to research in andunderstanding of urban education, whole school reform, or reform in otherdisciplinary areas (social studies, language arts, etc.). Because the researchmethodologies developed and/or tested are applicable to projects across EHR, ithas contributed to all divisions in that directorate as well as to theDirectorate for Social, Behavioral, and Economic Sciences.

Contributions to Other Disciplines

Although Bridging did not contribute directly to theinstitutionalization of undergraduate inquiry courses in mathematics, physics,and biology in the state, its promulgation of the success of Discovery’ssuccessful institutes in those disciplines did. The State of Ohio now requiresa course similar to the ones adapted (physics) or developed (math and biology)by Discovery in all science or math teacher preparation programs. Further, thephysics course has become a standard one in many physics departments in stateuniversities, including The Ohio State University.

Contributions to Education and Human Resources

Section II.3 describes indetail the opportunities for developing the research skills of teachers andfaculty that were provided by Bridging. In addition, two doctoral dissertations have been completed and one isin progress. Master and doctoral degree students from Indiana University andMiami University as well as Curtin University (Australia) have used Bridging data and methodologies.

 

Particularlyeffective has been Bridging’sefforts to improve the performance, skills, and attitudes of underrepresentedgroups. Both its TIMSS training and its Evaluation Institute (Section II.3)enrolled large numbers of African American and South African educators. Inaddition, most of the teacher researchers and administrators, listed in I.1 and2, are from minority groups. Efforts were made to involve minority graduatestudents in the collection and analysis of data (see Section I.1). Last, as acollaborative research project involving directly four major researchuniversities and five primarily African American middle schools (the casestudies), Bridging providedexposure to research and technology to young people (again predominantly minorityyouth).

Contributions to Resources for Science and Technology

In this area, Bridging’s contributions supplemented those of Discovery. Bothprojects were instrumental in raising the knowledge of and concern aboutquality science and mathematics education for all children. As a result, at theGovernor’s request, the state has developed the Ohio Resource Center forScience, Mathematics, and Reading. In addition, Bridging researchers were asked to do a needs assessment forscience and math by the Ohio Coalition for Mathematics and Science (a businessand industry group). Its database, as discussed above, is a unique resource forstate, national, and international researchers.

Contributions Beyond Science and Engineering

Clearly, one of the most pressingsocial problems today is the persistence of achievement gaps, particularly inmathematics and science, among identifiable subgroups of students. Bridging’s most important contribution is an in-depth,cross-year analysis of the factors contributing either to a gap or helping todiminish any gap. Bridging’seffective dissemination of the results of Ohio’s five systemic reforms of mathand science education also resulted in changes in teacher education programsand teacher licensure. Ohioans know and understand a lot more about effectiveinstruction and the importance of parental involvement and informedadministrators because of this project. And ways to enhance learning have beenidentified for teachers and administrators as well as for university scientists,mathematicians, and science and math educators.