The State of Ohio has mandated a transition from teacher certification to licensure. In response, Miami University is implementing a new Licensure program for preschool through 3rd grade teachers. With the support of the Kellogg Foundation, Miami is using this transition as an opportunity to establish a collaborative campus-and-community-based model for the preparation of those working with young children. Four specific project objectives support this work: reconstructing the curriculum, reconstructing roles, reconstructing organizational infrastructure, and reconstructing policy. Assessment efforts have been of two kinds, process assessment and outcomes assessment. Process assessment determines which practices are being put into place and how effective they are. Outcomes assessment determines how students are changed through learning and experience.
Conclusions from the process assessment suggest that an innovative curriculum has been developed and put into practice. Creating cohesive cohorts and an emphasis on partnerships and field experiences are among the most successful aspects of the new Licensure program. Much of the success of the new curriculum can be attributed to students' field experiences in preschools, primary schools, and early intervention social service agencies. Faculty are engaged in a process of redefining their roles. An important new role is the position of Coordinator. Kellogg Foundation support has made possible several high-quality professional development opportunities that were highly rated and have influenced teaching and professional practice. Strides have also been made toward reconstructing the infrastructure. The educational infrastructure now includes formal links with service learning agencies and strong partnerships with many community-based organizations. Early Childhood Education students have greatly benefited from these partnerships. In terms of reconstructing policy, continuing policy issues include further curricular reforms and the relationship among programs on the three Miami campuses. The interdepartmental and interdisciplinary nature of the new program requires a policy-making apparatus that transcends traditional departmental lines.
The results of outcomes assessment provide solid evidence that the new Licensure curriculum is having a positive impact on student learning. Licensure students scored significantly better than both students in the old Certification program and a comparison group of Miami students on reliable and valid measures of knowledge, the application of knowledge, the ability to articulate knowledge, and attitudes. This superior performance is especially pronounced in the domains of Theories of Learning and Development, Active Learning, Partnerships, Planning Instruction, and Ohio Laws and Regulations. Indeed, Licensure students performed significantly better than the comparison group in each and every domain.
Students in the Licensure program had some serious complaints, but were generally quite positive about the quality of their education. Significantly, these students appeared quite confident in their preparation for a career in early childhood education. Students cited their field experience in schools and agencies as a particularly strong aspect of their education. "Glitches" and uncertainty associated with being the "guinea pig" group were a major source of complaints, and faculty and administrators must continue to "smooth out these wrinkles." Of particular note as a source of confusion are problems of communication within and between departments such as Teacher Education and Educational Psychology. Consistency in staffing, including a commitment to staffing continuing faculty, is also of concern to students. The successful resolution of these issues appears to be essential to the long-term success of the Licensure program.
Both qualitative and quantitative data paint a bright picture of the Licensure program students and the Campus-and-Community-Based Early Childhood Initiative. These students are well prepared for fieldwork and are highly rated by professionals in the field. Students in the new program are demonstratively superior in their relevant knowledge, and their ability to apply and articulate their knowledge, and they have more desirable attitudes. Partnerships between the university and schools and agencies are at the heart of this initiative, and the evidence suggests that these partnerships are paying off for students.
II. Introduction and Overview
The State of Ohio has mandated a transition from teacher certification to licensure. In response to these demands, Miami University is phasing out its teacher Certification program for teachers of Kindergarten through 8th grade and implementing a new Licensure program for preschool through 3rd grade teachers. With the support of the W. K. Kellogg foundation, Miami has undertaken an ambitious early childhood initiative centered on the creation of partnerships with preschools, primary schools, and social service agencies in the community. The overarching goal of the project, and specific objectives of the initiative, have remained unchanged from those originally proposed to the Kellogg Foundation: "To establish a collaborative campus-and-community-based model for the preparation of those who work with young children." Four objectives support this goal, (1) reconstructing the curriculum, (2) reconstructing roles, (3) reconstructing organizational infrastructure, and (4) reconstructing policy.
Appreciable efforts have been made to assess the impact of this new initiative and to incorporate assessment into the decision making process. Assessment activities have been of two general kinds, process assessment and outcomes assessment. The purpose of process assessment is to determine which practices are being put into place and how effective they are. The purpose of outcomes assessment is to determine how students are changed through learning and experience. Thus, as conceptualized here, assessing professional development workshops for faculty falls under the heading of process assessment, and assessing students' knowledge falls under the category of outcomes assessment. Below we will first consider the results of our process assessment with separate sections on each of the four project objectives: Reconstructing Curriculum, Reconstructing Roles, Reconstructing Organizational Infrastructure, and Reconstructing Policy. Next we will consider the results of our outcomes assessment with separate sections on learning outcomes in four interrelated areas: Knowledge, the Application of knowledge, the ability to Articulate knowledge, and Attitudes. This will be followed by sections devoted to outcomes from preschool and social service agency evaluations, and from feedback from student focus groups.
III. Process Assessment
Objective 1: Reconstructing Curriculum
The first cohort of students in the new Licensure program have finished three years of coursework and will begin student teaching in the fall. Students are taught in cohort groups of about 30 students who will earn their degrees by taking a set of courses together and by working collaboratively. Thus, each class year is composed of several cohorts at any given stage in the process. The new Licensure program is designed to meet the need for caring, competent educators in the field of Early Childhood Education who can offer developmentally appropriate experiences spanning preschool and early elementary school.
The restructured curriculum places emphasis on diversity and at-risk children, health and nutrition, family- and community-centered course work, special education and inclusion of children with special needs, development of children aged 3-5, professionalism, and cooperation. There is now an increased emphasis on literacy and language arts. By creating a Licensure program that includes preschool through grade 3 the transition from preschool to grade school is now the focus of learning -- not only for the child but also for the family as a whole.
The creation of the new Licensure program, including the innovative organization around cohorts, has proceeded as planned. A two-year Associate Program in Early Childhood Education has been created and implemented, however, all students must come to Oxford to take the Literacy block and the Content Integration block to complete a four-year degree. New courses such as Families Across the Lifespan and Perspectives in Parenting have strengthened the curriculum.
During the 1998-1999 academic year, four cohorts were accepted into the new program. Of the 120 students form across the three Miami campuses, approximately 8 were Miami University Hamilton (MUH) students, 15 Miami University Middletown (MUM) students, and the remainder students on the Oxford campus. In 1999-2000, five cohorts were accepted totaling 150 students. Approximately 7 were MUH and 20 MUM students. During 2000-2001, another five cohorts were accepted totaling 144 students. Approximately 10 of them started on MUH and 20 started on MUM, with the remainder starting on the Oxford campus.
Other objectives in reconstructing the curriculum included coordinating the programs at Oxford, Hamilton, and Middletown, supporting literacy programs for young children of the Miami Tribe, and increasing service learning opportunities. More representatives from the regional campuses are taking part in Early Field Block meetings and the Advisory Committee. Agency and preschool meetings have been implemented to facilitate communication between Early Childhood faculty, agencies, preschools and students. Literacy programs for young children such as the Miami Tribe initiative and "Reach Out and Read" have continued to thrive. Sue Root, a Regional Service Learning Representative from Alma College, helped Miami obtain mini-grants for additional service learning opportunities, and Early Childhood representation has increased at the annual Service Learning Institute. However, meeting the needs of students at the branch campuses, as well as the main campus, while remaining faithful to the goals and objectives of innovative curriculum reform remains an important issue.
Nine cohorts of about 30 students have finished their first two field experiences with local agencies and preschools between August 1999 and April 2001. Before the first cohort had their field experiences, most students did not have any previous experience with social service agencies. Although they anticipated their placements positively, they were not sure what to expect. When asked prior to the field experiences what they expected to learn, 70% of the first cohort said that they were interested in learning about the overall day-to-day experiences of educators, 43% said that they were interested in learning about the services that local agencies provide for children and families, and 47% were looking forward to having new experiences.
Surveys of the students administered after the first field placement in social service agencies were very positive. When asked to rate their general field experience on a scale from 1 to 5 with 1=very negative and 5=very positive, the mean was 4.38 for the first cohort in the fall of 1999, and 4.41 for the students one year later. When asked "How would you feel about future ECE students visiting this agency?" the mean score was 4.50 for the first cohort, and 4.30 for the next set of cohorts in the fall of 2000. When asked to rate their level of knowledge about social service agencies three months ago (before the course and placement) on a scale of 1 to 10, 1=lowest knowledge and 10=highest knowledge, the 1999 cohort's "before" mean was 3.76. When asked the same question regarding their level of knowledge today their mean was 7.91. The 2000 students' mean was 3.40 for three months ago, and when asked the same question regarding their level of knowledge today the second set of students' mean was 8.25. These results suggest that the students enjoyed their field experiences and increased their knowledge of the day-to-day work of social service agencies.
The second field placements were in preschool settings. Students' comments about the experience were generally quite positive. When asked to rate the experience "in general" on a five point scale where 1=very negative and 5=very positive, students in the spring of 2000 cohort produced a mean rating of 4.73, and one year later students yielded a mean of 4.6. When asked "How would you feel about future ECE students visiting this pre-school" students in the first cohort gave a mean rating of 4.71 and students in the spring of 2001 provided a mean rating of 4.47. Students were also asked to rate their own knowledge about preschools three months ago (before the course and placement) and today. On a ten point scale, with 10 being most knowledgeable, students in the first cohort rated their knowledge three months ago with a mean of 5.74 and their knowledge today rated a mean of 8.87. Students in the 2001 cohorts rated their knowledge 3 months ago with a mean of 5.72 and their knowledge today rated a mean of 8.86. These results indicate that students felt very positive about their preschool placements and believe that they learned from the experience.
Critical Concerns
There have been some difficulties in meaningfully integrating all of the courses into the curriculum and related field experience. Further efforts will be needed to create a more fully integrated curriculum. Many students reported a degree of disorganization in their experience (see below). Needless to say, some "bumps in the road" are to be expected in a new and innovative program.
Conclusions
An innovative curriculum has been created and put into place. Creating cohesive cohorts, and an emphasis on partnerships and field experiences, are two of the most successful aspects of the new Licensure program. The curriculum is meeting the need for early childhood educators who can bridge the gap between preschool and elementary school. These students are well-trained in inclusion, diversity, developmental disabilities, special education, identification of at-risk children, teaching for literacy, assessment, and technology. Much of the success of the new curriculum can be attributed to their field experiences in preschool and elementary settings and early intervention social service agencies. Students clearly enjoyed learning from their field experiences.
Objective 2: Reconstructing Roles
The important issues in Reconstructing Roles include capacity-building, professional development, recreation of formal and informal roles and the creation of new roles. In the 2000-2001 academic year the position of Coordinator was created by the Department of Teacher Education and filled by Dr. Ed Jones. The Coordinator works to facilitate communication and collaboration among faculty and students. However, the Coordinator does not have line authority, and the support of relevant department chairs and the dean remains untested. Other key initiatives to support the reconstruction of roles include putting more Early Childhood educators into collaborative roles with the Butler County Childcare Coalition and efforts led by Bobbe Burke assisting new faculty in collaborating with agencies and schools to increase service learning opportunities. Early Childhood faculty have taken part in numerous conferences with organizations such as NAEYC (The National Association for the Education of Young Children), ODE (The Ohio Department of Education), NNER (The National Network for Educational Renewal), NCCCC (The National Coalition for Campus Children's Centers), and BCCCC (The Butler County Childcare Coalition). These state and national conferences and presentations have provided the opportunity for faculty to share ideas with others and to build upon their present knowledge.
Kellogg foundation support has made several professional development sessions possible for Miami University faculty and students, as well as community partners including local teachers and social service agency professionals. For example, the Lillian Katz presentation "The Project Approach" was supported by Kellogg funds. Assessment results from 62 participants (Miami faculty, students, and social service agency professionals) were very positive. On a scale from 1=excellent to 5=poor, participants responded to the question "My overall impression of this early childhood professional development activity is:" with a mean of 1.63 (SD=.81). When asked to respond to the question "Ideas from today's session will influence how I do my work in early childhood education; I will use information and materials from today's session," participants responded with a mean rating of 2.05 (SD=.87) on a scale from 1=to a great extent and 5=not at all.
Another example is the presentation "Reading, Writing, Research and Reality" by Kathy Roskos. This presentation provided information on the development of literacy in young children. Assessment results from 37 participants were very positive. On a scale from 1=excellent to 5=poor, participants responded to the question "My overall impression of this early childhood professional development activity is:" with a mean of 1.65 (SD=. 63). When asked to respond to the question "Ideas from today's session will influence how I do my work in early childhood education; I will use information and materials from today's session," participants responded with a mean rating of 1.97 (SD=.76) on a scale from 1=to a great extent and 5=not at all.
Similarly, the Content Integration and Literacy Workshop during January 2001 was attended by 21 teachers and 10 Miami faculty currently teaching in the content and literacy blocks. Participants rated the workshop on a 5 point scale from 5=excellent to 1=poor, and the Workshop received a mean rating of 4.21 (SD=.60) for overall quality. When asked whether "Ideas from today's session will influence how I do my work in early childhood education; I will use information and materials from today's session," Miami faculty rated the workshops at 4.33 (SD=.72) and teachers 3.47 (SD= .91) on the same 5 point scale.
During the Summer of 2001, a three session Early Childhood Teaching Workshop on active learning was available to Miami University students, faculty and community partners. The workshop was specially tailored to the interests of community partners who had been surveyed earlier that spring. Sessions on "Play and Active Learning" were conducted by Doris Bergen and Ann Wagner, "Scaffolding Active Literacy Learning to Meet Proficiency Standards" was conducted Deborah J. Leong, and "Active Learning in Mathematics: Connecting to Proficiency Standards" was conducted by Pamela Termeer. These presenters were described by participants as "well-prepared, knowledgeable, and personable. Excellent!" On a 5 point scale with 5=excellent to 1=poor, these workshop leaders were rated 3.714 (SD=0.49).
In April, 2001 Kellogg funds supported a new approach to strengthening professional development and partnerships with regional schools. An Interactive Videoconference Workshop among Miami faculty and teachers at schools that had hosted Content Integration and Literacy Block students provided opportunities for reflection and assessment, and also helped communicate faculty's expectations for students. The teachers watched Miami students present Literacy Block and Content Integration Block projects. This provided the educators with a better understanding of the expectations for ECE students. Several teachers reported taking away ideas on activities for their own classrooms, and were grateful for the chance to voice their opinions, and express any concerns, before receiving the students for their field experience. Afterwards, teacher and Miami faculty from the Content Integration and Literacy Blocks met concurrently to reflect upon and assess the field experiences of Spring 2001. When asked to rate the Interactive Videoconference Workshop on a 5 point scale with 5=to a great extent, and 1=not at all, participants' mean rating of their overall impression of the workshop was 4.14 (SD=.66). When asked to rate individual student presentations on the extent to which they "conveyed the kinds of expectations that Miami faculty have for their students" the mean ratings of presentations ranged from 4.67 to 4.76 on the same 5 point scale. Finally, when asked to rate the reflective discussions on providing "information that I as a collaborating teacher can use to support Miami students," teachers gave a mean rating of 4.56 (SD=.62).
These results, and similar feedback from several other professional development opportunities, indicate that Miami faculty and community partners enjoyed several high-quality professional development opportunities. Many of these workshops were led by nationally recognized experts in their fields. Participants indicated that these experiences were important and influenced their teaching and professional practice.
Critical Concerns
Lines of communication among Early Childhood educators have been facilitated by the creation of the Coordinator position, and meetings of the early field block faculty, the literacy block faculty, the content block faculty, the Advisory Committee, and the Kellogg Coordination and Planning Committee. However, as discussed in greater detail below, communication among all of the relevant parties remains a problem.
Faculty are in the process of successfully reconstructing their professional roles. This process has been facilitated by a series of influential and high caliber professional developmental opportunities supported by the Kellogg Foundation. However, inconsistencies in staffing (discussed in greater detail below) severely undermine these efforts. For example, in the fall of 2001, we anticipate 10 faculty teaching in the Literacy Block. Of these only four (4) will be full-time continuing faculty, with the other six (6) being part-time faculty who are not tenured or tenure track.
The labor-intensive nature of organizing the field experience is significant, and the potential for faculty burnout is a real concern. We believe that this successful aspect of the new curriculum would not be sustainable without the Coordinator. Faculty and administrators must continue to recognize the additional tasks associated with the new curriculum and clearly and unambiguously support the Coordinator in organizing faculty.
Conclusions
Faculty (and to a lesser extent teachers and agency professionals) are engaged in a process of redefining their roles. An important new role is the position of Coordinator that was recently filled by Dr. Ed Jones. Kellogg foundation support has made possible several high-quality professional development sessions for Miami University faculty and students, and community partners. Participants repeatedly indicated that these experiences were very positive and influenced their teaching and professional practice. Great strides have been made in bringing about the necessary changes in roles. However, a lack of communication among all of the relevant parties, and inconsistencies in staffing, undermine these efforts.
Objective 3: Reconstructing Organizational Infrastructure
The early childhood initiative has led to the creation of partnerships drawn from various organizations. At Miami University these partners include departments and programs in the School of Education and Allied Professions, administrative services, regional campuses, the Center for Human Development, Learning, and Teaching, and the Institute for Educational Renewal. Community-based organizations include preschools, early childhood education centers, child care centers, social service agencies serving children and families, family resource centers, and elementary schools. Partners based at Miami University hold lead responsibilities for preparing future early childhood educators, health and human service workers, and child and family caregivers. Partners in community-based early childhood centers and schools hold lead responsibility for the education and healthful development of young children. Partners based in agencies and family centers hold lead responsibility for providing support services to children and families. As collaborating partners, all are learning to share responsibility for helping each other improve the quality of practice for both pre-service and in-service practitioners.
Reconstructing the infrastructure entails enhancing formal links for service learning, developing early childhood partner sites, maintaining and expanding collaborative relationships with other Early Childhood groups in the region, making and implementing governance decisions, and developing and field-testing cohorts in their field experiences. With funding provided by the W. K. Kellogg Foundation, Miami University has forged strong partnerships with literally dozens of community-based organizations including preschools, primary schools, and social service agencies. As described in greater detail elsewhere, Miami Early Childhood Education students have greatly benefited from these partnerships. For example, in the spring of 2001, Miami faculty visited agency sites during the students' field experience to pinpoint any problem areas and further strengthen the partnerships. Notes from each meeting indicate that the field experiences were going well and that the agencies would eagerly welcome ECE students in the future.
Miami faculty have begun to establish a more coherent and holistic approach to each of their classes by including aspects of child development, inclusion, exceptionalities, case studies, co-operative learning, and authentic assessments into their classes. Specific instances of reconstructing the organizational infrastructure include the creation of a more interdisciplinary curriculum. Departments such as Family Studies and Social Work, Music, Physical Education, and others have collaborated to provide a more comprehensive curriculum for the Early Childhood licensure. However, as evidenced by difficulties in communication discussed elsewhere, additional work in reconstructing the organizational infrastructure may be warranted.
Critical Concerns
Teaching teams are routinely completed too late for advanced planning and organization. For example, in the fall of 2000, the late assignment of faculty to the Early Field Block team (the last member being appointed in August) hindered collaboration. In the future, teaching teams should be assigned much earlier, and kept in place for more than a single academic year.
Participation in the Early Childhood Education team and block meetings has been mixed. Faculty in the departments of Teacher Education and Educational Psychology have been more regular in their attendance than those from other departments. This is a case where the support of the deans and relevant department chairs for the Coordinator may be especially helpful.
By all accounts, the current instructors have performed very well. However, having relatively few tenured or tenure track faculty involved threatens continuity from one year to the next, and works against developing institutional memory.
Conclusions
Funding provided by the Kellogg Foundation has facilitated a reconstruction of the infrastructure. The educational infrastructure now includes formal links with service learning agencies and early childhood partner sites. Miami University has forged strong partnerships with many community-based organizations. Early Childhood Education students have greatly benefited from these partnerships. Creating a coordinated system for forming successful teaching teams drawn from different departments remains a critical concern. Teaching teams must be formed in a timely fashion and maintained for more than one academic year.
Objective 4: Reconstructing Policy
Some specific local policies, as well as regional and national policies, are in need of reform. One objective is encouraging awareness of and participation in state, regional and national conversations. Another is involving a wider variety of voices in policy discussions and selecting priorities to support collaborative efforts. Miami faculty have taken part in policy meetings with outside agencies such as Hamilton County Head Start, Warren County Children First Council, ODE, OHEC (The Ohio Higher Education Consortium), NAECTE, NAEYC, and BCCCC (The Butler County Childcare Coalition). These faculty also serve on daycare advisory councils, professional development committees and the like. There appears to be a dearth of faculty at Miami University who specialize in early childhood.
Critical Concerns
Miami is in the process of hiring new faculty who may (among other things) teach in the new curriculum. It may be desirable to have a broader constituency involved in recruiting and hiring new faculty with the aim of selecting individuals who are focused on early childhood and particularly well suited to teaching in the new curriculum.
Recently, there has been a good deal of "turnover" at Miami in the office of the Dean of the School of Education and Allied Professions, the Provost's office, and the President's office. There is a critical need to include the Dean (and occasionally even the Provost) in key policy decisions, and their support of the Coordinator is essential. One important policy issue is the relationship between programs at the main and branch campuses. Decision makers must balance the goals of having branch campus students "feed into" the four year program at Oxford without weakening the pedagogical integrity of the four year program, or undermining its most innovative aspects.
Conclusions
Miami faculty have been active in the policy arena at national, state, and local levels. For the new Licensure program, continuing policy issues include further curricular reforms and the relationship among programs on the Oxford, Hamilton, and Middletown campuses. The interdepartmental and interdisciplinary nature of the new program calls for a policy making apparatus that transcends traditional departmental lines. We encourage the Coordinator, department chairs, and dean's office to work together to facilitate inclusive and coordinated policy making.
IV. Outcomes Assessment: Student Performance and Feedback
To assess the strengths and weaknesses of the new Early Childhood Education Licensure program, we collected data about student outcomes, and feedback about student performance, from several sources. The first of these sources is a set of four instruments measuring students' knowledge, students' ability to apply their knowledge, students' ability to articulate their knowledge, and students' attitudes. The second of these sources is feedback about students' field work from agency professionals and preschool teachers. This feedback includes quantitative ratings as well as written feedback. Finally, students themselves offered feedback in the setting of focus group sessions conducted with students in the old Certification program and the new Licensure program. The results of these efforts are described below.
Learning Outcomes Assessment
In the spring of 2001 the last group of students in the old Certification program were completing their coursework prior to student teaching. At the same time, the first cohort of students in the new Licensure program were completing their coursework and field work prior to student teaching. This provided a natural opportunity to compare the two groups on a number of outcome measures, and to conduct focus group interviews to learn about their educational experiences.
We developed pencil and paper instruments to measure learning outcomes in four interrelated areas: Knowledge, the Application of knowledge, the ability to Articulate knowledge, and Attitudes. In consultation with faculty, we identified fourteen (14) outcome domains that were investigated with each instrument. Sources for these domains include the Praxis III Professional Assessment Criteria, curriculum objectives, and previous research on training childcare providers (see Wolfe, Wang, Bergen, and Colquhoun, 1999-2000). These domains are Active Learning, Meaningful Learning, Partnerships, Nurturing Relationships, Organizing Content for Student Learning, Creating an Environment for Student Learning, Teaching for Student Learning, Teacher Professionalism, Evaluation/Assessment, Inclusion and Diversity, Ohio Laws and Requirements, Behavior Management and Discipline, Theories of Development and Learning, and Planning Instruction and Strategies.
We developed a large number of Knowledge, Application, Articulation, and Attitude items within these domains (n=167 items total) and participants rated each item on a 5 point agree-disagree scale. We received feedback from faculty about this set of items, and revised the set prior to administration. We used students enrolled in a Social Psychology course as a comparison and development group. This group was selected because these College of Arts and Sciences students are generally bright, interested in human behavior, but not specifically trained in early childhood education. Following a procedure developed by Wolfe, Wang, & Bergen (1999) knowledge, application, and articulation, as well as attitudes, were measured with agree-disagree items. We pre-tested these items on the Social Psychology students (n=30) and selected a subset of items with the best psychometric properties.
This procedure yielded a 22 item Knowledge instrument. An example of a knowledge item in the domain of Evaluation and Assessment is "Portfolio assessments are mainly a means of assessing older students." The reliability of the Knowledge instrument was measured by a satisfactory Cronbach's Alpha of .82.
The Application instrument consisted of 21 items. An example of an application item in the domain of Ohio Laws and Regulations is, "Miss Dana knows that one of her students has not had the required vaccinations for school because the family does not believe in Western Medicine. Miss Dana needs to respect the family's decisions, and keep this confidential." The reliability of the Application instrument demonstrated by a Cronbach's Alpha of .84.
A brief, 10 item instrument was used to measure Articulation. An example of an Articulation item from the domain of Theories of Development and Learning is, "I can't really put into words what I know about development." A Cronbach's Alpha of .69 is acceptable but lower than ideal.
Attitudes were measured with a 29 item instrument. An example of an Attitude item in the domain of Inclusion and Diversity is "I feel that laws requiring 'inclusion' in the schools are politically correct nonsense." A Cronbach's Alpha of .86 indicates that this is a reliable instrument.
Items from all four instruments were presented in a mixed, randomly selected order. High numbers on the 1-5 scale were associated with correct responses (and positive attitudes) on approximately half of the items. The content validity of the instruments was established through feedback from program faculty. An additional measure of predictive validity is the superior performance of the education students from both programs over the Social Psychology Comparison group (see below). These instruments may be of use in future assessment research.
The instruments were administered by the authors in separate required classes for both the Certification and Licensure students on April 17, 2001 after the professor had left the room. Participants were assured that their responses would be anonymous and confidential and that the results would not be shared with faculty until after the end of the semester. Thirty-four (34) Licensure students and 17 Certification students participated.
Results from the Knowledge instrument indicated that Licensure students scored significantly higher than the Certification students, and that both of these groups of education students scored significantly higher than the Social Psychology comparison group, F (2,76)=20.427, p<.0001. As Table 1 indicates, all three groups scored high in absolute terms. However, Fisher's PLSD test revealed significant differences among groups.
Table 1. Knowledge scores by group.
Results from the Application of knowledge instrument indicate that the Licensure group was significantly higher than both the Certification and Comparison groups, F(2,73)=10.15, p<.0001. As indicated in Table 2, the Certification group was slightly, but not significantly, higher than the Comparison group, Fisher's PLSD p>.09.
Table 2. Application scores by group.
Results from the Articulation instrument indicated that Licensure students scored significantly higher than the Certification students, and that both of these groups of education students scored significantly higher than the Social Psychology Comparison group, F (2,76)=15.397, p<.0001 (see Table 3). Fisher's PLSD test revealed that these differences are statistically significant.
Table 3. Articulation scores by group.
Results from the Attitude instrument indicated that Licensure students scored significantly higher than the Certification students, and that both of these groups of education students scored significantly higher than the Social Psychology Comparison group, F (2,76)=24.165, p<.0001 (see Table 4). Fisher's PLSD test revealed that these differences are statistically significant.
Table 4. Attitude scores by group.
Taken collectively these results provide solid evidence that the new Licensure curriculum is having a positive impact on student learning. Licensure students scored significantly better than both students in the old Certification program and a Comparison group of Social Psychology students on reliable and valid measures of Knowledge, the Application of Knowledge, the ability to Articulate knowledge, and Attitudes. Of course, students were not randomly assigned to groups, so other factors may account for these findings. Attempts were made to collect data from Certification students who were already student teaching, however, responses were returned from only nine of these students. Interestingly, these nine students scored significantly lower than the Certification students currently taking classes. This appears to contradict an interpretation of these results based on the peculiar characteristics of this particular group of Certification students.
There were too few items to create reliable sub-scales for each of the 14 domains on each of the four instruments. Collapsing across each of the four instruments creates a set of 14 relatively crude diagnostic instruments to investigate which domains are associated with improvement. The Licensure group scored significantly higher than the Certification group on Active Learning, Partnership, Ohio Laws and Regulations, and Planning Instruction, and both groups scored significantly higher on these measurers than the Comparison group. The Licensure group scored significantly better than both the Certification and Comparison groups on Theories of Learning and Development, and there were no significant differences between the Certification and Comparison groups on this measure. Both the Licensure and Certification groups scored significantly higher on Meaningful Learning, Organizing Content, Creating an Environment for Learning, Teaching for Student Learning, Teacher Professionalism, Evaluation and Assessment, Inclusion and Diversity, and Behavior Management, with no significant differences between the Licensure and Certification groups on these measurers. Licensure students scored significantly higher than the Comparison group students on Nurturing Relationships without any other significant differences. Taken collectively, these results indicate that the Licensure students performed better than the Comparison group in each and every domain. The Licensure program appears to be more effective than the old Certification program in preparing students in the domains of Active Learning, Partnerships, Ohio Laws and Regulations, Planning Instruction, and Theories of Learning and Development.
Preschool and Social Service Agency Evaluations of Students
In 2000, and again in 2001, students spent an average of 4 days working at one of 21 social service agencies in southwest Ohio. These agencies include Catholic Social Services, Early Start, Oxford/Talawanda Community Services, and Warren County Family and Children First, among others. At the end of this field experience professionals at the agencies rated the students on a number of dimensions pertaining to their personal and professional qualities. The category Personal Qualities is a composite of separate ratings of enthusiasm, flexibility with responsibility, creativity, self-motivation, rapport with staff, rapport with clients, and professional appearance. Students were rated on a 4 point scale with 1 = needs help, 2 = acceptable, 3 = good, and 4 = outstanding. The category Professional Qualities is a composite of separate ratings of awareness of professional role, organizational ability, communication skills, ability to utilize suggestions/criticisms, and self-evaluation and reflection.
Numerical ratings, and written comments, were generally positive. In 2000, students received a mean rating of 3.48 on personal characteristics, and a mean rating of 3.13 on professional characteristics. In 2001, those ratings improved to 3.62 (SD=.51) for personal characteristics and 3.65 (SD=.52) for professional characteristics. A pair of t-tests reveal significant improvement from 2000 to 2001 on both personal and professional qualities. For personal qualities T(61) = 1.90, p<.05. For professional qualities T(58) = 3.93, p<.001. These ratings suggest that the students in the Licensure program are well-prepared for their field placements with agencies, and that their preparation is improving. We speculate that the improvement is due to "working out the bugs" in the groundwork faculty and students did in anticipation of the field experience.
In the fall of 2000 students completed a week-long field placement with preschools in the region. The students were rated by preschool teachers on personal and professional qualities employing the same instrument used by agency professionals described above. Using a 4 point scale, students earned a mean rating of 3.62 (SD=.51) on Personal Qualities, and a mean rating of 3.56 (SD = .52) on Professional Qualities. These ratings suggest that the students were also well prepared for their preschool placements.
The Student Experience: Feedback from Focus Groups
Purpose and Method
Separate focus group interviews were conducted with students in the Licensure and Certification programs to gain insights into their experiences. We conducted the interviews in two separate required courses after the professor had left the room. Seventeen (17) Certification and 34 Licensure students participated. Students were assured that their responses would not be shared with faculty until after the end of the semester and that individuating information would be omitted from the final report. The focus group sessions were tape recorded and transcribed. Our general strategy was to ask open-ended questions such as "How do you feel about your experience in this program?" and to encourage students to "play off" one another's responses. A close reading of the transcripts reveals several themes that we have organized under 8 headings: Disorganization and Confusion, Communication Between Departments and Faculty, Faculty Staffing, Special Education, Program Requirements, Field Experience and Partnerships, Preparation for Teaching, and Advantages of the Programs. The students seemed appreciative of the opportunity to provide feedback and "blow off steam."
Disorganization and Confusion
Students in both the Licensure and Certification programs expressed concerns about disorganization and uncertainty about program requirements and what they were "supposed to be doing next." Much of the confusion in the new Licensure program was attributed to being the first cohort to go through the program. The phrase " guinea pigs" was used by several students to describe their feeling of being experimental subjects, and sometimes with little patience. However, there were also concerns that things were not improving fast enough. For example, one student in the Licensure program said,
My roommate's sister is three semesters behind us and it hasn't gotten any better for her, and she has the exact same problems as we do. It's frustrating to us because we're told that we are the guinea pigs and we'll try this out and get your feedback and fix it, well, 3 semesters later it's still the same.
I agree that we are getting the shaft, and they are trying to figure out what works and figuring out a lot of stuff that doesn't work, but I think it's going to take over a couple years to change.
Concerns about confusion were not limited to the Licensure group. Indeed students in the Certification program expressed even more frustration. For example, one Certification student reported
I had to go to the dean yesterday because nobody else in the building knew how to tell me how to register for student teaching, like it doesn't make sense that no one knows what's going on, like everyone gives you the run around.
Or you call five different people, It's like for some majors I know, like pre-Med, have a booklet and it tells them what they need to do each semester and it has when they need to take the MCAT, and when they need to sign up for classes, when they have to have these forms turned in, and we have to have a lot of stuff like that turned in by the time we graduate, but no one knows when.
Comments such as these indicate that students in both groups experienced some chaos and uncertainty in their programs. By in large, Licensure students attributed the confusion to growing pains and were generally forgiving, if annoyed. Certification students were less benign in their attributions, and suggested that faculty indifference or ignorance was sometimes at fault.
Communication Between Departments and Faculty
Students in both programs identified a lack of communication between faculty and staff in the departments of Teacher Education and Educational Psychology as a major source of confusion. For example, a student in the Licensure program said,
I think that part of the problem is general planning. I think it's the two departments that aren't talking to each other (EDP and EDT). I don't think that they are really sitting down and talking as a group… every time I call one department they say they don't know what to do, you need to talk to the other department, and the other department says the same thing, I think that we are just stuck in the middle and we are having to answer to two of the departments, and I don't think they are really talking to each other.... I told my sister who was talking about coming to Miami for regular education major to look at other colleges.
I think, there are different departments in the building, Teacher Education, EDP, but they're both...you have to take classes from each department, and you never know who to talk to for both. Do you go to the student teaching office, there are all these different offices and if you call this office and they don't know and say oh well you have to call them, and you say I just called them and they told me to call you, I mean, seriously you will get that back and forth.
Faculty Staffing
Students in the Licensure program expressed concerns about the stability of faculty in required courses. For example, one Licensure student said,
They can't seem to staff it appropriately, like the credit hours and we are getting a totally different experience than everybody else and then those next people are going to get a different experience and so on….there's no consistency in staffing.
Special Education
Several students in the new Licensure program expressed concerns about the Special Education specialization. Issues raised include whether Special Education students should have their own cohort, and whether the requirements are appropriate. For example, one Special Education student in the Licensure program said,
There are only 7 of us in Special Ed in the new 'guinea pig' program that we have going on here and if they are going to have Special Ed in this program then it should NOT be mixed with regular ed. And if it is then the regular Ed should have to take what we have to take or we should be double majoring because I feel way more prepared for regular Ed than for Special Ed and we just got done with our method block and you guys have seven classes in your method block and we have 2 and yet we are expected to do everything that you are expected to do plus the 6 other classes outside of those classes so I think that if they are going to do something like that then they need to take that into account, like put us in a different cohort all together because the mix isn't working and it's way more stressful on the seven of us besides the fact that we always get singled out.
To go along with them, I also feel like all the Special Ed majors have had no experience in the classroom. We have to go out with the regular Ed students, and you're not even guaranteed to get into the area that you want. If I'm mild/moderate I want to be in a mild/moderate classroom…
I think that Special Ed is more a philosophical problem in the program because they basically put us in with regular Ed because there is a big push for inclusion now and you need to know how to work with regular ed. teachers and you need to be exposed to this curriculum because you are going to be working with them, but the problem is that until we have established our own curriculum and until we know how we are going to work with our own students there is no way for us to collaborate intellectually with other teachers because we haven't established the basis for that.
This line of discussion suggests that students are cognizant of some of the pedagogical and philosophical issues with which professionals in the field are grappling. However, these concerns also suggest that it may be advantageous for faculty to give further thought to the best approach to Special Education. In any case, it is advisable for faculty to place greater emphasis on explaining to students the rationale for curricular decisions.
Program Requirements
A few students in the Licensure program took exception to the rationale for specific required courses. For example, one Licensure student said,
I feel like it could have less classes. I don't see why I need three health classes, I am bummed about it. I thought when I came to college, I am majoring in education, but I thought I would be able to take classes outside of education and that really bothers me, and I won't be able until second semester next year and that's only one, so that's frustrating.
Field Experience and Partnerships
Licensure and Certification students differed dramatically in their evaluation of their field experience, and their feeling of preparation for student teaching. Students in the Licensure program were widely enthusiastic about their experience in the field. For example, one Licensure said,
I think that one of the big advantages of being in this new program over the old program is our early and multiple experiences in the field because before in the old program they only had two weeks. The semester before they student taught was their only experience in the field before student teaching and we had the opportunity as early as sophomore year.
I started here as a freshman when the new program was just beginning, and that's really hard because every class I take now, we're the guinea pigs, so lets just try something new on you and if it doesn't work then so be it. And while it's been really frustrating, the one thing I have to say is that we've gotten to see all the different (field) settings, all the different levels and I've been doing that since I was a freshman.
I kind of feel shaky, now I feel a little better, but I'm not sure how much you guys know about it but this it the first semester that we've actually been able to get out in the field, get our feet wet, and interact with kids, and it's kind of scary because I'm out there thinking, oh my god I'm gonna have to do this for the rest of my life and I'm not sure it's what I want to do, so it's sort of scary that way.
Preparation for Teaching
Perhaps the most striking difference between the Certification and Licensure groups was their assessment of their preparation for a career in early childhood education. In part because of their field experience, the Licensure students appeared much more confident in their training. For example, one Licensure student said,
I know a lot of our classes may seem like we didn't learn a lot, I think that after going through this I feel more prepared to teach than, say my roommate whose in the certification program and it seems like I've taken a lot more from these three years than she's taken from five and there has to be someone first to go through it and I think that the licensure program has prepared us.
On a positive note, as unprepared as everyone thinks they are, I have seen people in this class and lesson plans they have put together, and I have also been out there in the field, and I think that you guys are on top of everything and are doing great stuff, especially compared to teachers I've seen out there who have all this experience and I think that everyone in here can do a better job than some I've seen out there. I don't know compared to other schools, I think this is a great school and the education program has some problems not because of the transition, but I think that all in all, we are getting a good education.
By way of contrast, the Certification students indicated that they were less confident in their preparation. For example, one Certification said,
(In) every class that we have taken up to now we were expected to do everything we are learning to do now, I don't know, like the lesson plans, we still had to do lesson plans, we were just never taught how… at least I never did it in any of my classes... I agree with them that this is way too late in our program, I mean this is our fourth year…
I think that there is nothing that is going to prepare you for your own classroom than being out in the field. It's the best way to prepare you. Reading about it in a text book and reading about how to teach social studies and how to teach reading, is not going to teach you more than actually being out there, thrown into the classroom, and having your students there…
Advantages of the Programs
Both the Licensure and Certification students expressed a preference for their respective programs. However, students in the two groups cited different reasons. Students in the Licensure group generally cited satisfaction with the quality of their education. For example, one Licensure student said,
I'm not a special education major, so I think that they have issues that I don't know enough about to comment on, but for the regular education students I feel that we are very well prepared and I think that the program is a good program. I think there are problems with it, but I think that there are probably problems with every program in this school. Nothing is perfect, but in general I think that we have had a lot of field experiences, we have developed a unit, we have done things that we all came back after the field and said we would never do this and we want to do this, I would change this if it was my own class, and I think we have learned, and I think that in general it is a good program, but there are glitches like in scheduling and teachers and professors, but in general (it's a good program).
Certification students expressed a preference for their program, primarily because Certification had given them greater flexibility in their employment. When asked, "Do you wish that you were in the new program?" one student said, "I just want to be done, but, probably, yes." Another Certification student replied, "No. I want to be K-8. I am happy with K-8 and I want to teach all four subjects." A third student elaborated,
We (our certification group) have the option whether I want to teach 1st grade one year and then 8th grade, and I decide I want to get a different job, I decide that that's what I want to do and I can go to 8th and that's not a problem.
I don't understand the whole state thing because there is such a need for teachers out there and they change the whole program and say that you need to be concentrated in a certain level.
Needless to say, these changes in Ohio law are the responsibility of the state legislature, and not Miami University. The favorable comments about the quality of the Licensure program were echoed by several students and seemed well-received by the group. The Licensure students put the "glitches" in context and expressed appreciation for their professional education.
Conclusions
"Glitches" and uncertainty associated with being the "guinea pig" group were a major source of complaints from the Licensure students, and faculty, staff, and administrators must continue to "smooth out these wrinkles." Of particular note as a source of confusion are problems of advising and communication within and between the departments of Teacher Education, Educational Psychology, and other relevant programs. Consistency in staffing, including a commitment to using continuing faculty, is also of concern to students. The successful resolution of these issues appears to be essential to the long-term success of the Licensure program. The focus group interviews also suggest a need for faculty to be more thorough in explaining to students the rationale for curricular decisions. This need is especially keen in the area of special education, and a further consideration of the special education curriculum may be warranted.
Students in the Licensure program had some serious complaints, but were notably positive about the quality of their education. Significantly, these students appeared quite confident in their preparation for a career in early childhood education. The students cited their field experience in schools and agencies as a particularly strong aspect of their education. This suggests an appreciation for the value of the partnerships between the university and schools and agencies formed with the support of the Kellogg foundation.
Student Performance and Feedback: Conclusions
These qualitative and quantitative data paint a bright picture of students in the new Licensure program and the Campus-and-Community-Based Early Childhood Initiative. These students are well-prepared for field work in preschools and social service agencies, and are highly rated by professionals in these settings. Students in the new program are demonstratively superior in their relevant knowledge, and their ability to apply and articulate their knowledge, and they have more desirable attitudes than students in the old Certification program and a comparison group from the College of Arts and Sciences. This better performance is especially pronounced in the domains of Theories of Learning and Development, Active Learning, Partnerships, Planning Instruction, and Ohio Laws and Regulations. Although students voiced some serious concerns they were positive about the quality of their education and confident in their preparation for a career in early childhood education. They referred to their field experience in schools and agencies as an especially strong aspect of their education. Of course, partnerships between the university and schools and agencies are at the heart of the Campus-and-Community-Based Early Childhood Initiative supported by the Kellogg foundation.
V. Conclusions
Process and outcomes assessment together inform a comprehensive understanding of the Early Childhood Initiative. An innovative curriculum has been developed creating cohesive cohorts and emphasizing partnerships with organizations in the field. Much of the success of the new curriculum can be attributed to students' field experiences in preschools, social service agencies, and primary schools during the literacy and content integration blocks. Several high-quality professional development opportunities have helped faculty redefine their roles. The educational infrastructure now includes links with service learning agencies and strong partnerships with many community-based organizations. Early Childhood Education students have greatly benefited from these partnerships. Continuing policy issues include further curricular reforms and the relationship among programs on the three Miami campuses.
The results of outcomes assessment indicate that the new Licensure curriculum is having a positive impact on student learning. Licensure students scored significantly better than other students on measures of knowledge, the application of knowledge, the ability to articulate knowledge, and attitudes. This superior performance holds in every domain measured, and is especially strong in the areas of Theories of Learning and Development, Active Learning, Partnerships, Planning Instruction, and Ohio Laws and Regulations. Students in the Licensure program registered some serious complaints, but were generally very positive about their education. Students appeared quite confident in their preparation for a career in early childhood education and cited their field experience in schools and agencies as a particularly strong aspect of their education. The data support these self-assessments and suggest that the partnerships at the heart of the initiative are paying off for students.
References
Wolfe, C.R., Wang, A. & Bergen, D. (1999). Assessing the Winning Teams Program of Interactive Satellite-Based Training. Behavior Research Methods, Instruments, & Computers, 31, 275-280.
Wolfe, C.R., Wang, A., Bergen, D., & Colquhoun, D. (1999-2000). The "Winning Teams" model of interactive satellite-based training: Application and evaluation. Journal of Educational Technology Systems, 28, 185-207.