THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY 

United States results from the achievement test for population 2 (13 year olds) for the Third International Mathematics and Science study were released at a National Center for Education Statistics press conference in Washington, D.C. on November 20 at 11:00 AM. Speakers included Secretary of Education Richard Riley; Pascal Forgione, Jr. Commissioner of Education Statistics, National Center for Education Statistics; Neal Lane, Director of the National Science Foundation; and Bruce Alberts from the National Academy of Sciences. A summary of the report appears in this newsletter. This work was supported by National Center for Education Statistics and the National Science Foundation.

Earlier in the day at 10:00 AM, international achievement results for the same population were released at a press conference at Boston College with presentations from Tjeerd Plomp, Chairman, International Association for the Evaluation of Educational Achievement (IEA) in Amsterdam, sponsors of the TIMSS; Albert Beaton, TIMSS International Study Director; and Michael Martin, TIMSS International Deputy Study Director. Ina Mullis, CO-Deputy Study Director gave the opening remarks. The two reports: Mathematics Achievement in the Middle School Years: IEAís Third International Mathematics and Science Study and Science Achievement in the Middle School Years: IEAís Third International Mathematics and Science Study can be obtained at the cost of $30.00 each (including U.S. domestic postage and handling) by mail: TIMSS International Study Center, CSTEEP, Campion Hall 323, Boston College, Chestnut Hill, MA 02167 or by fax: 617-552-8419; telephone: 617-552-4521; email: timss@hermes.bc.edu with Visa and MasterCard. The reports are also available on the World Wide Web at:
http://wwwcsteep.bc.edu/timss.

Achievement results for populations 1 and 3 (9 year olds and end of secondary schooling) will be released in 1997.

Data from forty-one countries are included in this report which presents the mathematics and science performance of population 2 students. Thirteen-year-old students comprise TIMSSí population 2 and testing within each country involved the two adjacent grades containing the majority of these students. Students in the upper grade are the focus of TIMSS analyses and reports and, in virtually all countries, these students were in either the seventh or eighth year of schooling. In the U.S., the TIMSS focus is on students in grade 8. In all 41 TIMSS countries, students took a single test containing both mathematics and science questions. Students responded to three types of items for both subjects: multiple choice, short answer, and extended response. Approximately 80% of the mathematics items and 75% of the science items were multiple choice. Nationsí scaled scores were computed using information across all item types.

How well did U.S. students do on the TIMSS test?

In general the performance of US students was disappointing; the average score of U.S. eighth grade students was somewhat below the international average in mathematics and somewhat above the international average in science. Figures 1 and 2 show how U.S. students did in mathematics and science compared to similar-aged students in 40 other countries.

A single score for any nationís average student performance suggests a degree of precision that is unwarranted. The scores reported in Figures 1 and 2 were obtained by sampling students within each country in order to create a reasonable estimate of a nationís average performance. These estimates represent a range within which the nationís actual average would most likely fall if all students were tested. Therefore, comparing U.S. scores to those of other countries is best presented using three broad categories: those that performed significantly better than the U.S., those that
 
 

did not perform significantly different from the U.S., and those that performed significantly lower.

As can be seen in Figure 1, the mathematics average student performance in 20 countries was significantly better than the average performance of U.S. students and in 13 countries was not significantly different from that in the U.S. The U.S. average student performance in mathematics was significantly better than that of only 7 nations. These include Lithuania, Cyprus, Portugal, Iran, Islamic Republic, Kuwait, Colombia and South Africa. Comparisons of the average performance in science looked somewhat different. Figure 2 shows that 9 nations performed significantly better than the U.S. while the performance of another 16 was not significantly different. The average U.S. student performance in science was greater than that of 15 nations.

Items on the TIMSS test were grouped according to six broad mathematics content areas and five broad science areas. Figures 3 and 4 present the average percent correct obtained by students in each TIMSS country for these content areas. In mathematics, the international rank of U.S. studentsí average percent correct is stronger in the areas of ëfractions & number senseí, ëdata representation, analysis, & probabilityí, and ëalgebraí than it is in the areas of ëgeometryí and ëmeasurement.í In science, U.S. studentsí international standing is stronger in the areas of ëearth scienceí, ëlife scienceí, and ëenvironmental issuesí than in ëchemistryí or ëphysics.í In these figures, every 2 percentage points means that students correctly answered about 3 items. A difference of 10 percentage points, therefore, represents a difference in studentsí performance on about 15 items.

Educators and policy makers in the U.S. and other countries have been concerned to provide equal opportunities for boys and girls in mathematics and science. Of particular interest, therefore, is the fact that the U.S. was one of 11 countries in which there were no significant difference between the performance of eighth-grade boys and girls in either mathematics or science. The other ten countries in which this was observed are Australia, Colombia, Cyprus, Flemish-speaking Belgium, Ireland, Romania, the Russian Federation, Singapore, South Africa, and Thailand.

While these results clearly have implications for U.S. mathematics and science education, caution is advised in drawing conclusions and applications to any specific school setting. Unlike most other TIMSS countries, the U.S. does not have a single or centrally coordinated education system. In terms of curricular vision and control, the thousands of local school districts may represent a more appropriate level for international comparisons. Indeed, many U.S. states have larger populations than some TIMSS countries. New Jersey, for example, is more populous than Austria, Denmark, Norway, or Switzerland. Furthermore, in previous international comparisons, the scores of some states were similar to the highest -scoring nations while those of other states were about the same as the lowest-scoring nations. Future analyses will further elucidate the relevance of the U.S. TIMSS results for more regionally- and locally-defined situations.

Summary written by Leland Cogan, Michigan State University

The report entitled ìPursuing Excellence: U.S. Eighth-Grade Mathematics and Science Teaching, Learning, and Achievement in International Contextî may be obtained by contacting the National Center for Education Statistics, Office of Educational Research and Improvement, U.S. Department of Education, 555 New Jersey Avenue NW, Washington, D.C. 20208-5574, Telephone 202-219-1395. It is also available on the World Wide Web for viewing and downloading: http://www.ed.gov/NCES/timss.

No one is at the helm of mathematics and science education in the U.S. In truth, there is no identifiable helm. No single coherent vision of how to education todayís children dominates U.S. educational practice in either subject, nor is there a single, commonly accepted place to turn to for such visions. Our visions ñ to the extent that they exist at all ñ are multiple.

These splintered visions produce unfocussed curricula and textbooks that fail to define clearly what is intended to be taught. They influence teachers to implement diffuse learning goals in their classrooms. They emphasize familiarity with many topics rather than concentrated attention on a few. And they likely lower the academic performance of students who spend years in such a learning environment. Our curricula, textbooks, and teaching are all ìa mile wide and an inch deep.î

This preoccupation with breadth rather than depth, with quantity rather than quality, probably affects how well U.S. students perform in relation to their counterparts in other countries. It thus determines who our international ìpeersî are and raises the question of whether these are the peers that we want to have. In todayís technologically oriented global society, where knowledge of mathematics and science is important for workers, citizens, and individuals alike, an important question is: What can be done to bring about a more coherent vision and thereby improve mathematics and science education?

Reforms have already been proposed by political, business, educational and other leaders. Extensive efforts are underway to implement these standards, but the implementation process itself is shaped by the prevailing culture of inclusion. Like the developers of curricula and the publishers of textbooks, teachers add reform ideas to their pedagogical quivers without asking what should be taken away.

Splintered Vision: An Investigation of U.S. Mathematics and Science Education represents an effort to describe the nature of the diffuse vision of mathematics and science education in the U.S. and raises questions relevant to policy making.

An expanded executive summary of ìSplintered Vision: An Investigation of U.S. Mathematics and Science Educationî and a pre-print version of the entire book are available on the U.S. National Research Center web site: http://ustimss.msu.edu/. The book is copyrighted by Kluwer Academic Publishers and will be available in late winter. For ordering information, contact Kluwer Academic Publishers Group, Order Department, PO Box 358, Accord Station, Hingham, MA 02018-0358, telephone: 617-871-6600; fax: 617-871-6528; email: kluwer@wkap.com. ISBN paperback: 0-7923-4441-3; hardback: 0-7923-4440-5. ìSplintered Visionî is co-authored by William H. Schmidt, Curtis C. McKnight and Senta A. Raizen.
 

A video survey of eighth-grade mathematics lessons in Germany, Japan, and the United States was done as a part of the U.S. TIMSS project. It is the first to collect videotaped records of classroom instruction ñ in any subject ñ from nationally representative samples of teachers. The study sample included eighth-grade mathematics classrooms: 100 in Germany, 50 in Japan, and 81 in the United States. The three samples were selected from among the participating TIMSS schools and classrooms which were designed to be representative of eighth-grade classrooms in the three countries.

One lesson was videotaped in each classroom at some point during the school year. Tapes were encoded and stored digitally on CD-ROM, and were accessed and analyzed using multimedia database software developed especially for this project. All lessons were transcribed, and then analyzed on a number of dimensions by teams of coders who were native speakers of the three languages. Analyses focused on the content and organization of the lessons, as well as on the instructional practices used by teachers during the lessons.

Although the vast video data will continue to provide rich analysis opportunities for researchers, a number of differences in instructional practices across the three cultures have already been identified. These differences fall into four broad categories: (1) how lessons are structured and delivered, (2) the kind of mathematics presented in the lesson, (3) the kind of mathematical thinking students are engaged in during the lesson, and (4) how teachers view reform.

How Lessons are Structured and Delivered

To understand how lessons are structured it is important first to know what teachers intended students to learn from the lessons. Information gathered from teachers in the video study indicated an important cross-cultural difference in lesson goals. Solving problems is the end goal for the U.S. and German teachers: how well students solve problems is the metric by which success is to be judged. In Japan, problem solving is assumed to play a different role. Understanding mathematics is the overarching goal; problem solving is merely the context in which understanding can best grow.

Following this difference in goals, we can begin to identify cultural differences in the scripts teachers in each country use to generate their lessons. These different scripts are probably based on different assumptions about the role of problem solving in the lesson, about the way students learn from instruction, and about what the proper role of the teacher should be.

Although the analyses are preliminary at this point, we believe that there is a clear distinction between the Japanese script, on one hand, and the U.S. and German scripts, on the other. U.S. and German lessons tend to have two phases: an initial acquisition phase and a subsequent application phase. In the acquisition phase, the teacher demonstrates and/or explains how to solve an example problem. The explanation might be purely procedural (as most often happens in the U.S.) or may include development of concepts (more often the case in Germany). Yet still, the goal in both countries is to teach students a method for solving the example problem(s). In the application phase, students practice solving examples on their own while the teacher helps individual students who are experiencing difficulty.

Japanese lessons appear to follow a different script. Whereas in German and U.S. lessons instruction comes first, followed by application, in Japanese lessons the order of activity is generally reversed. Problem solving comes first, followed by a time in which students reflect on the problem, share the solution methods they have generated, and jointly work to develop explicit understandings of the underlying mathematical concepts. Whereas students in the U.S. and German classrooms must follow the teacher as she leads them through the solution of example problems, the Japanese student has a different job: to invent his or her own solutions, then reflect on those solutions in an attempt to increase understanding.

In addition to these differences in goals and scripts, we also found differences in the coherence of lessons in the three countries. The greatest differences were apparent between U.S. and Japanese lessons. U.S. lessons were found to be less coherent than Japanese lessons by several criteria: U.S. lessons were more frequently interrupted, both from outside the classroom and within; U.S. lessons contained significantly more topics ñ within the same lesson ñ than Japanese lessons; Japanese teachers were significantly more likely to provide explicit links or connections between different parts of the same lesson.

The Kind of Mathematics that is Presented

Looking beyond the flow of the lessons, we also found cross-cultural differences in the kind of mathematical content that was presented in the lessons. One finding concerned the level of mathematics presented. When viewed by international curriculum standards, the average eighth-grade U.S. lesson in the video sample dealt with mathematics at a 7th grade level, whereas in Japan the average level was 9th grade. The content of German lessons averaged 8th grade level.

The quality of the content also differed across countries. For example, most mathematics lessons include some mixture of concepts, and applications of those concepts to solving problems. How concepts are presented, however, varies a great deal across countries. Concepts might simply be stated, as in ìthe Pythagorean Theorem states that a² + b² = c²,î or they might be developed and derived over the course of the lesson. More than three-fourths of German and Japanese teachers developed concepts when they included them in their lessons, compared with less than one-fifth of U.S. teachers. None of the U.S. lessons included proofs, whereas 9% of German lessons and 51% of Japanese lessons included proofs.

Finally, an independent group of American college mathematics teachers was asked to evaluate the quality of mathematical content in a sample of the video lessons. They based their judgments on a detailed written description of the content that was altered for each lesson to disguise the country of origin (deleting, for example, references to currency that might give the country away). They completed a number of in-depth analyses, the simplest of which involved making global judgments of the quality of each lessonís content on a three-point scale (Low, Medium, High). Whereas 30% of the Japanese lessons and 23% of the German ones received the highest rating, none of the U.S. lessons received the highest rating. 87% of U.S. lessons received the lowest rating, compared with only 13% of Japanese lessons.

The Kind of Mathematical Thinking in which Students are Engaged

When we examined the kind of work students engaged in during the lesson we found a strong resemblance between Germany and the U.S., with Japan looking distinctly different. Three types of work were coded in the video study: Practicing Routine Procedures, Applying Concepts to Novel Situations, and Inventing New Solution Methods/Thinking. More than 90% of student working time in Germany and the U.S. was spent in practicing routine procedures, compared with 40% in Japan. Japanese students spent the majority of their time inventing new solutions and engaged in conceptual thinking about mathematics.

Teachers and Reform

A great deal of effort has been put into the reform of mathematics teaching in the U.S. in recent years. Numerous documents ñ examples include the NCTM Curriculum and Evaluation Standards and the NCTM Professional Teaching Standards ñ encourage teachers to change the way they teach, and there is great agreement, at least among mathematics educators, as to what desirable instruction should look like. Although most of the current ideas stated in such documents are not operationalized to the extent that they could be directly coded, it is possible to view some of the indicators developed in the video study in conjunction with these current ideas. When the video data are viewed in this way, it seems clear that Japanese teachers, on average, come closer to implementing the spirit of current ideas advanced by American reformers than do American teachers.

Interestingly, the U.S. teachers who participated in the video study believe that they are implementing current reform ideas in their classrooms. When asked specifically to evaluate their videotaped lesson, almost three-fourths of the American teachers rated it as reasonably in accord with current ideas about the teaching and learning of mathematics. They were more than twice as likely to respond this way than either the Japanese or the German teachers.

Teachers who said that the videotaped lesson was in accord with current ideas about the teaching and learning of mathematics were asked to justify their responses. Although the range and variety of responses to this question were great, the vast majority of American teachersí responses pertained to surface features, such as the use of manipulatives or cooperative groups, rather than to the deeper characteristics of instruction.

In fact, the findings of the video study highlight an extremely important fact about efforts to improve mathematics instruction: Written reports which are disseminated to teachers may have little impact on practices in the classroom. One reason for this may be that teachers may not have widely shared understandings of what such terms as ìproblem solvingî really mean, leading to idiosyncratic interpretations in the classroom. Video examples of high quality instruction tied to descriptions of what quality instruction should look like may help, in the future, to address this problem.

Contributed by James Stigler, Department of Psychology, UCLA
 

In Characterizing Pedagogical Flow (Schmidt et al., 1996), the authors point out that schools and schooling are social institutions functioning according to prevailing cultural mores and values. Each country has a rich cultural heritage that informs and shapes the nature of schools and schooling. Examining some of the practices and social characteristics that accompany schooling in various nations can provide a valuable background against which studentsí achievement may be evaluated. The first international report on TIMSS achievement released November 20, includes some of the results from TIMSSí investigation of schools, teachers, and studentsí background. In addition, the U.S. report released by U.S. Secretary of Education, Richard Riley, presents U.S. TIMSS achievement together with selected findings concerning teachers, students, and schools in the 41 TIMSS countries. Against this backdrop, the report focuses in more depth on comparisons with two important U.S. trading and political partners, Germany and Japan, from which additional data were collected as part of the U.S. TIMSS effort.

Mathematics and science in a world context

As a nation, the U.S. does not have a single coherent vision of what students should learn through mathematics and science education. The U.S. is atypical among TIMSS countries in its lack of a nationally- or regionally-defined curriculum. TIMSSí study of curricula found that current U.S. curricular standards are unfocused and aimed at the lowest common denominator. They are, in other words, a mile wide and an inch deep.

The TIMSS analysis found that, before high school, states in the U.S., on average, intend students to cover more topics in mathematics and science than most of the other countries studied. This is especially true in mathematics. Similarly, U.S. mathematics and science textbooks included far more topics than was typical internationally, and gave significantly less coverage than the international average to the five most emphasized topics. In addition, what may be considered ìbasicî in eighth-grade mathematics dramatically differs in the U.S. from Germany, Japan, and most of the other TIMSS countries. In the U.S., basic content included arithmetic, fractions and a relatively small amount of algebra. In Germany, Japan and most other countries, basic content for all students included intense coverage of algebra and geometry ñ subjects that in the United States are reserved for students in higher-level classes.

TIMSS data shows that this lack of a clear focus is evident in what is taught as well as in the number of topics intended to be taught. U.S. curricula and textbooks divide attention among too many topics and deliver too little attention to most of them. Not surprisingly, the same lack of focus is evident in how math and science are taught. Teachers skip among many topics and attempt to cover far more than their German and Japanese counterparts. U.S. teachersí instructional practices are as splintered and fragmented as the curricula that shape them and the textbooks that support them. They do not show the focused coherence evident in many other countries whose student achievements are higher than ours.

Critical nature of curriculum

Since Colemanís report in the ë60s on the equality of educational opportunity in the U.S., many have concluded that the critical determinants of studentsí learning are sociological and economic. Analyses of TIMSS data, however, suggest that what occurs in schools is critical to studentsí learning. The curriculum students encounter in schools ñ the specific topics that are taught and how these topics are presented and developed ñ fundamentally shapes what students learn and are able to do.

Rather than the focused coherence seen in other countries, U.S. lessons, as reported in Characterizing Pedagogical Flow and the videotape study of eighth grade math classrooms, often consist of ìepisodic encountersî between students and curricular content. Topics and concepts are presented in a fragmented and disjointed manner in which underlying themes or principles are either not identified or simply stated but not developed. In other countries, math and science are presented with greater logical coherence ñ more as a story line that is developed both within each lesson and across a series of lessons.

Unless the U.S. vision of mathematics and science changes, classrooms will not change. A clear, logical coherence in the curriculum is an essential prerequisite of coherent curricular development in daily lessons. The U.S. has about 15,000 curricula in this country, most of which are as unfocused as our national vision. As an essential first step towards change, focus must be brought to what we want to accomplish and how we are going to get there.

Some commonly held myths addressed

Suggestions for reforming education in general and mathematics and science education in particular have often centered around adjusting the quantity of something. Some have suggested that students performance would be improved if, for example, students should watch less TV or more homework was given student to do. Others have suggested that schools should be in session for longer periods of time or that teachers should have more education. These and other commonly-held myths are directly addressed by TIMSS data:

ï U.S. teachers of math and science have more college education than their colleagues almost anywhere in the world. The U.S. report indicates that teachers of almost half of the U.S. TIMSS students hold masters degrees ñ a proportion exceeded by only four other TIMSS countries. Few Japanese teachers have more than a bachelorís degree. All German teachers have a bachelors degree for which they must complete about six years of university study, write a thesis, and pass an examination. The German teachersí bachelor degree is considered the equivalent of a U.S. masters degree.

ï U.S. teachers assign more homework and spend more class time discussing it than teachers in Japan and Germany.

ï U.S. students are required to spend more time in mathematics and science classes than either German or Japanese students.

ï U.S. students report about the same amount of out-of-school math and science study as their Japanese and German counterparts.

ï Heavy television watching is as common among Japanese eighth graders, who do better than U.S. students, as it is among American eighth graders. In general, eighth-grades in Germany, Japan, and the U.S. appear quite similar in their focus upon school, friends, and recreational activities.

ï Student diversity and poor discipline are challenges not only for U.S. teachers, but for their German colleagues as well. Despite the fact that the common U.S. practice of grouping students for instruction according to ability is not allowed in Japan, Japanese teachers report that students of differing academic abilities present a challenge to the same extent as their German and U.S. colleagues. Despite many stories in the popular media, severe discipline problems or threats to personal safety are not widespread nor unique to the U.S. Teachers of 76% of U.S. students and 65% of German students reported that threats to their own or studentsí safety were ìnot at allî a problem.

The U.S. report also shows that U.S. mathematics and science teachers are scheduled to teach more classes than either their German or Japanese counterparts and that Japanese teachers commonly have more opportunities to discuss teaching-related issues with their colleagues than do U.S. teachers. Although studentsí use of time appears quite similar, the meaning of their school experiences differ. German eighth-grade students have already been assigned to several different levels of vocational and academic tracks. While Japanese eighth-grade students are not tracked or grouped in any way, they are preparing to take a high-stakes examination at the end of ninth grade which they must pass in order to enter high school.

Written by Leland Cogan, Michigan State University.
 
 

CURRICULUM AND ACHIEVEMENT:
SEARCHING FOR THE EMPIRICAL LINK
 
 
 
 
 

The Third International Mathematics and Science Study was not only a study of student achievement in over 40 countries but was also a study of the curriculum students received. TIMSS researchers analyzed mathematics and science textbooks and curriculum guides for the TIMSS testing grades in participating countries as well as expert data on topic introduction, coverage, and focus in each country. TIMSS researchers hope to use the data to explain differences in student achievement across nations. However, the determinants of student achievement are complex, and the impact of the curriculum on achievement is not always direct. What teachers teach, their instructional practices, and other environmental factors often act as intermediary variables to achievement. Additionally, we cannot assume that one path to high student performance exists. Different curricular patterns may prove equally successful in countries with different cultural backgrounds, priorities, and resources.

The results of the curriculum analysis component of TIMSS show that significant variation in the content of mathematics and science curricula exists across nations. Many of these results are presented in two international reports on the TIMSS curriculum analysis: Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School Mathematics (Schmidt, W.H., McKnight, C.C., Valverde, G.A., Houang, R.T., & Wiley, D.E., 1996) and Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School Science (Schmidt, W.H., Raizen, S.A., Britton, E.D., Bianchi, L.J., & Wolfe, R.G.). These reports show that nations vary on when they introduce topics, how long they plan to cover topics, the number of topics intended for coverage in any given year, how topics are treated in textbooks and curriculum guides, what they expect students to be able to do with the content, and the relative emphases of mathematics and science topics and performance expectations. Some similarities exist among countries, but many differences exist also. For example, Figures 1 and 2 show the relative textbook space allocated to eight mathematics content areas and six science content areas for clusters of countries with similar math (Figure 1) and science (Figure 2) curricula. The countries were grouped according to patterns of topic emphasis in their 8th grade mathematics or science textbooks. The content area proportions were calculated from the mean proportions of textbooks space for all topics within a content area for all countries within a group.

Figure 1. Relative proportion of textbook space allocated to eight content areas for six groups of countries statistically clustered according to topic coverage in 8th grade mathematics textbooks.
 
 
 

a = Numbers

b = Fractions & Decimals

c = Measurement

d = Geometry

e = Proportionality

f = Algebra

g = Data

h = Validation & Structure

Figure 2. Relative proportion of textbook space allocated to six content areas for eight groups of countries statistically clustered according to topic coverage in 8th grade science textbooks.
 
 
 

a = Earth science

b = Life sciences

c = Physics

d = Chemistry

e = Interactions of science, math, & technology

f = Food production & storage
 
 
 

Definite variation exists across most content areas. Of particular interest is the treatment of algebra (f) in Figure 1 across the groups of countries for mathematics. Also interesting are the patterns of earth science, life science, and physics across the science groups. One would expect these variations to lead to variations in patterns of achievement across the countries. Indeed, of the nine countries from the mathematics cluster group 6 that participated in the TIMSS achievement testing, seven (Bulgaria, Czech Republic, Slovak Republic, Japan, Korea, the Netherlands, and Singapore)were among the top performing countries and outperformed the United States. In Science, of the seven countries in cluster group 8, four (Czech Republic, Japan, Korea, and Slovenia) were among the top performing countries and also outperformed the United States.

In addition to system-level resources, student motivation, and other factors that may interact with student learning, the following curricular variables must also be considered in investigations of the link between curriculum and achievement. First, the curriculum students have received prior to 8th grade must be taken into account. Students do not enter 8th grade at an even starting point. Therefore, the expected effects of the 8th grade curriculum must be considered in relation to where students began. Additionally, the mathematics (or science) topics are not necessarily independent. It is likely that students will improve their skills in simple arithmetic, for example, while studying algebra. This means that the interrelationships among topics must also be explored. Another complication is the fact that the same topic is not treated in the same way across countries, and countries do not always expect the same type of topic mastery. Some countries may have a curriculum that stresses problem solving and reasoning, while others may stress basic understanding and knowledge. Data are available on performance expectations in textbooks and curriculum guides, and achievement items all have complex ìsignaturesî that describe the topic(s) and performance expectation(s) measured by each item. This complexity is certain to have an impact on student performance.

An additional issue in curriculum-achievement linking relates to the sensitivity of the test to curricular differences. It is unlikely that a strong curricular link will be found when analyzing total scores, and even, content area scores such as algebra, geometry, etc. Aggregations of items across many different topics and performance expectations will likely wash out the evidence of curricular effects. Teachers do not teach ìmath;î they do not even teach ìalgebra.î What they do teach are complex interrelated topics and performances. A single classroom lesson is more similar to an individual item or a set of related items within a content domain than it is to a group of items covering a large number of content domains. Therefore, investigating the impact of curriculum on achievement will likely be more successful at the item level - or as close to it as possible.

TIMSS researchers have already begun initial steps in the investigation of the impact of curriculum on achievement. For example, the U.S. TIMSS curriculum analysis report entitled A Splintered Vision: An Investigation of U.S. Science and Mathematics Education (Schmidt, W.H., McKnight, C.C., and Raizen, S.A.) discusses and provides evidence of the unfocused nature of U.S. math and science curricular intentions, textbooks, and teacher practices, and contrasts these data with similar data from Japan and Germany. Many topics are included in the U.S. math and science curriculum during one year, and the majority of these topics remain in the U.S. curriculum longer than topics included in the curricula of other countries. The authors of A Splintered Vision offer the premise that producers of U.S. textbooks and curriculum guides have attempted to answer calls for curricular reform by adding new content to already existing materials instead of devoting time to restructuring the materials. They also suggest that U.S. teachers, inundated with a myriad of competing visions, are attempting to ìsatisficeî by covering all the topics they confront in their resource documents and meeting all the instructional demands placed on them by those with a stake in education. In keeping with theìincremental assembly lineî philosophy in American society, U.S. teachers also tend to lean toward a piecemeal approach to education. The results of the TIMSS achievement study, summarized elsewhere in this newsletter, show that U.S. students do not fare well within a system dominated by such a splintered vision. Additional analyses, although preliminary, show the performance of U.S. students as being below the median for many mathematics and science topics (i.e., specifically and narrowly defined content areas such as, equations and formulas and weather and climate). Some countries performing better than the U.S. cover sometimes half as many topics as we do in a given year, and topics remain in the curricula for shorter periods of time, giving the impression that a more focused curriculum may lead to higher achievement. However, not all countries performing better than the U.S. have a highly focused curriculum, so other explanations to the achievement differences, like sequencing and emphasis, are also being investigated. A report on these results is planned for the near future.

Written by Pamela Jakwerth, Michigan State University.
 
 
 
 

This is a reminder that the book: Characterizing Pedagogical Flow: An Investigation of Mathematics and Science Teaching in Six Countries is currently available through Kluwer Academic Publishers Group, Order Dept., PO Box 358, Accord Station, Hingham, MA 02018-0358; Telephone: 617-871-6600; Fax: 617-871-6528; email: kluwer@wkap.com. ISBN 0-7923-4273-9. Price: $49.00 in paperback. Ordering information is also available on the World Wide Web at:
gopher://gopher.wkap.nl:70/00gopher_root1%3A%5B
book.soci.f500%5Df5101601.txt.

The following summary of Characterizing Pedagogical Flow is reprinted from the U.S. NATIONAL RESEARCH CENTER REPORT NO. 6.

Characterizing Pedagogical Flow presents conclusions from the Survey of Mathematics and Science Opportunities (SMSO), a multi-disciplinary, multi-national research project that blended quantitative and qualitative methodologies. The purpose of SMSO was to develop a comprehensive battery of instruments addressing the student, teacher, school, and curriculum factors for use in the Third International Mathematics and Science Study (TIMSS) recently conducted by the International Association for the Evaluation of Educational Achievement (IEA). In order to develop instruments that appropriately and meaningfully assessed key factors in classroom pedagogy that influence nine-year old and thirteen-year old studentsí mathematics and science achievement, SMSO focused on what occurs in these studentsí mathematics and science classrooms.

Over 120 observations were conducted by the SMSO team in nine-year old and thirteen-year old studentsí mathematics and science classrooms. Both types of classrooms were observed in each of the six SMSO countries: France, Japan, Norway, Spain, Switzerland, and the United States. Part I of Characterizing Pedagogical Flow chronicles what SMSO learned about mathematics and science curriculum and pedagogy. The work produced portraits of mathematics and science education that were dramatically different for each of the countries involved. Part II presents generalized case studies for each countryís mathematics and science classrooms. These provide the invaluable contextual insight essential for understanding the cross-national differences discovered among countriesí mathematics and science curriculum and pedagogy.

The book proposes that cross-national differences may be explained by the interaction of curriculum and pedagogy in a culturally unique manner. The school subjects of mathematics and science are highly influenced by cultural perspectives. In a manner comparable to what might be expected in the teaching of history and language, culture influences the teaching of mathematics and science to yield qualitatively different classroom learning experiences from one country to another. Classroom lessons exhibited a ìcharacteristic pedagogical flowî ñ a particular way in which subject matter and pedagogy interacted. It is further proposed that these observable characteristics that lessons exhibited are a function of the ideas, beliefs, theories, and pedagogical repertoires that teachers possess. Each SMSO country demonstrated a distinctive ìcharacteristic pedagogical flowî in their mathematics and science classrooms yielding the portraits mentioned above. Presumably, these were based on commonalities teachers within a country shared ñ certain national beliefs and ideologies about education, teachersí training, and other key experiences.

These ideas have profound implications for how international education research is conducted and interpreted. To begin with, substantive discourse among multi-national representatives is essential in any cross-national investigation of education. Such discourse challenges implicit assumptions and enables the identification of important differences that are most likely to have explanatory significance. Members of the SMSO team concluded that survey instruments may more profitably be designed to look for differences in qualitative distinctions rather than mere quantitative distributional differences in, presumably, common practices such as the amount of homework students do or the amount of time a teacher lectures during lessons. A further recommendation is that serious international surveys also need to include a qualitative component in order to contextual and understand the meaning of a surveyís results.

Written by: Leland Cogan, Michigan State University
 
 
 

FUTURE AVAILABILITY OF REPORTS

Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in Mathematics - Volume I will be published in February 1997 by Kluwer Academic Publishers Group. The ISBN for the paperback is: 0-7923-4437-5; hardback: 0-7923-4436-7. Contact information is the same as above.

A Splintered Vision: An Investigation of U.S. Science and Mathematics Education will also be published mid winter by Kluwer Academic Publishers. ISBN for the paperback will be: 0-7923-4441-3; hardback: 0-7923-4440-5.

Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in Science - Volume II will follow from Kluwer Academic Publishers. ISBN for the paperback will be: 0-7923-4439-1; hardback: 0-7923-4438-3.

A set of the three volumes will also be available. ISBN for the set in paperback: 0-7923-4443-X; hardback: 0-7923-4442-1.

Prices have not yet been established.
 
 

UPCOMING PRESENTATIONS ON TIMSS

The following presentations on TIMSS are currently scheduled to be given by Bill Schmidt, National Research Coordinator for U.S. TIMSS:

December 16, Heritage Academy-Longmeadow, MA (Gilbert Valverde will be speaking);

December 19, Michigan Business Roundtable in Ann Arbor;

December 19, Ohio Business Roundtable in Columbus;

January 8, Joint Mathematics Conference, San Diego;

January 10, Michigan Department of Education Science Assessment Symposium;

January 22, American University, Washington, D.C.;

January 30, Concordia University;

January 31, Association of American Publishers in Orlando;

February 3, National Academy of Sciences in Washington, D.C.;

February 14, AAAS in Seattle;

March 21, NARST, Oak Brook, Illinois;

March 24-28, AERA, Chicago;

April 18, NCTM, Minneapolis;

May 1-3, Education Writers Association, Washington, D.C.;

May 8-10, Mathematics and Science Conference, Kellogg Center, MSU, East Lansing, MI.
 
 
 

Visit the U.S. TIMSS National Research Center web site at: http://ustimss.msu.edu/

National Research Coordinating Committee 

Dr. Eugene Owen, National Center for Education Statistics; Dr. Lois Peak, National Center for Education Statistics; Dr. Larry Suter, Division of Research, Evaluation and Communication, National Science Foundation. The committee is chaired by Dr. William Schmidt, Michigan State University. 

U.S. National Steering Committee Members 

Dr. Gordon Ambach, Executive Director, Council of Chief State School Officers; Dr. Deborah Ball, Associate Professor, Michigan State University; Dr. Audrey Champagne, University at Albany-SUNY; Dr. Jewel Plummer Cobb, California State, Los Angeles; Dr. David Cohen, Professor, The University of Michigan; Dr. John Dossey, Distinguished Professor of Mathematics, Illinois State University; Dr. Emerson Elliott, National Center Education Statistics; Dr. Sheldon Glashow, Higgings Professor of Physics and Mellon Professor of the Sciences, Harvard University; Dr. Larry Hedges, Professor, Department of Education, University of Chicago; Professor Henry Heikkinen, Department of Chemistry and Biochemistry, University of Northern Colorado; Dr. Jeremy Kilpatrick, Regents Professor of Mathematics Education, University of Georgia; Dr. Mary Lindquist, Fuller E. Callaway Professor of Mathematics Education, Columbus College, and President of the National Council of Teachers of Mathematics; Dr. Marcia C. Linn, Professor and Director, Instructional Technology Program, University of California-Berkeley; Dr. Robert L. Linn, Professor and CO-director of the Center for Research on Evaluation, Standards, and Student Testing, University of Colorado; Dr. Paul Sally, Professor of Mathematics, University of Chicago; Dr. Richard Shavelson, Professor, School of Education, Stanford University; Dr. Bruce Spencer, Professor, Department of Statistics and School of Education and Social Policy, Northwestern University; Dr. Elizabeth Stage, New Standards Project, CO-Director for Science, University of California, Office of the President; Dr. James Taylor, Hill and Knowlton, NYC; Dr. Kenneth Travers, University of Illinois, and Dr. Paul H. Williams, Professor, Department of Plant Pathology, University of Wisconsin. The committee is chaired by Dr. William Schmidt, Professor, National Project Coordinator for U.S. TIMSS and Executive Director of the U.S. National Research Center for TIMSS located at Michigan State University. 

We want to hear from our readers, so if you have questions, want additional information about a topic addressed in the newsletter or the study in general, please feel free to contact us. You can write or call Gilbert Valverde, Associate Director - TIMSS, 457 Erickson Hall, Michigan State University, East Lansing, Michigan 48824-1034, telephone 517-353-7755, fax 517-432-1727, or E-Mail, valverde@pilot.msu.edu.

We also are available to make presentations to professional groups or associations who might be interested in being introduced to TIMSS or receiving updates on the project. If your organization is interested in hearing more about TIMSS and would like a representative from our office to present to your group, please contact the National Research Center at Michigan State University.

If you or someone you know did not receive this newsletter directly, but would like to be on our mailing list, please send your name and address along with your request to Jacqueline Babcock, 464 Erickson Hall, Michigan State University, East Lansing, Michigan 48824-1034, telephone 517-353-7755, fax 517-432-1727, or E-Mail: jbabcock@pilot.msu.edu.

This newsletter is published by the TIMSS U.S. National Research Center located at Michigan State University. The newsletter is edited by Jacqueline E. Babcock.

The U.S. National Research Center is funded by a grant from the National Science Foundation in conjunction with the National Center for Education Statistics.