What is NAEP Long

While most NAEP assessments are periodically updated, the long-term trend assessments in reading and mathematics are designed to track changes in student performance back to 1971 for reading and 1973 for mathematics. Since this predates the state-level NAEP assessments, long-term trend results are reported only on the national level. These results allow us to see how we have progressed as a nation and provide comparisons between nations.

In 1990, the nation’s state and federal leaders set six national goals for education in the 21^st century:

Measuring students’ progress toward higher academic achievement has been the mandate of NAEP since 1969. One important feature of NAEP is its ability to document trends in academic achievement in core curriculum areas over an extended period of time. By making information on student performance –and instructional factors related to that performance—available to policymakers at the local, state, and national levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education.

Measuring changes in student achievement over time requires that past procedures are followed exactly. The first NAEP assessments were given in three subject areas---science, mathematics, and reading. Students were assessed at ages 9, 13, and 17. In 1984, a fourth subject, writing, was added. The assessments were generated from different frameworks than the current NAEP assessments in these subjects, and scores are generated on different scales.

While the current NAEP assessments have evolved as theory and curricula changed due to emerging findings in education research, NAEP long-term trend assessments have not. This kind of stability permits the comparison of results between decades. It is not possible to compare results from the current (often called “main”) national or state NAEP with those of the long-term trend assessment.

The No Child Left Behind Act specifies that NAEP is to carry out three types of assessments in reading and mathematics—national, state, and long-term trend. In particular NAEP is directed to “continue to conduct the trend assessment of academic achievement at ages 9, 13, and 17 for the purpose of maintaining data on long-term trends in reading and mathematics.”

It is currently the practice, both nationally and internationally, to initiate large-scale assessment in the fourth year of regular, full-time school. Unfortunately, the age of students at this stage of their education varies across the world. According to the international Progress in Reading Literacy Study (http://pirls.bc.edu), fourth-year students range in age from 9 to 11 across the nations assessed. Schools in the United States will often have fourth graders who are slighter younger or somewhat older than their peers. In Maine , state policy directs that they be 10 by October 1; however, students are very occasionally held back for a year.

Thus, a main NAEP state assessment will be administered to a population that is slightly diverse in terms of age. The long-term trend study is not.

Doing long-term trend studies creates challenges for NAEP in that past practices must be followed even if they have been replaced in current assessments with better practices; e.g., increased accommodations for students with disabilities, improved background questionnaires and administrative procedures, and content specifications aligned with current state and local standards.

NAEP could initiate new long-term trend studies with the current main assessments, but the same problems would arise in future decades as new education theory emerges from research. There is some discussion regarding doing this for writing and science. The 1999 writing long-term trend results have not yet been released and remain under review by the federal government. The current (main) writing assessment is based on a framework adopted for the 1998 assessment and uses very different scoring methods than the long-term trend writing assessment.

Prior to 1998, writing responses were scored twice—once using a primary trait scoring guide for content and structure, and a second time using an analytical scale for mechanics. Since 1998, NAEP writing has been scored using a modified holistic scale (which is described in NAEP Newsletter #4).

Fortunately, the reading and mathematics assessments have proved extremely stable over time. Scoring methods have not changed very much and have proved the most reliable and consistent of all of the subjects assessed by NAEP.

PAST LONG-TERM TREND RESULTS

The overall pattern of performance in science (the first subject NAEP assessed) has been one of early declines followed by a period of consistent improvement. Scores began to rise for 17-year-olds in 1982; for 13-years-olds, in 1977; for 9-year-olds, in 1973.

The overall pattern of performance in mathematics is one of consistent improvement across all the years of the assessment, with the exception of 17-year-olds. This age group showed declining results up to the early 1980’s followed by moderate gains since then; thus their scores are not significantly different between the earliest years of the assessment and the present.

The overall pattern of performance in reading has been one of improvement across the assessment years; however, the pattern of significant increases in scores from 1971 to 1988 was not sustained in the 1990’s. The average scaled score of 17-year-olds were not significantly different between the earliest years of the assessment and the present. The scores for 13-year-olds rose moderately across the assessment years. The performance of 9-year-olds improved up to 1980 but has declined slightly since then.

The long-term-trend assessments predate the implementation of Achievement Levels of Basic, Proficient, and Advanced; student achievement was compared across years purely on the basis of average scaled scores. However, cut points of 150, 200, 250, and 300 are used for comparison of student performance across the nation.

Descriptions were later attached to these score levels as follows:

In Science, a student achieving a score of 350 was believed able to “infer relationships and draw conclusions using detailed scientific knowledge.” At 250, a student “understands and applies general information from the life and physical sciences.” At 150, a student is said to know “everyday science facts.”

The greatest increases in these levels across all the years of the science assessment were 6% each at 200 and 250 for age 9; for age 13, 9% at 250; and 6% at 300 for age 17.

Descriptors at cut points for mathematics ranged from the ability to solve multi-step problems at the top of the scale to knowing “some addition and subtraction facts” at the bottom.

In mathematics, there was an 8% increase at 300 for 17-year-olds; 14% at 250 for 13-year-olds; and 12% at 200 for 9-year olds (as well as 10% at 250) across all the years of the assessment.

For reading, descriptors rose from the ability to “carry out simple, discrete reading tasks” (150) through the ability to “search for specific information, interrelate ideas, and make generalizations” (250) to the ability to “synthesize and learn from specialized reading materials” (350).

In reading, the most significant increases across all the years of the assessment were 5% at 200 for 9-year-olds; 4% at 300 for 13-year-olds; and 3% at 250 for 17-year-olds.

MAINE SCHOOLS PARTICIPATING IN 2004

This school year, long-term trend in reading and mathematics will be administered at schools in Athens, Bingham, Canaan, Clinton, Dover-Foxcroft, Fairfield, Greenville, Guilford, Jackman, Madison, Milo, Pittsfield, Sangerville, Skowhegan, and Starks.