Christine Merrell and Peter Tymms, of the CEM Centre at the University of Durham, report on their international research on the use and benefits of baseline assessment for the Performance Indicators in Primary Schools (PIPS) project.

How similar are the cognitive developmental profiles of young children growing up in different countries and cultures? If we consider the variety of situations in which children are raised and the different language structures that they must learn, we might expect that when children from different cultures start school they would have very different developmental profiles.

Many international studies have compared the literacy, mathematics and science skills of older children, which only partially reflect the standard of educational provision in different countries. The results depend on many factors including pre-school provision. If we want to know about the impact of schooling one essential requirement is to know where children start.

There has been no large-scale international study that has attempted to use a common baseline assessment to compare the starting points of children on entry to formal school. As well as being of general interest and a prerequisite for the meaningful interpretation of the assessments carried out with older children, an international baseline assessment would enable researchers to compare the impact of countries' policies on pre-school experiences, the age of starting school and to interpret later attainment data more clearly.

The PIPS (Performance Indicators in Primary Schools) On-entry Baseline Assessment, developed by the Curriculum Evaluation and Management Centre (CEM Centre), is used to assess children starting school in several countries (see table). It is a computer-delivered, adaptive assessment in which the teacher works with individual pupils. Taking about 20 minutes per child, the assessment covers handwriting, vocabulary, reading, phonological awareness and mathematics development. It was adapted for use in Australia, New Zealand and Scotland with little change to the content. The Dutch version and the British Sign Language version were specially developed to maintain the characteristics of the items as far as possible.

Before making comparisons between groups of pupils with different backgrounds it is important to be assured that the assessment able to make fair comparisons. To do this each item in the assessment was compared across the groups and the question asked: Is this item relatively harder or easier for this particular group? We found that the relative difficulties of the assessment items were remarkably similar across all the groups. For example, if children in New Zealand found it easy to count to seven, so did the children in the Netherlands. We did notice that some items behaved a little differently, mainly within the vocabulary part of the assessment. For example the words 'pigeon' and 'wasp' were much harder, relatively speaking, for Australian children than English children. These unusual biased items, and others like them, were dropped from the next stage of the analysis.

Children in different countries start school at different ages and looking at the relationship between starting points and age is vital. The three graphs do this for vocabulary, reading and mathematics development.

The curving line on each graph shows how vocabulary, reading and mathematics scores rise as the pupils get older. Each group is represented by a rectangle; the centre of the rectangle is the mean score for the group. The length of the side of each rectangle is an indication of the confidence that we have in its position. The longer the side the less sure we are of the precise score and the biggest squares are for the smallest groups.

Graph 1 Vocabulary

The children in the EAL group were assessed on their English vocabulary, which they were just beginning to acquire, and so it comes as no surprise to find that their scores were low for their age. There were also low scores among the deaf group, which included some children whose first language is British Sign Language (BSL). Again this is no surprise and we know from other analyses that even mild hearing loss is associated with lower than expected vocabulary levels.

The children in the Australian indigenous group had low scores for their ages and there is more than one likely explanation for this discrepancy: amongst other things the indigenous population, on average, have low social and economic status, as well as a very high incidence of hearing loss. The mean score of the Dutch group was high and raised suspicions that this may be because of the difficulty of translating a vocabulary assessment. An alternative explanation might be that the Dutch group was much smaller than the others and the children within it might have had characteristics that were atypical of the country as a whole.

One point, which the eagled eyed reader may have spotted on the diagram, is worth noting. The New Zealand rectangle is very narrow and that is because, unlike other groups shown on the diagram, the children all start school at the same age - on their fifth birthday.

Graph 2 Reading

There is less variation around the line for the mean reading scores than for the vocabulary scores. The assessment was presented to the children in such a way that their English language competence was less important than in the vocabulary section. For example, they were asked to "point to some writing" and they were asked this in their own language, be that Bengali, BSL or whatever. As a result the larger discrepancies seen in the first graph are largely ironed out. The surprises come from Scotland, New Zealand and WA (Year 1).

It seems likely that pre-school had an impact on the scores of the Australian children starting Year 1. They had higher than expected scores for reading, which might well have been directly taught. It also seems that this might be the explanation for the high scores in New Zealand, where the influence of Marie Clay's work might be being seen. On the other hand the lower Scottish scores present a puzzle and no explanation is offered here.

Graph 3 Mathematics

As with the reading results, there is much less variation on maths scores between groups, with the exception of Australian children starting Year 1 and the Australian indigenous group. The maths scores of the Scottish group are now in line with expectations.

Useful international reports

Progress in International Reading Literacy study - www.nces.ed.gov/surveys/pirls
Programme for International Student Assessment - www.pisa.oecd.org
Trends in International Mathematics and Science Study - http://nces.ed.gov/timss/

Subscribe to Literacy Today