(full report)
The profession needs valid, reliable and manageable instruments
for assessing adult literacy and numeracy, and the National Research
and Development Centre for Adult Literacy and Numeracy (NRDC) in particular
needs such instruments for its own research programme. This report provides
a review of existing assessment instruments.
There had been no previous thorough review of adult literacy
and numeracy assessment instruments used in Britain. A review was particularly
opportune in 2002, when this project began, because several new instruments
had appeared recently (the national tests of key/basic skills at Levels
1 and 2, the Basic Skills Agency's Initial Assessmentpack (2nd edition),
Cambridge Training and Development's Target Skills, the tests for the
Skills for Life survey of adult basic skills needs being conducted in
2002/03). A total of 15 quantitative, summative instruments used to
assess adult literacy and/or numeracy in Britain in the period 1991-2002
were identified, obtained and analysed. The analysis was carried out
against a checklist and framework derived from theory, previous analyses,
and the research team's experience.
The major criteria for useful instruments were that they
should be secure (unpublished, or not readily available), be aligned
to the new QCA National Standards, and (for use in research projects),
have parallel forms. No wholly suitable instruments meeting these criteria
were found. For the 2004 sweep of the British Cohort Study 1970 it was
recommended that the instruments used in the early 1990s be used with
some modification. ¡ö For NRDC's research projects it was recommended
that new literacy and numeracy instruments be commissioned.
The report also provides a brief history of quantitative
and qualitative assessment instruments used in Britain in the period
1972¨C2004, a brief review of some United States (US) instruments, and
criteria that should be met by good tests. During 2003 a new literacy
assessment instrument meeting all NRDC¡¯s requirements was developed
for NRDC by the National Foundation for Educational Research (NFER),
and a review of this is also included. However, for NRDC's numeracy
projects a less than fully satisfactory adaptation of items from the
2002/03 Skills for Lifesurvey of needs was developed in 2003. A review
of this is included, with a recommendation that a better instrument
be developed.
Brooks, G.,
Heath, K. & Pollard, A. (2005). Assessing adult literacy
and numeracy: A review of assessment instruments. London:
NRDC.
Download the report (pdf) from www.nrdc.org.uk
This review carried out by Prof Murphy from Nottingham
University on behalf of the Association of Teachers and Lecturers (ATL)
investigates how examination results are produced and the ways in which
they are used. In particular, this research explores the dependability
of using individual grades as the ultimate statement of the achievements
of students in individual subjects, of comparing the performance of
students in different subjects, and of comparing standards across periods
of time. This study cautions that examination results are not exact
measures, and therefore need to be used with a full understanding of
their limitations as indicators of educational achievement. Furthermore,
it warns that decisions about assessment involve compromises, and that
there is a risk of giving grades unwarranted significance in forming
judgments about individuals or about the success of teachers, schools
or the education system.
Murphy, R. (2004). Grades of uncertainty. Reviewing
the uses and misuses of examination results. London: Association
of Teachers and Lecturers.
Order this publication from www.atl.org.uk.
This LSDA/LSRC report reviews the research literature
on assessment in post-compulsory education in the UK and explores how
assessments are experienced by learners and how they affect motivation
and learning. This review indicates that learners across all sectors
prefer coursework assessment and practical tests over end-of-course
texts, which may precipitate drop out and deter progression. Furthermore,
assessment policies and methods that encourage the active engagement
of tutors and learners in feedback processes are more effective in improving
retention and raising achievement than those that do not. Overall, a
key finding is that more research is needed to identify what kinds of
support have a beneficial effect in what circumstances.
Torrance, H. & Coultas, J. (2004). Do summative assessment
and testing have a positive or negative effect on post-16 learners'
motivation for learning in the learning and skills sector? London: LSDA.
Download the report from www.lsda.org.uk.
This review was undertaken to provide some research evidence
about the dependability of summative assessment by teachers and the
conditions that affect it.
Concerning the reliability of assessment based on teachers' judgments,
this review found that the reliability of portfolio assessment where
tasks were not closely specified was low; studies of the National Curriculum
Assessment for students aged 6 and 7 in England and Wales in the early
1990s found substantial error and evidence of bias in relation to different
groups of students; studies of the NCA for 11-year-olds in England and
Wales in the later 1990s showed that results of TA and standard tasks
agreed; and that the clearer teachers were about the goals of students'
work, the most consistently they apply assessment criteria.
Concerning the validity of assessments based on teachers' judgment,
this report highlights that teachers' judgment of the academic performance
of young children are influenced by the teachers' assessment of their
behaviour - which adversely affects the assessment of boys compared
to girls; teachers' judgments of students' performance are likely to
be more accurate in aspects more thoroughly covered in their teaching;
and that the introduction of TA as part of the national curriculum assessment
initially had a beneficial effect on teachers' planning, whilst there
was a decline in later years as support for TA declined and the focus
changed to other initiatives.
This review further shows that there is bias in TA relating to student
characteristics, including behaviour, gender and special educational
needs. Similarly, academic achievement and verbal ability may influence
judgment when assessing specific skills. There is variation in the level
of TA and in the difference between TA and standard test s or tasks
that is related to the school. There is also a call for teachers to
follow agreed procedures if TA is to be sufficiently dependable to serve
summative purposes.
These findings are evaluated in the light of their implications for
policy, practice and research.
EPPI-Centre (2004). A systematic review of the evidence
of reliability and validity of assessment by teachers used for summative
purposes. London: EPPI-Centre, Institute of Education, University of
London.
Download the full report at eppi.ioe.ac.uk
This is the first thorough review of adult literacy and
numeracy assessment tools used in Britain. 15 instruments developed
in Britain between 1992 -2002 were identified, obtained and analysed
against a checklist and framework derived from theory, previous analyses
and the research team's experience. According to this review, there
are no suitable tools for measuring progress made in research intervention
studies in literacy. New assessment tools are currently being developed
by the NRDC team.
Brooks, G., Heath, K. & Pollard, A. (2003). Assessing
adult literacy and numeracy: A review of assessment instruments. London:
NRDC.
Download the summary of the report at www.nrdc.org.uk
This QCA reports outlines the findings from the first
large-sample experimental comparisons of large-scale UK public examinations/tests
from different years. It concludes that TAs have shown less change in
standards than the national tests. The authors state, 'National testing
in its current form is expensive, primarily because of the external
marking of the tests, and the time may soon come when it is thought
that these resources may make a better contribution elsewhere' (Massey
et al., 2003, p 239).
Massey, A., Green, S., Dexter, T. & Hamnett, L. (2003).
Comparability of national tests over time: Key stage test standards
between 1996 and 2001.
Download the full report from http://www.qca.org.uk/
The aims of this review by the EPPI-Centre were to conduct
a systematic review of the research evidence of the impact of summative
assessment and testing on students' motivation for learning; to determine
the conditions and processes that have a positive or a negative impact
on students' motivation; to identify the actions that could be taken
to increase the positive and decrease the negative impact of assessment
on students' motivation; to consider the evidence relating motivation
for learning to learning achievements and learning strategies; and to
make recommendations for policy and practice based on these findings.
The main findings included:
- after the introduction of the National Curriculum Tests in England,
low achieving pupils had lower self-esteem than higher-achieving
pupils, whilst there was no correlation between self-esteem and
achievement prior to this
- when passing tests is high stakes, teachers adopt a teaching style
that emphasises transmission teaching of knowledge, thereby favouring
those students who prefer to learn in this way and disadvantaging
and lowering the self-esteem of those who prefer more active and
creative learning experiences
- repeated practice tests reinforce the low self-image of the lower-achieving
students
- students are aware of a performance ethos in the classroom and
that the tests give only a narrow view of what they can do
- students dislike high-stakes tests, and particularly girls show
high levels of test anxiety and prefer other forms of assessment
- teacher feedback that is ego-involving rather than task-involving
can influence the effort students put into further learning and
their orientation towards performance rather than learning goals.
- high stakes assessment can create a classroom climate in which
transmission teaching and highly structured activities predominate
and which favour only those students with certain learning dispositions
- an education system than puts great emphasis on evaluation produces
students with strong extrinsic orientation towards grades and social
status.
Older students (aged 11 +) are more likely than younger
ones to have a better understanding of simple grades are less likely
to report teachers' grades as being fair but attached more importance
to them. Also, lower-achieving students are doubly disadvantaged by
summative assessment since being labelled as failures has an impact
not just on current feelings about their ability to learn, but lowers
further their already low self-esteem and reduces future effort and
success. This review concludes with several implications for assessment
practice and policy.
Harlen, W. & Deakin Crick, R. (2002). A systematic
review of the impact of summative assessment and tests on students'
motivation for learning. London: EPPI-Centre, Social Science Research
Unit, Institute of Education.
Download the full report at eppi.ioe.ac.uk
80px.gif)
