9 Nov 2015

How reliable are NCEA exams?

7:30 am on 9 November 2015

NCEA exams start this morning with the level one maths exam.

The PPTA says the NCEA has problems caused by 'political meddling'.

About 145,000 students are expected to sit NCEA and Scholarship exams this year. Photo: 123RF

For the next four weeks, hundreds of thousands of teenagers will cram like crazy so they can either gain the last few credits required for their NCEA certificate, or secure an elusive merit or excellence endorsement. Meanwhile, thousands of parents try not to lose it as they watch their child avoid studying.

Many people regard the exams as the most reliable part of the NCEA system and hold the three-hour tests in higher esteem than the internal assessments conducted by teachers throughout the year.

There are also the Cambridge exams, tests in primary schools, and exams and tests at universities and polytechnics.

But how accurate are exams and tests at measuring students' capabilities?

Michael Johnston, senior lecturer in education at Victoria University, said exams were not as reliable as many people seemed to believe.

"Exams like any other educational measurement are not super reliable. Never have been. Never will be. And I think that there is a sort of a culture of thinking they're more reliable than they are.

"With that said, I think that there's been a lot of good work undertaken on the NCEA system over the last few years that has ensured that they're as reliable as possible given the timeframe within which they're run."

Dr Johnston said one of the things that affected the accuracy of exams was how people felt on the day they sat them.

"It's certainly the case that if you have a short space of time within which to demonstrate your achievement then things like feeling off-colour or nervous or worried about it or whatever it is can affect your performance and similarly you can get lucky and do better than perhaps you deserve to."

no caption

Three academics said the Qualifications Authority did a good job of making sure NCEA exams were fair and accurate. Photo: 123rf

University of Auckland senior lecturer Maree Davies said exams were not a foolproof method of assessment.

She said one of the first hurdles to overcome to make them as accurate as possible was asking the right questions in the right way.

"They are always dependent on the examiner's abilities to write questions that are not ambiguous for the students.

"The difficulty with exam questions is that they have to bring out the knowledge that the students have, but at the same time be open-ended enough that they can really stretch those kids who are capable of gaining an excellence pass in NCEA or say in the nineties for Cambridge."

Mrs Davies said multiple-choice questions were the hardest to get right.

"The semantics of the sentences need to be very clear to the students. They have done quite a bit of research on multiple-choice questions and you may have a good understanding about a topic but just the way the question is framed you misunderstand the options."

University of Auckland academic Earl Irving said students' results could vary because of who marked their test papers.

"Some markers are regarded as being easy, others as hard. So what's being looked for through the check-mark process is to make sure that that marker is consistently easy or consistently hard because then you can adjust for it if it is necessary."

All three academics say the Qualifications Authority does a good job of making sure NCEA exams are fair and accurate - and none want to do away with exams.

Dr Irving said it was good to have a mix of internal and external assessment.

"Knowing that exams are one of those things that you get a one-off chance and it's a hit and miss at the end of a year and it takes the whole block of work and so on, we spent many years trying to come up something that was a bit better than that, so the current system which involves both internal and external assessment, I think is a better way of getting a fuller picture of what the students are capable of doing and what they know."

NCEA and Scholarship exams continue until 4 December.

@RNZeducation asked Twitter followers how accurate they thought exams were. Here are some of their responses:

Get the RNZ app

for ad-free news and current affairs