Language test masquerading as literacy and numeracy test

Gunbalanya School in West Arnhem Land (Source:

Last week, the results of the 2012 National Assessment Program – Literacy and Numeracy (NAPLAN) were published. As has been the case since NAPLAN was first introduced in Australia in 2008, the Northern Territory (NT) has, once again, underperformed dramatically. More than 30% of Year 3 students in the Territory perform below the national minimum standard in Reading, Writing, Spelling and Numeracy. For Grammar and Punctuation the number of NT students performing below the national minimum standard is close to 40%. Across Australia as a whole, these numbers are between 5-7%.

Around 40% of students enrolled in NT schools are indigenous. Across Australia as a whole, that number is 4%.

Putting two and two together, it won’t be long before we’ll see yet another highly politicised debate about aboriginal education. Conservatives will blame ‘underperforming schools’ and progressives will blame ‘systemic socio-economic disadvantage.’ As usual, both sides will be right and wrong in their own ways and after a while the failure of aboriginal education in this country will be shelved as too intractable for yet another year.

Meanwhile, few will stop to consider that NAPLAN doesn’t actually tell us anything about literacy and numeracy achievements in remote NT schools because NAPLAN is a test designed and standardized for first language speakers of English while English is a second language (ESL) across remote NT locations.

Those who do recognize the fact that aboriginal children are being tested in an additional language on a test designed for first language speakers usually dismiss that problem as minor, as, for instance, Indigenous Education 2012 does. The authors of that report argue that language is not an issue because it is not an issue for migrant children for whom English also constitutes an additional language. Indeed, the difference between migrant ESL test takers and first language test takers seems to be only 1 or 2 percentage points on average, with many ESL students outperforming their mother tongue peers.

A recent article in the Australian Review of Applied Linguistics explores the fallacies of the argument that language does not matter in low literacy and numeracy achievements in the NT. The authors, Gillian Wigglesworth, Jane Simpson and Deborah Loakes, argue that there are a number of linguistic challenges faced by aboriginal students in remote locations when it comes to literacy and numeracy assessment in English.

First, most ESL kids in cities grow up in suburbs where English is the language of wider communication. School is thus rarely the only domain where they are exposed to English. This is different in remote communities: English is often exclusive to the school.

Second, most migrants come from literate backgrounds where education is highly valued. This is usually not the case in remote indigenous communities.

Third, the problems inherent in speaking two clearly distinct languages are much easier to recognize and to address than the problems inherent in speaking a different language that is not recognized as such. While aboriginal languages have become relatively rare, most indigenous people in remote locations now speak Kriol. Creoles spoken in Australia differ widely but most have English as the lexifier language and are structurally based in an indigenous language. Australian creoles thus often sound like English but may, for example, not have subject-verb agreement nor distinguish singular and plural. When examining Year 3 NAPLAN sample tests, the researchers identified many linguistic problems that would have made the test misleading to a Kriol speaker.

As an example they examine the spelling test item: “We jumpt on the trampoline.” Test takers have to correct the underlined item. Leaving aside the fact that presenting an incorrect item to a learner is highly problematic in itself, test takers would need to identify that “jumpt” is in the past tense and that the final [t] sound is therefore graphically represented as <ed>. However, past tense in Australian creoles would usually be realized as bin jamp. This spelling item is thus testing grammatical knowledge that Kriol speakers are unlikely to have.

The problem is compounded by the fact that ear infections are extremely high in remote indigenous communities and about 70% of all children there are affected by some form of hearing loss. Final stops such as [t] are extremely difficult to hear with high frequency hearing loss.

The problem is also compounded by the fact that the reading passages in the test are littered with cultural concepts quite alien to the experience of children in remote Australia. The sample tests examined by Wigglesworth, Simpson and Loakes are populated by cinemas, paperboys, picket fences, letter boxes and parking meters – none of which exist in remote communities.

In sum, the researchers demonstrate that the NAPLAN test is linguistically and culturally problematic for creole-speaking children in remote communities. A standardized test designed for first-language speakers of English will always fail second-language speakers who are not even recognized as such.

In contrast to all the big problems bedevilling aboriginal education in this country, the language problem is actually relatively easy to fix: bilingual education with the use of the mother tongue in the early years of schooling and simultaneous systematic instruction in English as an additional language work well in minority contexts elsewhere. And, of course, tests designed for the actual population of test takers rather than an imaginary monolingual mother tongue speaker of Standard English. Gillian Wigglesworth, Jane Simpson, & Deborah Loakes (2011). NAPLAN LANGUAGE ASSESSMENTS FOR INDIGENOUS CHILDREN IN REMOTE COMMUNITIES: ISSUES AND PROBLEMS Australian Review of Applied Linguistics, 34 (3), 320-343.
Available for open access here.

Author Ingrid Piller

Dr Ingrid Piller is Professor of Applied Linguistics at Macquarie University, Sydney, Australia. Ingrid’s research expertise is in the fields of intercultural communication, bilingual education and the sociolinguistics of language learning and multilingualism in the contexts of migration and globalization.

More posts by Ingrid Piller
  • Suzana Stapar

    Dear Ingrid,
    I agree with your point of view and support it fully. It is a sad picture that is painted in the article and very dissapointing in the fact that we are letting down one whole generation of children in the indigenous communities. I would only like to add that, even though it looks as if ESL learners in the cities are better off when it comes to language testing, in many instances they do struggle, and with the current migrant cohort, many of them have parents who are illiterate in their first language, and struggle with learning English, too, thus are unable to help their children. I am an ESL teacher and am interested in the language acquisition. I think that a huge number of migrants, recent and long term ones have been marginalised due to their language, and some, sadly, will never learn.

  • Thanks, Suzana! I agree – migrant ESL kids come from all kinds of backgrounds, of course – it just didn’t all fit into one blog post 😉 Relevant blog posts about how language testing leaves migrant ESL kids behind are available here and here.

  • Hanna Torsh

    I particularly like the point you make about how the language problem isn’t that hard to fix – it needs acknowledgement of the actual language repertoire of the speakers and that the NAPLAN is not designed for them. I would argue that seeing language problems for language problems is always tricky in the Australian context, and that the more we can try to be accurate about what languages are actually being used in the community the better educational policy we can develop. One way to improve the general understanding of language would be to have more language learning going on, whether it be community, indigenous or foreign language learning. Getting away from monolingualism would be a start to a better understanding of where a problem does come from language and where it doesn’t, although of course there are many grey areas as you pointed out with the hearing issues in indigenous communities. It’s interesting that even with regard to English, it is only since the coming of large numbers of international students in Australian universities that Academic English has started to be explicitly taught, that is that it is acknowledged that there are different Englishes. Before that it was all glossed as English and if you didn’t speak “right” at school it was your fault, not that of an education system in which your language was systemically marginalised.

  • Khan

    Dear Ingrid,

    Very interesting post. I once taught on IELTS preparatory classes in my uni. While teaching them the listening section, I realised the educational and cultural gap between the content of the IELTS listening and the experiences of average Pakistani students. Even if Pakistani students master the listening skills in English , their world is entirely different from the world of the test. I really wonder what exactly they are tested on?

  • Pingback: Dodgy data and language misdiagnosis | Language on the Move()

  • Hanne Houbracken

    “Meeting the targets for life expectancy, reading and numeracy and employment seems an unlikely prospect at this stage”
    – Peter Harris, chair of the Productivity Commission.
    Sadly, as with many Indigenous issues in Australia, the government refuses to acknowledge the real problem, which is a combination of factors including issues of historical, health, social and educational disadvantage. Instead of focusing on the gap between Indigenous and non-Indigenous students, more attention should be paid to quality education for all. Resources should be available everywhere, especially in already disadvantaged remote communities. Only with equal education, equal literacy and numeracy levels can be achieved.

  • Jay Mi Tan

    “Bilingual education with the use of the mother tongue in the early years of schooling and simultaneous systematic instruction in English as an additional language work well in minority contexts elsewhere. And, of course, tests designed for the actual population of test takers rather than an imaginary monolingual mother tongue speaker of Standard English”. This excerpt illustrates great solutions, but most often than not, standardised or high-stakes tests are governed by regulated ‘bodies’ who sadly do not understand the significance or implications of the tests. In my opinion, many high-stakes testing or standardised tests to some extent are part of political moves. It is usually the case that the decision making parties involved provide solutions that they “think” are best for all stakeholders. However, the considerations of stakeholders such as test takers and teachers are often neglected, and the “best solutions” often affect the validity and reliability of testing.