Do a student's NAPLAN test results give a genuine, quality assessment of a child's ability?

I will be engaging in the reasoning process, Investigation, from the Dimension 4 (Dimensions of Learning), in an attempt to meaningfully research the known, explore contradictions and confusions and come to a resolution for the question - Do a student's NAPLAN test results give a genuine, quality assessment of a child's ability?

Dimension 4 - Investigation

(left click on table to enlarge view)

NAPLAN

Known or agreed upon

NAPLAN stands for National Assessment Program- Literacy and Numeracy. It is a national assessment program that was introduced in 2008 to monitor development in literacy and numeracy. The NAPLAN tests assess students in Year 3, 5, 7 & 9 to ensure that they meet a national minimum standard. The minimum standards and tests were developed by Australian Curriculum Assessment & Reporting Authority (ACARA) in consultation with the state, territories, the Australian Government, public and independent  schools. The tests comply with Statements of Learning for English and Statements of Learning for Mathematics documents.

Over time the tests will be able to monitor progress of students and their support requirements. This year 2010 will be the first time since implementation that NAPLAN can produce data to monitor progress.  The analysis of student performance is processed centrally to ensure consistency across all states and territories.

NAPLAN is  a formal national test instrument for of all year 3, 5, 7 & 9 students providing comparable students performance data at the same point in time (annually in May).  

Assessment for Learning

Known or agreed upon:

Assessment for learning is the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there. (Assessment Reform Group 2002).

 

(left click concept map to enlarge)

This concept map was constructed from the 10 principles of Assessment for Learning. They are researched based principles, from Assessment Reform Group,  for assessing and guiding learners and classroom practices.

http://www.assessment-reform-group.org/publications.html to read the complete publication "Assessment for learning the 10 principles"

Contradictions and Confusions

NAPLAN is a national formal test which is administered to Year 3, 5, 7 & 9 students to ensure that they meet national minimum standards. As yet we do not have a National Curriculum in place. Each state currently has their own curriculum documents and guidelines and the tests are based on ‘Statements of minimum standards in literacy and numeracy’ therefore students across Australia are in differing places in their learning journey.

Queensland Study Authority (QSA) documents state that NAPLAN contains essential skills, knowledge, understanding and capabilities that students should have the opportunity to learn by the end of Year 3,5,7 & 9 however the tests are administered relatively early in the school year, annually in May. If the tests are designed to identify students that are not meeting national minimum standards then the timeframe to support them in their learning is also dramatically reduced as the data is not returned to the schools until the last week of Term 3. In independent schools this could give as little as eight weeks remaining in the school year.  NAPLAN is a single point in time test of limited test length that (set time constraints) which does not ensure success for students who take longer to process the information and formulate their answers.

NAPLAN has a largely multiple choice content, the Assessment Reform Group (ARG) suggests their research indicates low achievers, especially the older pupils, are more likely to minimise their effort and respond randomly or by guessing the answers.  Independent Schools Queensland (ISQ) in a briefing document, ‘Standardised Testing: Getting it right for Educational Improvement’, indicated that a multiple choice format favours boys and those of European back ground (Sacks 1999).  The NAPLAN test documents start off at an average standard and fairly quickly increase in complexity, leading to frustration and disillusionment in the average to low achievers. With this in mind students are placed in a testing situation where it is unlikely that they will experience success.  


The impact of practising for the test, in itself places stress and undue pressure on learners to achieve. While directly focussing on test preparation, the curriculum is inadvertently being narrowed and learning experiences across all curriculum areas is diminished.  It encourages them to adopt test-taking strategies designed to avoid effort and responsibility. Repeated practice tests are, therefore, detrimental to higher order thinking (Paris, Lawnton, Turner & Roth 1991).


The NAPLAN test documents have a linguistic focus and in some cases the readability of the questions rates higher than the age of the students participating in the test (ISQ 2010). Providing assessment tasks that cater for all learning styles are more likely to impact positively on student success and motivation.


With the introduction of the My School website the NAPLAN results seem to have become an assessment for accountability of schools and teachers and less an assessment for student learning. Publishing results in a league ladder style has been scrapped in many countries around the world due to the negative impact they have on student self esteem and they do not work to improve education standards (ISQ 2010).

Original Research

(Left click survey to enlarge view)

Survey Analysis

Q1.  71% of the survey group indicate that they strongly disagreed or disagreed that NAPLAN gives a genuine assessment of a student’s ability. The remaining 29% gave a neutral response to this question.

Q2. 71% of the survey group indicated that they strongly disagreed or disagreed that the format and sequence of the test ensures student success. The remaining 29% gave a neutral response to this question.

Q3. 100% of the respondents indicated that NAPLAN did not cater for all learning styles

Q4. 71% of the survey group indicated that they agreed or strongly agreed that practising for the NAPLAN test narrows the curriculum taught and diminishes the learning experience.  One respondent had a neutral view and one strongly disagreed with this statement.

Q5. 85% of respondents believed there was insufficient time available to provide learning support to students after the test data was returned to the school.

Q6. 57% of respondents believed that indigenous and ESL students were disadvantaged by NAPLAN testing. After discussions with some of the teachers it was found that not all teachers were aware of the rulings regarding ESL students and when exemptions applied.

Q7. This questions received some mixed responses with 28% were neutral and 28% agreed and the remaining 44% disagreed. There did not seem to be a clear consensus on whether the NAPLAN results were comparable with other assessment tasks.

Survey Conclusion

The general consensus of the teachers surveyed was that the NAPLAN test in its current format did not ensure success for all learners and that they would like to see a greater variety of tasks to cater for all learning styles, needs and backgrounds. It is believed that these changes would make participation for all students from all socio-economic and cultural backgrounds equitable.

School Based Examples

(As a teacher aide in learning support I have been involved with NAPLAN, especially with students requiring special consideration, since it’s inception in 2008.  

A)   A year 3 student (2010) works through the first 5 numeracy questions and begins to feel frustrated and demoralised. As the paper is multiple choice he decides to randomly shade bubbles without working out each problem. He achieves Band 4, reaching the National Average. Is this a genuine assessment of his ability or an assessment of his ability to shade random bubbles? Will he now get the support he requires in numeracy?

B)   A year 3 student (2010) writing task. She is an academically high achieving student but is reluctant to write (but she will type stories on a computer given time). She achieves Band 3 for writing. Is this a true assessment of her ability?

C)   A year 9 student (2010). Has received psychological testing with an IQ in mid 70’s. Not low enough to have an Educational Adjustment Plan (EAP) but in need of assistance with reading (current reading age approx. year 4). She has special provisions for NAPLAN (reader), but under the guidelines no assistance can be given in literacy but in numeracy the question can be read but not paraphrased or reworded. Is there any chance of this student being able to achieve success?  Is this a completely demoralising experience?  Do we not know what the outcome will be already?

Is NAPLAN ensuring success for all students?

Resolution

There is much discussion and controversy relating to NAPLAN testing. This is evidenced by the 270 submissions that have been made by various members of the community to the ‘Senate Inquiry into the administration and reporting of NAPLAN testing’. Many members of the teaching community are aware that NAPLAN testing is here to stay but are against NAPLAN testing in its current format. This can also be evidenced from my own survey responses received.

In an effort to increase the positive and reduce the negative impact NAPLAN has on student motivation and self esteem teachers can;

·        Reduce the focus on practising for the test and teaching directly to the test content. Weave the types of questions that they will encounter into everyday learning activities and games.(Dept of Ed & Earlychildhood development, Victoria) 

·        Provide opportunities for students to develop and use higher order thinking skills. Be creative and engage all learners.

·        Provide encouragement (on strengths) and feedback (especially on weaknesses) to students. Show the learner how to develop and improve in these areas

·        Recognise all educational achievement in all areas and encourage students to do their best

·        Develop their own personal skills in the area of assessment by attending professional development, continuing self education, discussion and collaboration with colleagues

·        Educate parents on the purpose of NAPLAN and how it works alongside the current assessment with the emphasis on learning goals not on performance goals

·        DoL 5 – Habits of mind – Productive mental habits help students to be successful learners in whatever circumstances they encounter(Marazano Pickering p261)

NAPLAN and how it fits with the 10 principles of assessment for learning (ARG 2010) .

1.    Is part of effective planning – the content of the NAPLAN test is woven throughout the curriculum for Literacy and Numeracy

2.    Focuses on how students learn – linguistic focus and does not compliment all learning styles

3.    Is central to classroom practice – As test papers of marked independently it takes away from the classroom practice of observation, discussion, reflection, peer and self assessment and decision making

4.    Is a key professional skill – the test papers are marked and scored independently taking away the class teacher’s involvement with assessment (observation & discussion). Feedback and support can be given to student when data is received at the end of Term 3.

5.    Has an emotional impact – higher achievers will feel a sense of pride when completing the test however it is unlikely that a lower achiever will have any positive emotional impact. Practicing for the test also increases the pressure students feel to do well.

6.    Affects learner motivation – the format of the test does not have a positive impact on student motivation.

7.    Promotes commitment to learning goals and assessment criteria – teaching to the test reduces commitment to curriculum goals and reduces learning experiences however the content should be included in everyday teaching practices through the literacy and numeracy curriculum.  

8.    Helps learners know how to improve – Data is received late in term 3 giving a relatively short period of time to guide and show students how to improve. Teachers need to acknowledge strengths and give constructive guidance on how to address weaknesses when the data is available.

9.    Encourages self-assessment- NAPLAN does not encourage self assessment. There is no peer or self assessment content in NAPLAN testing.

10. Recognises all achievements – (Is about recognising students abilities in all areas of educational curriculum) NAPLAN is limited to literacy and numeracy and recognising strengths in these areas however it does not recognise student strengths in other curriculum and activity areas.

NAPLAN is a single point in time test of limited test length. On its own the data cannot possibly give a genuine, quality assessment. The information obtained can only become part of quality assessment when it works alongside best classroom practises of student assessment for learning. It should not become the performance focus goal but should be used as a tool for helping to set learning goals.

Referencing

Australian Curriculum Assessment & Reporting Authority (ACARA) National Assessment Program – Literacy and Numeracy. Retrieved from http://www.naplan.edu.au

Barry, K and King, L (2000). Beginning teaching and beyond (3^rd edition).  Katoomba, NSW: Social Science Press

Black, P., Broadfoot,P., Daugherty,R.,  Gardiner,J., Harlen,W., James,M., Stobart,G.,
               William,D.Assessment Reform Group (ARG) 2002 Assessment for learning: 10 Principles.
               Retrieved from http://www.assessment-reform-group.org/CIE3.PDF

Black, P., Broadfoot,P., Daugherty,R.,  Gardiner,J., Harlen,W., James,M., Stobart,G.,
               William,D.Assessment Reform Group (ARG) 2002 Testing, Motivation and Learning.
               Retrieved from
               http://www.assessment-reform-group.org/TML%20BOOKLET%20complete.pdf

Department of Education and Early Childhood Development Victoria ‘Teaching to the test’ – does it help or hinder learning? Retrieved from http://www.eduweb.vic.gov.au/edulibrary/public/publ/research/publ/researcharticle_teaching_to_the_test.pdf

Independent Schools Queensland (ISQ) Standardised Testing: Getting it right for educational Improvement. Retrieved from http://www.aisq.qld.edu.au

Independent School Queensland (ISQ) Submission to the senate education, employment and workplace relations committee: Inquiry into the administration and reporting of NAPLAN testing. Retrieved from http://www.aisq.qld.edu.au/files/files/whatsnew/ISQ-SubmissionNAPLANSenateInquiry(June2010).pdf

Marzano, R.J and Pickering, D.J. (1997). Dimensions of Learning Teacher's Manual (2nd edition). Aurora, Co: Mid-continental Regional Educational Laboratory.

Parliament of Australia Senate (2010) Senate education, employment and workplace relations committee: Inquiry into the administration and reporting of NAPLAN testing. Retrieved from http://www.aph.gov.au/senate/committee/eet_ctte/naplan/submissions.htm

Queensland Study Authority (QSA) 3579 Test Reporting Handbook 2010. Retrieved from http://www.qsa.qld.edu.au/downloads/early_middle/3579_handbook_reporting_10.pdf

Queensland Study Authority (QSA) P-12 Assessment Policy. Retrieved from http://www.qsa.qld.edu.au/downloads/approach/qsa_assessment_policy.pdf

Images used in Blog background retrieved from http://mwalker.com.au/?p=134