Investigating measurement invariance in PIRLS 2021 across English and isiZulu language groups

Abstract

Background: PIRLS 2021 results revealed that South African Grade 4 learners performed significantly lower compared to other countries in reading comprehension and that they did not reach the standardised international mean score of 500. It was also evident from the results that English learners performed relatively higher than isiZulu learners.

Objective: This study was initiated to investigate whether possible measurement invariance can help explain the difference in the scores of English and isiZulu learners. Measurement invariance is concerned with the metric and functional equivalence of the items in the two languages examined.

Method: A quantitative approach was utilised to reanalyse the PIRLS 2021 numerical data sets. Descriptive statistics were used to analyse the mean and raw scores. Rasch analysis was conducted to look for any Differential Item Function (DIF) in the items.

Results: Statistics revealed that there is a significant difference in the mean scores of the two languages. Analysis of raw scores provided evidence that some of the items lacked metric equivalence. Rasch analysis displayed that some of the items lacked functional equivalence.

Conclusions: The significant difference found in the scores implies that Grade 4 learners need more attention mastering the skill of reading literacy. The lack of metric and functional equivalence suggests that items in the passages need to be investigated and improved.

Contributions: This study has contributed to the International Large-Scale Assessment literature specifically relating to the equivalence of ILSA across languages. A significant gap in the learners’ scores was identified and measurement invariance helped explain the gap in the learners’ reading scores.