I experienced a methodological conundrum in the planning phase of my research. I explicitly wanted to focus on the reading comprehension of first-year South African students, but I could not find published examples of similar studies. What also contributed to my confusion was that, apart from knowing I wanted to study reading comprehension, I had difficulty generating specific objectives needed to determine the research design. Castles, Rastle and Nation (2018:28) explain reading comprehension as the ‘orchestrated product’ of a set of processes, and I did not know how to identify research objectives for this set of processes.

As I read about the value of both qualitative and quantitative reading comprehension research, I decided that more than one data source would be ideal for studying the multifaceted process of reading comprehension, given the fact that this can only be indirectly observed. Mixed-methods research is suitable for problems ‘in which one data source may be insufficient’ (Creswell & Plano Clark 2018:8), and multiple forms of inquiry would enable me to understand the reading comprehension abilities of undergraduate students in their first year of study, in a more complete way.

Creswell and Plano Clark (2018:1) state that ‘mixing methods is an intuitive way of doing research’, but as a novice in the first phase of a research project, I felt that my mixed-methods intuition was non-existent. When I reflect on its absence, I lacked knowledge and skills and was only at the beginning of acquiring sufficient professional experience (Guetterman 2017). I was in desperate need of some direction for determining the methodological design of my research project, and the sheer amount of information in mixed-methods research textbooks was overwhelming at the time. Sakata (2022:2) uses the term ‘messiness’ to describe the unique features of a mixed-methods research design, and I felt lost in this messiness. To find direction, I put the methodological literature aside and focused on my topic, namely reading comprehension.

I identified several models that conceptualise the reading comprehension process. The comparison of models falls beyond the scope of this article. The model, or heuristic, that I found most suitable for my research context was developed by the RAND Reading Study Group (RRSG) in a report on a long-term study of reading comprehension published in 2002. This study group acknowledge that reading comprehension research is problematic due to the ‘absence of […] theories and models to provide a coherent foundation’ (RRSG 2002:2). This study group identified substantial shortcomings in the knowledge base of different reading comprehension models. To address these shortcomings, the RRSG created a definition of reading comprehension and developed a reading comprehension heuristic (RRSG 2002:12).

According to this group, reading comprehension is ‘the process of simultaneously extracting and constructing meaning through interaction and involvement with written language’ (RRSG 2002:11). The heuristic points to the interrelated variables present when one reads with comprehension. The reader, text and reading activity are the three central variables. In addition, reading always happens within a fourth variable, the sociocultural context. This context refers to a reader’s daily environment. I preferred this heuristic to others as it was uncomplicated, and the four variables were instantly recognisable in the higher education context. The heuristic shows that reading comprehension is the result of four variables that interact with each other. Each variable on its own and in combination with the others must be considered when planning reading support (Woolley 2011). The heuristic provided me with the direction I needed to develop my research design.

The application of the heuristic to the undergraduate context enabled me to identify the possible data that could be collected and studied within each variable. This process also assisted me in constructing my research questions and hypothesis. Table 1 was part of my planning and indicates how I mapped out what data I could collect, the classification of the data and the identification of the data collection methods.

The tabularised information confirmed the suitability of a mixed-methods research design for my research project, as it became clear that quantitative and qualitative data were obtainable in my context. I continued to refine my qualitative questions and hypotheses (Andrianatos 2028:7). Reflecting on the recommendations of Creswell and Plano Clark (2018:165), I could have added the following mixed-methods research question: What are the reading comprehension challenges that South African first-year students experience at a university?

After the completion of my research project, I began to read more about mixed-methods integration. According to Hitchcock and Onwuegbuzie (2022:29), while much has been published about integration in mixed-methods research, ‘there is something elusive’ about it. They provide three reasons for this elusiveness, namely a lack of published integration examples, the room a researcher has for innovation in mixed-methods research integration, and the fact that integration is ‘not easy’ due to the different ways researchers understand data (Hitchcock & Onwuegbuzie 2022:31). Looking back, I can now see that I approached the integration intuitively, guided by the heuristic. I based the integration on my understanding of how the answers to the research questions and hypotheses fit into each variable. Retrospectively, I present Figure 1 to demonstrate my intuitive integration approach. I then moved on to develop my research design before planning the integration in more detail.

I planned to gather quantitative and qualitative data to learn more about each variable in the heuristic (Figure 1) and their interconnectedness. I adopted a pragmatic position as this approach is generally associated with mixed-methods research. Furthermore, researchers state that pragmatism is the preferable philosophical foundation to justify combining different methods in one research project (Creswell 2003; Maree 2007; Teddlie &Tashakkori 2009).

The data collection methods, scale of the research project, details, and in-depth inquiry needed led me to choose the convergent mixed-methods research design, specifically the ‘parallel-database variant’ (Creswell & Plano Clark 2018:73). The choice of this design was based on my intent ‘to obtain different but complementary data’ on reading comprehension (Creswell & Plano Clark 2018:68). Another reason for the design choice as identified by these two researchers was the limited time frame I had to gather data. I wanted to include students and lecturers involved in the same programmes to ensure that the quantitative and qualitative data involved the same participants. For this reason, I had to collect the data in a single semester (one phase), as student and lecturer pairings often change in different semesters. The convergent design also enabled me to directly compare lecturers’ and students’ perspectives (of students’ reading comprehension abilities; students’ use of reading strategies; student and lecturer perceptions of the assessments and prescribed texts) with my own perspectives informed by the quantitative instruments (the survey of reading strategies; scores of assessments; scores of the reading comprehension tests). I could ‘give voice’ to the students and lecturers as well as ‘present statistical trends’ with the use of this design (Creswell & Plano Clark 2018:72). These researchers also identify four major steps in the convergent design that I had to apply in my design. Firstly, quantitative and qualitative data are collected in a separated manner; secondly, the data of the two sets are analysed independently, and they are of equal importance. In step three, the data are merged or integrated and in step four, the interpretation takes place.

In the convergent design, integration involves ‘merging or bringing together the quantitative results with the qualitative results’ (Creswell & Plano Clark 2018:71). As I used the reading comprehension heuristic as a point of departure for my research design, I planned to make use of the heuristic with the variables as major topics or themes to organise the integration of the quantitative and qualitative results. Creswell and Plano Clark (2018:71) confirm that data can be integrated by ‘qualitative themes’. Looking back at the position I found myself in as a novice, I agree with Collins (2022) and admit that my own competency level of mixed-methods integration was a hindrance to the planning of my integration for the research design. Apart from knowing how the answers to the research questions fit into the four variables (Figure 1), I struggled to make the points of integration explicit in my initial research design. I indicated the integration as a single box, labelled ‘merging of data’ (Andrianatos 2018:60), but I have come to realise that this is not sufficient. In retrospect, the integration had to be more explicit so that the reader of the research could comprehend what exactly took place inside this ‘merging box’. Schoonenboom (2022) wrote a chapter about the meta-inference in mixed-methods research and, in my opinion, her exposition of mixed-methods integration complements the approach I employed in my own research, namely using qualitative themes to integrate the data. Schoonenboom (2022:98) defines a meta-inference as ‘a conclusion that connects or integrates various claims’ that result from the analysis of either quantitative or qualitative data. The quantitative and qualitative data are analysed separately, and quantitative and qualitative findings are reached. These findings then lead to separate conclusions or claims. It is at this point that integration happens: the claims are integrated, leading to a meta-inference – a ‘highly valued outcome in mixed-methods research’ (Schoonenboom 2022:98). In terms of the needs of my design, the merging of the claims was suitable. I first came to conclusions about each of the variables and, thereafter, I integrated the claims based on the variables and their integration. In 2018, I could not think about my integration on a meta-cognitive level or explain the points of integration in my design. The publication of Schoonenboom’s (2022) chapter provided me with professional experience and has enabled me to better reflect on the route I followed in the design of the integration of my research project.

I have now shared the progression of my planning by identifying the available data, classifying the data and selecting the methods of data collection (Table 1). I also demonstrated how the quantitative and qualitative data fit into the reading comprehension heuristic (Figure 1). I also explained my integration approach. Based on design examples from Creswell (2003:62–67), I have adapted my 2018 convergent mixed-methods research design and have added the claims of each variable, the integration of the qualitative and quantitative claims for the activity and reader and how I reached the meta-inference (Figure 2).

Content analysis and a coding process were used to analyse the responses to the questions of the semi-structured interviews and focus group interviews, and the faculty-specific documents. Twenty-two excerpts of academic documents were analysed with the Coh-Metrix Common Core Text Ease and Readability Assessor (T.E.R.A). This online tool was designed to analyse the readability and ‘feasability’ of texts (Graesser, McNamara & Kulikowich 2011:223).

The research project in which I executed the research design received ethical clearance from the Ethics Committee of the Faculty of Education of the North-West University, with the approval number NWU-00311-15-A2. All ethical guidelines were followed, and student and lecturer participants provided written consent.

Firstly, the statistical analyses of the survey enabled me to rank the reading strategies that the students reported using, from most used to least used. The reading strategy least used by the students who participated is the use of reference materials like dictionaries. The integration of the qualitative analysis on reading strategies confirmed that students do not make use of reference materials while they read. Students were reluctant to use these resources independently and reported that they would rather ask a lecturer for an answer, as this verbatim quote from a student illustrates, ‘I want an explanation before I read the textbook’ (female, student, 19-years-old, Faculty of Humanities). This was affirmed by lecturers; as one lecturer commented, ‘The students are lazy, and they want an explanation’ (female, lecturer, Faculty of Humanities). The integration of data thus enabled me to better understand the reasons for the results of the questionnaire, and by integrating quantitative and qualitative findings about the reader, task and the text, I was able to come to the following conclusion (Andrianatos 2018):

A second example of the value of integrating the claims is an explanation for a null hypothesis. The relationship between task achievement and reading strategy use was not strong enough ‘to justify accepting the alternative hypothesis’ (Andrianatos 2018:224). One reason for the absence of a statistical as well as practically significant relationship between task achievement and the use of reading strategies was evident in the qualitative data of two student groups, namely the students from the Faculty of Law and the Faculty of Engineering. After the focus group analyses, it was evident that these two groups of students did not rely on reading the prescribed academic text for any assessment. The Microsoft PowerPoint slides of the Engineering lecturer were sufficient for a multiple-choice test and for the task that the Law students had to complete in groups, it seems that the students were more focused on the presentations than the critical reading of the prescribed court case. This integration emphasised that if lecturers require the reading of a prescribed academic text for an assessment, they should design the assessment (task or activity) in such a way that it is dependent on the reading of the text.

As these two findings illustrate, the use of the mixed-methods research design advanced a greater understanding of the challenges inherent to the first-year student reader, prescribed texts, activities (assessments) and disciplinary contexts involved in reading comprehension. A quantitative methodology would have, for example, lacked explanations for the relationships between variables and may have placed too much emphasis on the assessment of reading comprehension. Although this is important, in my opinion, looking at assessment alone cannot provide the complete picture of reading comprehension. A qualitative methodology would have emphasised the perceptions of the lecturers and students, together with the document analyses. Such a methodology could make important contributions, but I would not have been able to investigate the relationships between the variables or what the absence of such relationships means for reading comprehension. The research design (Figure 2) enabled me to gather, analyse and integrate quantitative and qualitative claims connected to four variables in a reading comprehension heuristic (Figure 1). My meta-inference is that there is no easy fix for challenges students experience with reading comprehension. The challenges within each of the four variables should be addressed in a connected way by the role players involved. This meta-inference led to the design of a reading support framework (Andrianatos 2019).

Looking back, I identified a methodological limitation of my mixed-methods research design. It is clear in Table 1 that most of the data within the variables were collected qualitatively, with quantitative data collection only present in the variables of the reader and the task. When I look at how both data sets enriched the findings regarding the reader and the activity, I realise that quantitative data would also have enriched the findings within the variables of the text and context. In the planning phase of my research project, I chose data that was easily recognisable and accessible within each variable. I could have spent more time developing quantitative instruments like questionnaires about the text and the context to also enrich the findings within the variables of the text and context.