Investigating the Efficacy of Partial Decomposition in Kit-Build Concept Maps for Reducing Cognitive Load and Enhancing Reading Comprehension

Nawras KHUDHUR, Aryo PINANDITO, Yusuke HAYASHI, Tsukasa HIRASHIMA

Summary :

This study investigates the efficacy of a partial decomposition approach in concept map recomposition tasks to reduce cognitive load while maintaining the benefits of traditional recomposition approaches. Prior research has demonstrated that concept map recomposition, involving the rearrangement of unconnected concepts and links, can enhance reading comprehension. However, this task often imposes a significant burden on learners’ working memory. To address this challenge, this study proposes a partial recomposition approach where learners are tasked with recomposing only a portion of the concept map, thereby reducing the problem space. The proposed approach aims at lowering the cognitive load while maintaining the benefits of traditional recomposition task, that is, learning effect and motivation. To investigate the differences in cognitive load, learning effect, and motivation between the full decomposition (the traditional approach) and partial decomposition (the proposed approach), we have conducted an experiment (N=78) where the participants were divided into two groups of “full decomposition” and “partial decomposition”. The full decomposition group was assigned the task of recomposing a concept map from a set of unconnected concept nodes and links, while the partial decomposition group worked with partially connected nodes and links. The experimental results show a significant reduction in the embedded cognitive load of concept map recomposition across different dimensions while learning effect and motivation remained similar between the conditions. On the basis of these findings, educators are recommended to incorporate partially disconnected concept maps in recomposition tasks to optimize time management and sustain learner motivation. By implementing this approach, instructors can conserve cognitive resources and allocate saved energy and time to other activities that enhance the overall learning process.

• Publication: IEICE TRANSACTIONS on Information Vol.E107-D No.5 pp.714-727
• Publication Date: 2024/05/01
• Publicized: 2024/01/11
• Online ISSN: 1745-1361
• DOI: 10.1587/transinf.2023EDP7145
• Type of Manuscript: PAPER
• Category: Educational Technology

1.  Introduction

Reading comprehension is the skill of understanding a text and creating a mental representation of the knowledge gained from the text being read [1]. Reading comprehension requires a structured understanding of the text and the ability to transform the reading task from phrasal recognition into building propositions and then understanding the theme, hence, it requires high engagement and high text processing skills [2].

Concept map [3] is one method that provides such manipulation to support knowledge organization. Concept map is a graphical notation to represent and organize knowledge. It has a hierarchical structure showing the conceptual relationships of the subject of matter. Concept map consists of concepts as nodes and relational phrases as links. A link connects concepts together to create a meaningful unit of knowledge called a proposition. Thus, a proposition consists of one starting concept ― the source concept, connected to another concept ― the target concept, via a relational phrase ― a link. Many studies investigated the effects and benefits of concept mapping on reading comprehension [4]-[6]. Concept map equips the reader with the ability to connect different pieces of knowledge regardless of their order in the transcript. It also supports the readers to make connections between their prior knowledge and the new knowledge they gain while reading which is an important part of the reading comprehension process [3], [7].

In general, concept mapping has been put into two forms; building a concept map from scratch i.e. open-end concept map, and building a concept map from predefined nodes and links i.e. closed-end concept map (CCM) [8], [9]. In open-end concept mapping activity, learners begin with an empty canvas and make their way to build their concept map. On the contrary, in a CCM activity, learners use a finite set of concepts and links to build their concept map.

Kit-Build concept map (or Kit-Build), which adopts the CCM approach, further restricts the concept mapping activity to a set of concept map components provided by experts or teachers [10], [11] aiming to facilitate common ground understanding. In Kit-Build approach, domain experts or teachers create a concept map (expert map) of a learning topic and learners recompose the expert map by using the finite set of concepts and links generated by decomposing the expert map ― the kit. The concept map decomposition is by nature performed by disconnecting all of the connections between links and concept nodes from the expert map. The Full Decomposition (FD) approach of a Kit-Build concept map refers to this decomposition method. Figure 1 b) shows this traditional way of generating a kit from an expert map.

Fig. 1  An example of an expert map and the two decomposition approaches.

In Kit-Build approach, learners are able to focus on the recognition of given components and the relations between them since they are not required to create components (nodes and links) by themselves. Therefore, it is expected to promote structural understanding. A previous study showed the superiority of Kit-Build concept map recomposition compared to the summarizing method in supporting reading comprehension [12]. Kit-Build has been confirmed to be as effective as open-end concept mapping in reading comprehension and even outperforms it in the task of retention over the period of two weeks [13].

In the studies of Kit-Bulid concept map, learners’ understanding is defined as their ability to comprehend others’ perspectives, which can be referred to as empathetic understanding. It focuses less on arriving at the correct answer and more on grasping how someone else perceives a situation. Hence, it is crucial to emphasize that the absence of a learner’s ideas in the expert map does not necessarily imply their ideas are incorrect, rather it supports learners in their communication. For effective communication, the expert decides on the ideal knowledge to be shared by the learners through the expert map. The elements within the expert map serve as scaffolds to enhance this communication. In the learner-educator context, the learner gets a chance to reflect upon their understanding via educators’ understanding i.e. recomposing expert map. In the learner-learner context as in the case for reciprocal KitBuild [14], learners actively attempt to build the map of their partner, promoting effective communication among themselves. Being aware of both another person’s understanding and one’s own, can foster an environment for more effective discussion. This awareness helps learners appreciate the variations or gaps in comprehension among learners or with the educator. Viewing learning as a social activity underscores that this facilitation of communication inherently fosters learning itself.

Therefore, recomposing a kit into the concept map requires the skills to grasp the understanding of another individual (usually the teacher in the educational settings), which requires a high engagement from the learner [15]. In addition, the expert map is not shown during the recomposition activity of Kit-Build, encouraging the learners to think carefully about how to form the expected propositions using the given components only. Therefore, the task puts a lot of strain on learners’ working memory due to the gap in understanding between the learner and the map author. Learners need to recursively understand individual concept nodes, recognize two related concepts, and find the most suitable link to re-create the propositions until the whole concept map is fully recomposed back to the expert map. The pressure on the learner’s working memory during the concept map recomposition task of Kit-Build could impede their learning performance as confirmed by previous study [9]. Thus, a more efficient method for the Kit-Build concept map recomposition task is strongly demanded. An efficient recomposition task minimizes waste in the limited working memory resources during the recomposition task by lowering the cognitive burden on learners without sacrificing its benefits of learning.

In cognitive load theory (CLT), there are several suggestions to reduce the unnecessary cognitive load depending on the activity type. One such applicable suggestion is partial completion as it reduces the size of the problem space and lets the learners reach the solutions faster [16]. Based on this suggestion, this paper proposes a new technique for kit generation in which all of the propositions in an expert map are only partially dissected.

In this approach, instead of disconnecting both the source and target concepts from the link when generating the kit, the source concept is allowed to remain connected to its corresponding link while the target concept is disconnected for all of the propositions in the expert concept map. We call this decomposition approach, Target Search Decomposition (TSD). Figure 1 shows an example of kit generation for an expert map using both FD and TSD approaches. Figure 1 a) is an expert map used for demonstration. Figure 1 b) is a kit generated from the expert map using the traditional FD approach. The kit consists of a full set of concepts and links from the expert map with no connections among them. The learner should use this kit to recompose all the propositions back to the expert map without being able to refer to the expert map at any stage of the recomposition. Contrarily, Fig. 1 c) is a kit that is generated via TSD approach. The kit consists of a full set of concepts and links from the expert map where the connection between starting concept and the link is preserved. The learner is supposed to complete the connections for all of the links to recompose the same expert map; similarly without referring to the expert map during the recomposition.

In FD kit generation, even though learners may know the correct proposition, they would face the difficult task of manually searching through all of the given concept map parts to find the suitable pieces for each proposition which may not necessarily contribute to the learning process and could drain learners’ energy. While with TSD, learners can reduce the repetitive search task to only the target concept, hence, reducing the search space dramatically. This reduction affects the recomposition task of TSD kit by putting less load on working memory since the learner needs to keep less information in the working memory while searching for the target concept. Additionally, this reduction is expected to speed up the recomposition process safely by keeping the learner engaged in the important task of proposition completion. Completing the proposition by looking for the target concept needs the learner to review the proposition multiple times to be sure the target concept matches the proposition, which is believed to enhance comprehension.

Changes in the cognitive load can influence the learning outcome since the cognitive load is part of the learning process. Thus, in this research, we investigate the effect of TSD kit recomposition on cognitive load and learning quality in contrast to the FD kit recomposition approach. Similarly, changes in cognitive load can have an impact on the motivational state of learners, subsequently affecting the feasibility of the proposed approach. The rationale for this effect lies in the strong interconnection between motivation and cognitive load. When a challenge does not match the learners’ skills, it may result in learners’ loss of motivation and mental exhaustion [17]. Maintaining high levels of motivation among learners when using educational tools is a crucial aspect affecting the effectiveness of such tools [18]. This is because learners’ motivation is a key element in enhancing learning performance and overcoming challenges [19], [20]. Therefore, we are concerned about the case where decreasing the cognitive load leads to easier tasks to a degree that learners lose the motivation to engage with the learning activity. Flow state is one of the proposed methods to capture individuals’ optimal motivational state [17], [21]. Additionally, when learners are in an optimal motivational state where the skills and challenges are well balanced, learners tend to give a wrong reflection on their experienced cognitive load. [17] indicate that people may report low cognitive load experience on the subjective self-reported metrics even though the cognitive load experienced was high. This is a limitation of the subjective metrics of cognitive load that we want to control for it between the conditions.

The following are the research questions investigated in this study: Whether the concept map recomposition activity of TSD approach affects the: 1. Cognitive load. 2. Flow state. 3. Reading comprehension: 3a. Immediate reading comprehension. 3b. Retention of reading comprehension. In the first question, we aim to evaluate and describe the effects of TSD on cognitive load. The second question gives an insight into the changes that could happen to the nature of the activity in terms of difficulty versus the skill of the learners (i.e. flow state) on one hand, and on the other hand, it is used to confirm the strength of the cognitive load metrics. The third question shows the importance and value of the changes that may occur in terms of learning quality. This last question can show if the changes in the decomposition method are worth attention in the educational community.

The experimental results show that TSD kit recomposition has significant benefits in educational settings. Compared to the FD group, TSD learners experience a lower cognitive load while their motivation is still intact. Meaning that the lowered cognitive load is real. In addition, their activity is not becoming less challenging due to the narrowed problem space. More importantly, the results show that the learning quality of Kit-Build concept mapping with conventional kit generation is maintained despite reduced cognitive load across different dimensions. Thus, giving the TSD technique a significant overall advantage.

2.  Background and Literature Review

2.1  Concept map and Construction Types

Concept map is a graphical diagram that depicts knowledge in the form of interlinked concepts and relationships. Individuals could pour their understanding into concept maps. They could then use the map to convey their ideas to others more efficiently. The visual structure in a concept map could make important information and relationships between ideas clear and apparent.

Prior studies revealed that integrating learning activities with concept maps could significantly enhance students’ problem-solving skill performance [22]-[24]. In addition to promoting meaningful learning to students, using concept maps in learning could potentially improve students’ interests and learning achievements better than the traditional learning expository [25]. According to [26], concept mapping has the potential to enhance teacher-student interactions and foster the generation and expansion of knowledge. Even though prior studies indicated various challenges and limitations in utilizing concept maps in academic practices for both teacher and student [27], [28], concept maps can be accepted as an alternative method for learning [29].

Essentially, there are two types of concept mapping approaches in terms of how a concept map is constructed or developed. The first approach provides a flexible method where authors begin with a blank canvas, allowing them to freely incorporate and personalize their own interests, knowledge, and external resources into the concept map. This approach is commonly referred to as open-ended concept map (OCM) [30]. The second approach is referred to as a closed-end concept map (CCM). In CCM environment, the author is provided with a finite set of pre-selected concepts and links offering a different cognitive activity. CCM learning strategy could help teachers assess students’ understanding and improve the quality of their teaching [31].

This research utilizes concept maps as graphic organizers. By adopting concept maps, individuals can effectively organize and represent their understanding of a subject matter. This research specifically focuses on the recomposition of the teacher’s concept map, where learners are tasked with rearranging and reestablishing the interconnections between concepts and relationships as understood by the teacher to create a shared understanding within the classroom. Such understanding helps facilitate effective communication, collaboration, and problem-solving.

2.2  Kit-Build Recompositional Concept Map

Hirashima et al. [10] introduced recomposition concept mapping known as Kit-Build that adopts the CCM approach. As mentioned in the introduction, in Kit-Build, learners rearrange a set of concept map components created from the expert concept map. Expert concept map in the context of this study refers to the ideal map according to the understanding of a professional in the subject of matter. The expert map serves as the baseline for learners to have a shared understanding about a particular subject. The professional is the teacher in an educational setting. The basic steps of a Kit-Build activity are as follows: 1) The expert creates a concept map for a material. 2) The expert concept map is then decomposed into its basic parts by removing the connections, thus creating a kit of concepts and links. 3) The kit is then given to the learners to recompose it back to the expert map.

In Kit-Build, kit refers to the components that make up a proposition. In a proposition, there are two concepts connected by a single link. One of these concepts is known as the source concept. This concept represents the ‘subject’ or ‘agent’ within the proposition. It usually denotes the entity carrying out the action or being described by the proposition. The second concept is referred to as the target concept. It serves as the ‘predicate’ or ‘complement’ within the proposition, often providing information about the action or description associated with the subject. For example, in the proposition “Fish lives in the water”, The source concept is “Fish”, and the target concept is “in the water”. During the kit generation, the formation of the source and target concepts are derived automatically from the expert map based on the expert’s decision.

Since the concept of recomposition is central to our study, in the following sections we delve into key aspects of recomposition from theoretical and practical perspectives.

2.2.1  Recomposition’s Challenge and Learning Value: Theoretical Perspective

From a theoretical standpoint, recomposition emerges as a demanding cognitive process. When learners embark on recomposing a concept map from its specified components, the initial step involves recognizing these components correctly. Any misinterpretation at this stage may impede the entire recomposition process. Subsequently, learners must comprehend the meaning of each component and make connections that form meaningful propositions. Given the vast possibilities for connections, identifying the appropriate propositions to represent the target content’s essence is paramount. Moreover, recognizing the structure being recomposed is crucial to adequately complete the recomposition. For instance, previous research [32] reported that learners who recompose specific parts of a concept map sequentially obtained better learning results than learners who recompose a concept map randomly. This theoretical analysis underscores that recomposition is far from a simple replication task; it involves a profound understanding of the map’s meaning and structure.

Recomposition serves as a powerful tool for learning, primarily because it necessitates the active engagement of the learners’ cognitive processes. In the recomposition process, learners are required to draw upon their individual understanding. Correctly recognizing components, formulating propositions, and identifying substructures all demand the application of their own comprehension. As a consequence, recomposition inherently encourages learners to reflect upon their understanding. Learners cannot complete the recomposition if their understanding is not enough or wrong. Thus, recomposition serves as a mechanism through which learners become aware of gaps or inaccuracies in their knowledge, which, in turn, motivates them to improve their understanding.

2.2.2  Evidence of Recomposition’s Learning Value and Challenge: Practical Perspective

Recomposition, as a concept, is not merely theoretical; it finds validation in practical educational settings. Our university-level experiences substantiate that recomposition is no easy feat, even for students at this level. The effectiveness of recomposition concept maps has been demonstrated in several studies targeting university students, to cite a few [14], [32]-[38]. These studies have consistently reported significant achievements, providing empirical support for the theoretical underpinnings of the recompositional concept map.

In a recent investigation by [38], the significance of Kit-Build concept maps in fostering higher-order thinking was explored. Undergraduate students were assigned to two conditions: OCM and Kit-Build recomposition. The study found that Kit-Build was more effective in enhancing higher-order thinking skills than OCM. The authors contend that Kit-Build concept mapping enables students to allocate more cognitive resources to organizing and integrating knowledge, rather than becoming halted in defining concepts. By relieving students from lower-order tasks like concept definitions, they can more deeply engage in higher-order cognitive activities, such as analyzing relationships and synthesizing concepts, ultimately enhancing their higher-order thinking abilities.

Another study proposed reciprocal Kit-Build concept mapping. The study was implemented in a real classroom setting, allowing the reconstruction of a partner’s concept map. The proposed method revealed improved pair discussions about map differences before collaborating to construct a new map. This approach was deemed valuable for eliciting ideas, understanding partners, and integrating diverse perspectives [14], [34]. Furthermore, Pinandito [39] showcased that Kit-Build surpasses traditional OCM in collaborative environments, particularly in fostering meaningful discussions among students and enhancing overall learning outcomes. The Kit-Build approach encourages students to engage in discussions about the content of the concept maps, shifting the focus from procedural matters. The study also found that Kit-Build stimulates students to explore and contemplate ideas beyond those suggested by the prescribed components, thereby fostering curiosity and creating a “spread of effect” that motivates them to consider additional ideas.

2.3  Cognitive Load in Kit-Build Concept Map Recomposition

Cognitive load in learning concerns to the amount of information or instructional methods processed in one’s working memory during learning activities [16]. Because students have a limited capacity of working memory to process new information, instructors should efficiently manipulate students’ working memory with activities or learning tasks that directly contribute to learning [40]-[42].

Three categories of load are defined by CLT that includes intrinsic load, extraneous load, and germane load. [43], [44] explains each category of the CLT as follows. Intrinsic load, is a load related to the nature of the subject material and generally, it cannot be adjusted. Extraneous load, on the other hand, is generated by inappropriate instructional format. It is related to the cognitive effort that is not contributing to comprehending the subject material. Lastly, germane load refers to the utilization of memory resources to cope with the intrinsic load. Learning tasks should be designed in such a way that the available working memory capacity is efficiently used to achieve the highest return on mental effort investment. This means that extraneous load should be minimized so that working memory capacity is freed, which may permit an increase in the working resources devoted to intrinsic cognitive load (also called germane processing).

Several studies suggested that concept map composition is difficult and time-consuming [29], [45] and it also requires a significant effort to compose a good quality concept map for teaching [28], thus support for the authoring process in reducing the load is necessary [28], [46]. Composing concept maps from pre-selected concepts or links is considered more beneficial for students thus reducing the load in learning [47], [48].

One learning framework, which applies concept map recomposition strategy, is Kit-Build Concept Map [10], [11]. In its preliminary use, Kit-Build Concept Map was utilized to aid teachers in diagnosing students’ understanding. Recent studies related to Kit-Build concept mapping depicted how it could help both teachers and students in learning [28], [37], [49].

The nature of Kit-Build concept map activity requires the learner to hold multiple pieces of information in the working memory while searching for the matching node. As described by [9], recomposing a proposition in Kit-Build concept map could be in any order of Concept-Link-Concept (CLC) or Concept-Concept-Link (CCL) or Link-Concept-Concept (LCC). In CLC learner searches for a source concept first, then looks for an appropriate link while keeping the information about the source concept in memory. Finally, the learner has to look for the target concept while keeping the information of the source concept and the link in memory. Similarly, CCL requires finding the source concept first and keeping it in memory. Then finding the target concept before looking for the appropriate link between them. Again while keeping the information about both concepts in memory. In the LCC approach, the link is defined first, then the learner looks for the source concept while keeping the information about the link in memory. After finding the appropriate source concept, the search for the target concept starts while keeping the previously found information in memory. These search processes are repeated for each proposition until all of the provided nodes are reconnected recomposing the assumed correct propositions same as the expert map. [9] tried to reduce the cognitive load of the Kit-Build concept map recomposition by introducing auto-layout organization thus, removing the burden of positioning the pieces of the concept map. The results did show a reduction of the cognitive load in terms of activity time but at the cost of learning quality.

By introducing partial recomposition in this study, we change the instructional design of the Kit-Build such that the learner is allowed to make fewer searches to recompose each proposition. In CLT, this approach is described as partial completion [16]. It is said that partial completion could reduce the extraneous cognitive load by shrinking the search space. Thus, we define a new approach to generate the recomposing kit in Kit-Build concept map called Target Search Decomposition (TSD). In TSD, the generated kit has its source concept and link intact for each proposition. Reducing the search space could have positive impacts on the learning and cognitive load because the learner is allowed to keep the fewest possible information in the memory resulting in less repetitive work and increased focus. It could also have a negative impact on learning because it could make learners become less engaged with every piece of the given kit. These unknowns makes it necessary to investigate the impact of TSD on cognitive load and learning.

2.4  Flow in Kit-Build Concept Map Recomposition Activity

Flow is a psychological mental state that occurs when people are fully immersed or engaged in an activity. Flow referred to as the optimal state of motivation, can be experienced in daily life, including but not limited to sports, education, and creative working activities [50]. Flow also highly corresponds to a positive experience and is connected with one’s peak performance [51].

In learning, the flow state can be attained when learners get total focus and enjoy the learning activities at the same time. According to the flow model, flow occurs when learners’ current skill level and the challenge level of a learning task are balanced at their highest [52]. Performing the task while being in this flow state, learners could experience greater enjoyment, emotion, and happiness while doing the activity; hence, deeper learning and higher levels of satisfaction are reached [52]. Because flow could actuate one’s peak performance, tapping students into the flow state during learning activities becomes essential. The way how teachers design and constitute the learning environment to be more enjoyable and fun is necessary to cater to an optimal learning experience. Keeping learners in a flow leads to better use of educational tools and a more effective learning environment.

3.  Methodology

In this section, we provide a detailed overview of the experimental design and procedures to clarify the purpose and context of the experiment. The primary aim of our study was to investigate the impact of different levels of concept map decomposition (TSD and FD) on reading comprehension. To achieve this, we employed a between-subjects pretest-posttest design. The collected data encompass a range of variables, including reading comprehension scores, cognitive load measures, and flow experiences. These data were collected through a series of carefully designed tasks, as described below. Our analysis of this data will enable us to draw conclusions about the effectiveness of concept map decomposition levels in enhancing reading comprehension, shedding light on their pedagogical implications. Figure 2 shows the timeline of the experiment. The experiment consisted of two timely separate phases, the main phase, and an optional delayed phase.

Fig. 2  Timeline of the experiment

The main phase included the following tasks:

1. Take a demographic questionnaire.
2. Read usage instructions of Kit-Build tool.
3. Build a training map using Kit-Build tool.
4. Read a narrative.
5. Take the reading comprehension pretest.
6. Complete the core activity depending on the condition.
   1. TSD condition: Recompose a concept map from a kit where the source concept was already connected to its corresponding link in each proposition.
   2. FD condition: Recompose a concept map from a kit where the source concept, link, and the target concept were disconnected for all of the propositions.
7. Take a cognitive load questionnaire.
8. Take a flow measurement questionnaire.
9. Take the reading comprehension post-test.

Participants who successfully finished all tasks in the main phase were invited to take part in the delayed phase. The delayed phase was implemented after 2 weeks from the main phase. It consisted of only one task, namely the delayed reading comprehension test. The test was exactly the same as the main phases’ reading comprehension test.

3.1  Participants

A sample of 78 adults (39 for each condition) was recruited through an online crowd-sourcing platform Amazon Mechanical Turk also known as MTurk [53]. Similar to the traditional recruitment methods, the data that is collected using MTurk recruitment are reliable and even better in some terms [54], [55]. The recruitment task was published and managed via CloudResearch, an online crowd-sourcing platform that can be linked to MTurk. CloudResearch has easier task management and extends MTurk functionality to allow applying additional criteria to the recruitment process [56], [57].

Participants were recruited from July 19, 2021, to August 14, 2021. The recruitment task accessibility was restricted to the “CloudResearch Approved List” only to ensure the high quality of data and avoid automated programs. Additionally, in order to maintain consistency in language and context, participants were required to be residents of the United States or Canada, as the system and experiment were conducted in US English. Furthermore, participants were subject to specific eligibility criteria, including a minimum approval rate of 98% on the platform and a record of completing over 5000 tasks on MTurk. Participants were assigned to each condition randomly and permanently, no interchange was allowed. Participants who completed the initial task received compensation of \(\$6.00\), with a separate additional payment of \(\$0.80\) for those who later participated in the delayed phase. The rewards provided are intended to acknowledge and appreciate participants for their time and effort in the study.

3.2  Material

The narrative, comprehension questions, and expert map were adapted from a previous study material [15]. The narrative text¹ is based on a Wikipedia article about the komodo dragon but modified and shortened to 442 words with no graphic content. The comprehension questions consisted of 10 multiple choice items about the content of the material. Each question has 4 options including one correct answer. The question sequence and their corresponding options for each learner were shuffled between the pre, post, and delayed comprehension tests to reduce the chance of memorization. The expert map consisted of 17 propositions. From this expert map, two versions of the kit were generated for both conditions. TSD condition received a kit that is generated using the proposed approach; while the FD condition received a kit using the traditional approach.

A self-reported mental effort [58] and perceived difficulty [59] questionnaire were used to measure the cognitive load imposed by the concept map recomposition for both conditions. Both mental effort and perceived difficulty are important in measuring the cognitive load [59], [60]. The mental effort questionnaire is an indirect subjective method that has been widely validated and said to be sensitive to relatively small differences in cognitive load [61], [62]. Participants were asked to reflect on their mental effort on a 7-point Likert-type scale ranging from “Very low mental effort” to “Very high mental effort”. While the perceived difficulty questionnaire is a direct subjective measurement of the cognitive load [63]. The question asks the participants to reflect on their perceived difficulty of the task on a 7-point Likert-type scale ranging from “Very Easy” to “Very Difficult”. Both of the questionnaire has been confirmed and adequately used in previous related research by [9] to measure the same variables of perceived difficulty and perceived effort. In addition to the subjective methods, the user behavior during the concept map recomposition was captured to measure “time-on-task” and the “number of actions per proposition” during the recomposition activity. These variables are considered valid indirect objective measurements of the cognitive load [63].

Flow Short Scale adopted from [21] was used to measure the flow state during map recomposition for both conditions. The test has been used and validated by multiple research works [9], [64], [65] The test consisted of 10 items with a 7-point Likert-type scale.

3.3  Procedure

An online tool was built and used for this experiment named Kit-Build concept map. Participants used MTurk to connect to the tool and go through the consent form. In the consent form, they were informed about the experiment’s purpose, data collection, and data usage as well as the ability to leave the experiment at any time. The participants were also informed about conducting a follow-up test (delayed-test) for those who complete the current tasks successfully. Demographic questions followed the consent agreement. The demographic questions were about gender, age, and education level.

Afterward, they read the tool usage instructions and the flow of the required tasks. They also have been informed maximum time the experiment could take. At this point, they learned that once they proceed to the next step, all of the remaining steps will be automatically timed to a maximum of 5 minutes except for a map recomposition activity that will be 20 minutes. Thus, advised to keep focusing on the experiment and not to switch to external tasks. The time limit serves the purpose of keeping the participants engaged with the experiment and avoiding distraction by other tasks. Following the tool instruction section, they started a training map to get familiar with the tool features. The training concept map was unrelated to the rest of the experiment.

Next, the komodo-dragon reading material was given to the participants to read followed by the after-reading comprehension test (pre-test). After the pre-test, the participants were given a kit to recompose based on their condition. FD condition received a fully decomposed kit while TSD condition received a kit based on TSD approach where the first half of the proposition (source concept node and the correct link) was connected while the target concept was not. The time frame for this task was 20 minutes and the learner was restricted from moving to the next step until the concept map is completely and correctly recomposed or the designated time period is over. In that sense, learners could submit an incorrectly recomposed map after working on it for the specified 20 minutes. During the map recomposition, the learners could use a feedback feature to confirm the recomposition status. When clicked, the feedback feature highlights the correctly recomposed propositions and incorrectly recomposed propositions based on the expert map. This formative feedback directed the learners into looking for other alternatives to the wrong connections while keeping the correct ones unchanged. The questionnaire about concept map recomposition experience followed the recomposition step. The questionnaire was divided into two subsections, cognitive load, and flow-state measurement. Finally, the participants answered the after-recomposition reading comprehension questions (post-test) which consisted of the same questions as in the pre-test.

The second phase of the experiment (delayed phase) started in two weeks. Participants were contacted by email twice, the day before starting and at the time of starting the delayed phase. Access to answering delayed phase was open for 4 days to give a chance for more participation rate. Comprehension questions in this phase were the same comprehension test questions as in the main phase with the same time limit of 5 minutes.

4.  Analysis and Results

Data were collected from 78 participants using the provided online system, but 4 of them had to be excluded due to technical issues with data collection (N=74). At the outset of the experiment, we had emphasized that the system is fully compatible only with the latest versions of Google Chrome, Firefox, or Microsoft Edge, provided that JavaScript is not disabled. It is conceivable that these 4 participants encountered compatibility issues, which led to unsuccessful data collection.

Both conditions were compared in terms of each demographic data to ensure the demographic bias between the conditions. According to the chi-square test of independence, none of the demographic variables were related to the assigned conditions (gender: \(X^2 (2, N=74)=1.03, \mbox{P-Value} = .60\); age: \(X^2 (4, N=74)=1.93, \mbox{P-Value} =.75\)); education: \(X^2 (6, N=74)=6.04, p=.42\)).

The Shapiro-Wilk test for normality could not show the normality of residuals in any of the collected data. Therefore, data analysis is made using non-parametric two-tailed methods at a significance level of \(\alpha = 0.05\). Among these non-parametric tests, we have utilized the Mann-Whitney U test and the Wilcoxon signed-rank test. Mann-Whitney U test is a widely recognized non-parametric test, was selected for its robustness and appropriateness in situations where the data do not meet the assumptions of normality. This test is particularly well-suited for comparing two independent groups on a response variable. Similarly, the Wilcoxon signed-rank test, another widely recognized non-parametric test, was employed due to its performance in situations where normality assumptions are not met. It is well-suited for paired data analysis.

4.1  Cognitive Load

To address the first research question, whether the concept map recomposition activity of TSD approach affects the cognitive load, we have used the score of the questionnaire items for perceived effort and perceived difficulty. We also calculated the total time spent on the recomposition task for each learner in both conditions, and the number of actions needed to complete the recomposition task. We have analyzed the difference of these CLT measures between both conditions using non-parametric multivariate inference test [66], which controls for type I error. The method is implemented in the R package “npmv”. In this analysis, the condition was used as the predictor and the four cognitive load metrics were used as the response variable. The analysis result showed a significant difference in cognitive load with respect to the condition, \(F(2.21, 158.9) = 24.2, \mbox{P-Value} < .001\). Post-hoc analysis using Mann-Whitney U test revealed a significant difference for all of the response variables suggesting a lower cognitive load for the TSD group over the FD group in all four dimensions. The P-Values for this post-hoc analysis were adjusted using [67] method to control the false discovery rate and improve the power of the analysis. Table 1 presents the mean and standard deviation of cognitive load metrics for each condition, along with the corresponding sample sizes and statistical results.

Table 1  Mean and standard deviation M(SD) of cognitive load parameters and the statistical test result where N=37 for both conditions

To measure the effect size we used Relative Treatment Effect (RTE) that is generated within the same statistical test of “npmv”. RTE of treatment “k” is defined as the probability that a randomly chosen subject from treatment “k” displays a higher response than a subject that is randomly chosen from any of the treatment groups, including treatment “k” [66]. Table 2 shows the result of RTE for the cognitive load metrics between the conditions. The results show a very low probability of having a random sample from TSD condition experiencing a high cognitive load for all parameters. For example, the probability that a randomly chosen sample with a TSD condition experience a higher perceived difficulty from the whole sample is 0.26 where the general minimum/maximum possible effect is 0.23/0.77 for the TSD condition. The RTE suggests that the cognitive load metrics can discriminate one condition from the other almost perfectly using all response variables except for perceived effort which has a less distinctive probability compared to other parameters. The reason could be that both conditions engaged in information-processing resources in the recomposition activity to a high degree.

Table 2  Relative treatment effect (RTE) of the cognitive load metrics on concept map decomposition level using a non-parametric multivariate test

Next, we performed a correlation analysis between objective and subjective cognitive load measures for both the experimental (TSD) and control (FD) groups. Our purpose is to complement the subjective assessments with objective data, strengthening our comprehension of cognitive load in concept map recomposition task. This analysis supports a more profound understanding of the learner experience in this demanding context. The results of this analysis are presented in Table 3. In both groups, we observed significant positive correlations between “Time on Task” and both “Perceived Effort” and “Perceived Difficulty”. This shared pattern underscores the vital role of time learners spent on solving the task in influencing their cognitive load perceptions. It implies that as learners invest more time in the task, they tend to view it as both more effortful and more challenging. This dynamic relationship between task duration and cognitive load perceptions reveals that extended engagement is linked with heightened perceptions of effort and difficulty.

Table 3  Correlation Between Subjective and Objective Measures of Cognitive Load in Experimental and Control Groups

Similarly, the interplay between perceived effort, perceived difficulty, and the action map ratio provides valuable insights into our study’s cognitive load management. In both groups, we observed notable correlations that shed light on the connections between these variables. Notably, the “Action Map Ratio” reflects learners’ efficiency in interacting with the recomposition task. In the experimental group TSD, there was a positive correlation between “Action Map Ratio” and “Perceived Effort,” although it did not reach statistical significance. This suggests that in the TSD condition, where the task’s interaction is designed to be highly efficient, learners who engage more actively and performe a higher number of actions may not necessarily perceive the task as more effortful. This finding indicates that when the recomposition task is carefully structured to minimize learner effort, a higher level of interaction may not correspond to increased perceived effort. Conversely, in the control group, there was a significant positive correlation between “Action Map Ratio” and “Perceived Effort”. In this context, as learners in the control group engaged more actively and performed a higher number of actions on the concept map, they indeed reported the task as more effortful. This observation implies that, in FD condition, extensive engagement corresponds to increased perceptions of effort. Regarding “Perceived Difficulty”, significant correlations with “Action Map Ratio” were observed in both groups, underscoring the role of learner engagement. These correlations emphasize that higher levels of learner engagement and interaction on the concept map are associated with greater perceptions of task difficulty. In the TSD group, although the learners are relieved from making excessive actions, the intrinsic difficulty of the recomposition task persists. In contrast, the control group learners face additional difficulty due to the increased number of actions required during recomposition.

4.1.1  Reflection on the Cognitive Load Analysis

The analysis results could be explained mainly by the reduced problem space. Compared to FD approach, learners in TSD need to make fewer searches for each proposition which in turn reduces the overall needed time to recompose each proposition. Similarly, the minimum number of actions to recompose the concept map is reduced by half. let a node describe a concept or a link, per proposition, FD group needs to connect the first node to the second node, then connect the second node with the third node. While in TSD, the learner needs only to connect the second node which is the link to the target concept.

In addition to reducing the needed time and speeding up the recomposition process, the difficulty of the recomposition task and its effort are perceived to be lower compared to FD approach. In TSD, the learners are freed from the burden of repetitive tasks of heavy search/find that should be carried out normally when recomposing a concept map in FD approach. Additionally, the correlational analysis between the objective and subjective measures supports these analyzed measurements positively. It also gives a broader understanding for the design of educational tools and interventions aimed at optimizing cognitive load management. It underscores the importance of monitoring and regulating task duration as well as excessive learner actions to ensure their alignment with the intended learning outcomes. Overall, the TSD approach aids in the effective reduction of unnecessary cognitive load, especially in scenarios involving cognitively demanding tasks.

4.2  Optimal Motivational State

To address the second research question, whether the concept map recomposition activity of the TSD kit affects the flow state, we analyzed the self-reported flow state score. We confirmed the consistency of the questionnaire items using Cronbach’s Alpha \(\alpha=.75\). The mean flow score for the FD group was 4.74 (SD = 0.81), while the TSD group had a mean flow score of 4.91 (SD = 0.76) We have compared both conditions using Mann-Whitney U test by having flow state measurement as the response variable and condition as the predictor. According to the test result, there was no significant influence of condition on flow scores (\(U = 1300.5, n1 = n2 = 37, \mbox{P-Value} = .35\)).

4.2.1  Reflection on Optimal Motivational State Analysis

Usually, simplifying a task could lead to a state where learners lose interest in continuing to stay engaged. But, in TSD case we observe similar results to the original FD method; learners are still motivated in doing the recomposition activity. This is due to removing only the part of the activity that is repetitive and does not require a high skill whereas the core phase of the recomposition activity is still required in TSD.

This outcome offers two key perspectives on the activity. First, it reveals that making the recomposition task simpler through the TSD method does not make it less interesting for learners. That means, the learners of both conditions perceived similar challenges as well as the confidence to respond to the recomposition challenge. Second, it shows that the subjective self-reports of cognitive load metrics are strongly comparable and the differences between the two conditions’ self-reports are not due to their flow-state.

4.3  Reading Comprehension

To answer the third and final research question, whether the concept map recomposition activity of the TSD kit affects the a) immediate reading comprehension. b) retention of reading comprehension, we analyze the comprehension scores of after-reading comprehension test (pre-test) score, after-recomposition comprehension test (post-test) score, and delayed-test score.

Of the total of 74 participants, 54 participated in the optional delayed comprehension test. As mentioned previously, the participants were offered a reward for their participation in the experiment. It’s important to note that the amount of reward money provided may have influenced the level of enrollment, potentially contributing to the reasons why some participants chose not to participate in the delayed test conducted two weeks later.

Table 4 shows the mean and standard deviation of the comprehension scores, while Table 5 shows the mean and standard deviation for participants who also took the optional delayed comprehension test.

Table 4  Mean and standard deviation M (SD) of after-reading comprehension test (pre-test) score, and after-recomposition comprehension test (post-test) score along with the statistical test result

Table 5  Mean and standard deviation M (SD) of after-reading comprehension test (Pre-test) score, after-recomposition comprehension test (Post-test) score, and delayed comprehension test (Delay-test) score along with the statistical test result

4.3.1  Reading Comprehension within Each Group

We examined the reading comprehension within each condition. The Wilcoxon signed-rank test with continuity correction revealed a significant increase in reading comprehension in conditions TSD (\(Z = -3.75, \mbox{P-Value} < .001, r = .597\)) and FD (\(Z = -2.45, \mbox{P-Value} = .01, r = .419\)) when pre-test score is compared with post-test score within each condition. The result confirms that both kinds of concept map recomposition activities are useful in immediate reading comprehension. The results for FD condition align with previous studies where the benefit of Kit-Build is investigated. Above that, the results for the TSD condition confirm that the benefits of traditional Kit-Build on reading comprehension are preserved with the new approach. Conducting the same statistical test to compare the delayed-test score with the pre-test score within each condition showed that both conditions retention lowered to slightly below the period of after-reading phase (Condition\(_{\mathit{TSD}}\): \(Z = 1.29, \mbox{P-Value} = .195, r = .37\); Condition\(_{\mathit{FD}}\): \(Z = 1.95, \mbox{P-Value} = .051, r = .22\)). The statistical result does not show a significant difference of the two test scores in each condition. However, the results of FD condition are marginally significant affirming the possibility of a high rate of forgetting in the traditional kit generation method. Despite that, it is natural for the gained knowledge to dissipate over time which is happening in the delayed-test for both conditions. Although the average delayed test score is lower than the pre-test score, there could still be some learning gain relative to their before-reading stage since the pre-test score is measured after reading the text, meaning that their previous knowledge could be much lower than the pre-test score. Thus, the concept map recomposition activity may have helped both groups in retaining their basic comprehension.

4.3.2  Reading Comprehension between the Conditions

Our next analysis focused on assessing the potential variations in reading comprehension between the conditions. To confirm the homogeneity of both conditions, we compared their pre-test scores using Fligner-Killeen test of homogeneity of variances (F-K). F-K test is said to have robustness toward departures from normality. The test confirmed the homogeneity of the conditions (\(X^2 = 0.74, \mbox{P-Value} = .39\)). To analyze the immediate reading comprehension, we applied Mann-Whitney U test comparing the post-test score of both conditions. The median post-test score in both conditions were 9; the distributions in the two conditions were not significantly different (\(U = 1325, n1 = n2 = 37, \mbox{P-Value} = .48\)). There is a possibility of having a ceiling effect since the pre-test, as well as the post-test scores, are already too high narrowing the room to show differences in improvement. Despite that, the mean for the TSD condition tends to go higher than the FD group which could give us a hunch to expect the TSD condition to improve better.

Before testing for the retention of reading comprehension we decided to re-operate the F-K test to confirm the homogeneity of both conditions since some of the participants from both conditions did not take part in this delayed comprehension test. F-K test confirmed the homogeneity of the conditions (\(X^2 = 1.001, \mbox{P-Value} = .32\)). The median delayed-test score in the TSD condition and FD condition was 7.5 and 7.0 respectively. The Mann-Whitney U test comparing the delayed-test score of both conditions did not reveal any significant different between the conditions (\(U = 715, n1 = 32, n2 = 27, \mbox{P-Value} = .14\)).

4.3.3  Reflection on Reading Comprehension Analysis

The reading comprehension analysis implied that both groups significantly gain knowledge after the recomposition activity while the magnitude of this knowledge disperses in the period of two weeks. Although there is a possibility of a ceiling effect during post-test, comparing both groups’ knowledge, revealed that the quality of learning is similar between the two conditions. Thus, providing a partially recomposed kit did make the activity easier but did not lower the comprehension. What is more important is that, although we observe a similar learning effect, we should acknowledge that the TSD group spent significantly less time on the recomposition activity to accomplish this level of learning. It is expected that if the remaining time was utilized to perform another form of learning, the TSD approach learners would become superior in immediate as well as retention of reading comprehension.

These results can give a significant advantage to educators when designing the concept mapping activities. The proposed approach gives a chance to use a mixture of activities in less time thus encouraging the learners to get engaged better than having the time all allocated for one task.

5.  Discussion

The results of this study show that the high cognitive effort of traditional Kit-Build concept map recomposition (FD) can be minimized without losing the advantages of learning and motivation. Target Search Decomposition is proposed as one approach to minimize the unnecessary cognitive load and save a lot of time compared to the FD method. The core of the TSD approach is to disconnect the target concept of every proposition in the expert concept map when preparing for the recomposition task.

The cognitive load measurement included four dimensions, two of them subjective: perceived difficulty and perceived effort; another two metrics were objective measures: time spent on the recomposition task and number of actions needed during the recomposition task. The flow metric is used to support the subjective metrics and measure the motivation of the learners toward the proposed recomposition task. Finally, the reading comprehension test was used to evaluate the learners for their reading comprehension and their retention after two weeks.

Aligning with previous study results [9], the traditional method showed a high degree of cognitive load and good task motivation. Similarly removing some of the tasks in FD as in the previous study resulted in lower cognitive load. However, the task removed in this study was related to the act of recomposition directly not to the user interface as in the previous study, thus, it dramatically reduced the time, actions, effort, and difficulty to accomplish nearly the same outcome. This reduction of cognitive load measures is naturally expected because it is due to the reduction in the problem space. However, the proposed approach has successfully maintained the advantage of FD in terms of task motivation, immediate learning, and knowledge retention. Although, searching for information in the text is believed to be connected to a higher retention rate [15], this study gives educators a method to manage this search-and-find task effectively such that the invested effort gives a high learning return in much a shorter time and less cognitive effort.

As mentioned above, the proposed TSD approach removes tasks that are needed during the recomposition itself. Naturally, we expect learners who invest more effort in the learning task to gain higher knowledge [68]. The reason why the TSD approach does not suffer from losing the quality of learning can be explained as follows: While TSD eliminates the need to search for the source concept and an appropriate link for a proposition, it does not negate the necessity of investing effort in reviewing the content of the source concept and link. This is because the process of identifying the correct target concept to complete the knowledge necessitates a thorough review and comprehension of the first half of the proposition. That can explain why the difference between conditions in the probability of experiencing a low cognitive effort was less distinctive relative to other cognitive parameters according to the RTE result. We contend that it is this review process that sustains the effectiveness of learning in the TSD approach, aligning with the perspective presented by [15], which underscores the significance of reviewing as one of the central tasks during the recomposition activity.

The embodied difficulty of the Kit-Build recomposition task may not contribute to the learning outcome directly but it could improve the flow and keep learners motivated to continue the activity by creating more challenges for the learners to take. Thus, removing such cognitive load may cause a reduction in the motivation for Kit-Build activity leading to difficulty in implementing the proposed method in an educational environment. However, in this case study, the flow-state results confirmed that cutting the cognitive load via TSD approach does not remove the necessary cognitive load for learners to remain engaged. It is worth investigating how the components of TSD approach in the recompositional activity maintain the flow within the learners.

There can be concern surrounding the provision of source concepts and links to learners in advance in TSD. It is similar to the case of recomposition-based concept mapping and traditional concept mapping. However, this practice aligns with the concept of empathetic understanding, underlining the importance of replicating expert or partner maps as a valuable learning process. Therefore, aiding learners in this endeavor without compromising the educational benefits of the recomposition task in Kit-Build enhances its effectiveness in educational settings. This assistance not only conserves learners’ time and energy but also opens up opportunities for engaging in additional learning activities where the focus shifts to building creative knowledge structures.

The results of this study are specific to Kit-Build type recomposition. Nonetheless, the nature of the recomposition task in general could contain similar tasks that increase the cognitive load on learners. Thus, the findings of this research have the potential to be generalized on different kinds of concept map recomposition with demands for further research.

5.1  Limitations and Potential Future Studies

Target Search Decomposition of the expert concept map in Kit-Build, is likely to generate bias since part of the proposition will remain attached. This attachment gives TSD learners an upper-hand in the recomposition task.

In addition, the provision of source concepts and links may affect the meta-cognition skills of the learners. To confirm the effect of this customization during the comprehension task, we are considering further research to monitor and measure the meta-cognitive aspects of TSD compared to the traditional FD approach as well as to the open-end concept map.

Another important future research about TSD approach is to confirm the hypothesis we put in the discussion as the explanation to why TSD method maintains the same learning quality as FD method but in quite a shorter time, much less cognitive effort, difficulty, and actions. Besides, in this study, we implemented a uniform partial decomposition across all of the propositions in the expert map. However, there can be an ideal threshold for defining the level of partial decomposition. In this sense, another potential future research is to consider different levels of partial decomposition on the cognitive load and the quality of reading comprehension.

6.  Conclusion

The role of concept maps in reading comprehension is very crucial, and study toward improving such activity is necessary in many aspects of education. In this study, we have introduced a new approach to be used during concept map recomposition called Target Search Decomposition. The experimental results revealed the superiority of this approach over the traditional Full Decomposition approach in four different dimensions of cognitive load. The 4 dimensions include reduction in: perceived difficulty, perceived effort, the time needed to complete the recomposition task, and the number of actions needed to successfully recompose a proposition. Besides the cognitive load parameters, motivational aspects of the concept map recomposition were maintained since the learners reported a similar (slightly higher) state of flow during the recomposition activity in contrast to the full decomposition method. Moreover, the new approach preserves the learning quality of the concept map recomposition giving educators a remarkable advantage in setting such activities without exhausting learners’ cognitive load and motivation. The proposed TSD approach can be considered as the best customization to the traditional Kit-Build concept map recomposition investigated until now. Thus, we can recommend the use of the TSD approach in concept map recomposition activities since it reduces the strain on learners, reduces the needed time to complete the activity, hence, giving a better chance to educators to organize their materials so that the saved time and mental energy can be utilized toward other learning activities such as reviewing the material, pair discussions, or supplementary exercises.

References

[1] C. Snow, Reading for Understanding: Toward an R&D Program in Reading Comprehension, RAND Corporation, Santa Monica, CA, 2002.

[2] J.R. Kirby, “Reading comprehension: Its nature and development,” Encyclopedia of language and literacy development, pp.1-8, 2007.

[3] J.D. Novak, “Results and implications of a 12-year longitudinal study of science concept learning,” Research in Science Education, vol.35, no.1, pp.23-40, 2005.

[4] R. Dias, “Concept maps powered by computer software: a strategy for enhancing reading comprehension in english for specific purposes,” Revista Brasileira de Linguística Aplicada, vol.11, pp.896-911, 2010.

[5] M. Kalhor and G. Shakibaei, “Teaching reading comprehension through concept map,” Life Science Journal, vol.9, no.4, pp.725-731, 2012.

[6] E.M. Jackson and M.F. Hanline, “Using a concept map with recall to increase the comprehension of science texts for children with autism,” Focus on Autism and Other Developmental Disabilities, vol.35, no.2, pp.90-100, 2020.

[7] J.D. Novak and A.J. Cañas, “The theory underlying concept maps and how to construct them,” Florida Institute for Human and Machine Cognition, vol.1, no.1, pp.1-31, 2006.

[8] H.E. Herl, H.F. O’Neil Jr, G.K.W.K. Chung, and J. Schacter, “Reliability and validity of a computer-based knowledge mapping system to measure content understanding,” Computers in Human Behavior, vol.15, no.3-4, pp.315-333, 1999.

[9] P.G.F. Furtado, T. Hirashima, and Y. Hayashi, “Reducing cognitive load during closed concept map construction and consequences on reading comprehension and retention,” IEEE Transactions on Learning Technologies, vol.12, pp.402-412, 2018.

[10] T. Hirashima, K. Yamasaki, H. Fukuda, and H. Funaoi, “Framework of kit-build concept map for automatic diagnosis and its preliminary use,” Research and Practice in Technology Enhanced Learning, vol.10, no.1, p.17, 2015.

[11] T. Hirashima, “Reconstructional concept map: Automatic assessment and reciprocal reconstruction,” International Journal of Innovation, Creativity and Change, vol.5, pp.669-682, 2019.

[12] B.S. Andoko, Y. Hayashi, T. Hirashima, and A.N. Asri, “Improving english reading for efl readers with reviewing kit-build concept map,” Research and Practice in Technology Enhanced Learning, vol.15, no.1, p.7, 2020.

[13] M. Alkhateeb, Y. Hayashi, T. Rajab, and T. Hirashima, “Comparison between kit-build and scratch-build concept mapping methods in supporting efl reading comprehension,” The Journal of Information and Systems in Education, vol.14, no.1, pp.13-27, 2015.

[14] L. Sadita, T. Hirashima, Y. Hayashi, W. Wunnasri, J. Pailai, K. Junus, and H.B. Santoso, “Collaborative concept mapping with reciprocal kit-build: a practical use in linear algebra course,” Research and Practice in Technology Enhanced Learning, vol.15, no.1, 2020.

[15] P.G.F. Furtado, T. Hirashima, N. Khudhur, A. Pinandito, and Y. Hayashi, “Influence of access to reading material during concept map recomposition in reading comprehension and retention,” IEICE Transactions on Information and Systems, vol.E104-D, no.11, pp.1941-1950, 2021.

[16] J. Sweller, J.J.G. van Merrienboer, and F.G.W.C. Paas, “Cognitive architecture and instructional design,” Educational Psychology Review, vol.10, no.3, pp.251-296, 1998.

[17] W. Schnotz, S. Fries, and H. Horz, “Motivational aspects of cognitive load theory,” Contemporary motivation research: From global to local perspectives, pp.69-96, 2009.

[18] A.E. Widjaja and J.V. Chen, “Online learners’ motivation in online learning: the effect of online-participation, social presence, and collaboration,” Learning technologies in education: Issues and trends, vol.12, pp.72-93, 2017.

[19] Y.M. Tang, K.Y. Chau, Y.-Y. Lau, and G.T.S. Ho, “Impact of mobile learning in engineering mathematics under 4-year undergraduate curriculum,” Asia Pacific Journal of Education, pp.1-17, 2022.

[20] H. Tohidi and M.M. Jabbari, “The effects of motivation in education,” Procedia - Social and Behavioral Sciences, vol.31, pp.820-824, 2012.

[21] S. Engeser and F. Rheinberg, “Flow, performance and moderators of challenge-skill balance,” Motivation and Emotion, vol.32, no.3, pp.158-172, 2008.

[22] G.J. Hwang, F.R. Kuo, N.S. Chen, and H.J. Ho, “Effects of an integrated concept mapping and web-based problem-solving approach on students’ learning achievements, perceptions and cognitive loads,” Computers & Education, vol.71, pp.77-86, 2014.

[23] S.K. Kamble and B.L. Tembe, “The effect of use of concept maps on problem solving performance and attitude in mechanical engineering course,” Procedia - Social and Behavioral Sciences, vol.83, pp.748-754, 2013.

[24] M. Karadag, “The effect of concept map based education on the problem solving skills of students,” New Trends and Issues Proceedings on Humanities and Social Sciences, vol.3, no.1, pp.506-513, 2017.

[25] C.-C. Chiou, “The effect of concept mapping on students’ learning achievements and interests,” Innovations in Education and Teaching International, vol.45, no.4, pp.375-387, 2008.

[26] D. Hay, I. Kinchin, and S. Lygo-Baker, “Making learning visible: the role of concept mapping in higher education,” Studies in Higher Education, vol.33, no.3, pp.295-311, 2008.

[27] E.C. McCagg and D.F. Dansereau, “A convergent paradigm for examining knowledge mapping as a learning strategy.,” The Journal of Educational Research, vol.84, no.6, pp.317-324, 1991.

[28] A. Pinandito, D.D. Prasetya, Y. Hayashi, and T. Hirashima, “Design and development of semi-automatic concept map authoring support tool,” Research and Practice in Technology Enhanced Learning, vol.16, no.1, p.8, 2021.

[29] C.T. Machado and A.A. Carvalho, “Concept mapping: Benefits and challenges in higher education,” The Journal of Continuing Higher Education, vol.68, no.1, pp.38-53, 2020.

[30] K. Oliver, “A comparison of web-based concept mapping tasks for alternative assessment in distance teacher education,” Journal of Computing in Teacher Education, vol.24, no.3, pp.95-103, 2008.

[31] P.-H. Wu, G.-J. Hwang, M. Milrad, H.-R. Ke, and Y.-M. Huang, “An innovative concept map approach for improving students’ learning performance with an instant feedback mechanism,” British Journal of Educational Technology, vol.43, no.2, pp.217-232, 2012.

[32] R. Rismanto, A. Pinandito, B. Andoko, Y. Hayashi, and T. Hirashima, “Evaluating the kit-build concept mapping process using sub-map scoring,” Research and Practice in Technology Enhanced Learning, vol.19, p.021, 2023.

[33] W. Wunnasri, J. Pailai, Y. Hayashi, and T. Hirashima, “Reciprocal kit-build concept map: an approach for encouraging pair discussion to share each other’s understanding,” IEICE Transactions on Information and Systems, vol.101, no.9, pp.2356-2367, 2018.

[34] L. Sadita, P.G.F. Furtado, T. Hirashima, and Y. Hayashi, “Analysis of the similarity of individual knowledge and the comprehension of partner’s representation during collaborative concept mapping with reciprocal kit build approach,” IEICE Transactions on Information and Systems, vol.103, no.7, pp.1722-1731, 2020.

[35] L. Sadita, T. Hirashima, Y. Hayashi, P.G.F. Furtado, K. Junus, and H.B. Santoso, “The effect of differences in group composition on knowledge transfer, group achievement, and learners’ affective responses during reciprocal concept mapping with the kit-build approach,” Research and Practice in Technology Enhanced Learning, vol.15, no.1, pp.1-19, 2020.

[36] T. Hirashima, “Reconstructional concept map: automatic assessment and reciprocal reconstruction,” International Journal of Innovation, Creativity and Change, vol.5, pp.669-682, 2019.

[37] A. Pinandito, Y. Hayashi, and T. Hirashima, “Online collaborative kit-build concept map: Learning effect and conversation analysis in collaborative learning of english as a foreign language reading comprehension,” IEICE Transactions on Information and Systems, vol.E104-D, no.7, pp.981-991, 2021.

[38] Nurmaya, A. Pinandito, Y. Hayashi, and T. Hirashima, “Promoting students’ higher order thinking with concept map recomposition,” IEICE Transactions on Information and Systems, vol.E106-D, no.8, pp.1262-1274, 2023.

[39] A. Pinandito, C.P. Wulandari, D.D. Prasetya, N. Khudhur, Y. Hayashi, and T. Hirashima, “Efficient Online Collaborative Learning Through Concept Mapping with Kit-Build Concept Map,” pp.125-131, Association for Computing Machinery, New York, NY, USA, 2021.

[40] A. Artino, “Cognitive load theory and the role of learner experience: An abbreviated review for educational practitioners,” AACE Journal, vol.16, pp.425-439, 2008.

[41] R.E. Mayer, The Cambridge Handbook of Multimedia Learning, 2 ed., Cambridge Handbooks in Psychology, Cambridge University Press, 2014.

[42] J. Sweller, J.J.G. van Merriënboer, and F. Paas, “Cognitive architecture and instructional design: 20 years later,” Educational Psychology Review, vol.31, no.2, pp.261-292, 2019.

[43] J. Sweller, “Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load,” Educational Psychology Review, vol.22, no.2, pp.123-138, 2010.

[44] J. Sweller, “Cognitive load theory,” 2011.

[45] S.S. Tseng, P.C. Sue, J.M. Su, J.F. Weng, and W.N. Tsai, “A new approach for constructing the concept map,” Computers & Education, vol.49, no.3, pp.691-707, 2007.

[46] X. Huang, Q. Liu, C. Wang, H. Han, J. Ma, E. Chen, Y. Su, and S. Wang, “Constructing educational concept maps with multiple relationships from multi-source data,” 2019 IEEE International Conference on Data Mining (ICDM), pp.1108-1113, 2019.

[47] H.W. Lee, K.Y. Lim, and B.L. Grabowski, “Generative learning: Principles and implications for making meaning,” in Handbook of research on educational communications and technology, pp.111-124, Citeseer, 2008.

[48] M.K. Wilhelm-Chapin and T.A. Koszalka, “Generative learning theory and its application to learning resources (concept paper).” https://ridlr.syr.edu/concept-paper, 2016.

[49] J. Pailai, W. Wunnasri, K. Yoshida, Y. Hayashi, and T. Hirashima, “The practical use of kit-build concept map on formative assessment,” Research and Practice in Technology Enhanced Learning, vol.12, no.1, 2017.

[50] M. Csikszentmihalyi, Finding Flow: The Psychology of Engagement with Everyday Life., Basic Books, New York, 1997.

[51] M. Biasutti, “Flow and optimal experience,” Reference Module in Neuroscience and Biobehavioral Psychology, Elsevier, 2017.

[52] M. Csikszentmihalyi, Flow: the Psychology of Optimal Experience by Mihaly Csikszentmihalyi, CreateSpace Independent Publishing Platform, 2018.

[53] G. Paolacci, J. Chandler, and P.G. Ipeirotis, “Running experiments on amazon mechanical turk,” Judgment and Decision Making, vol.5, no.5, pp.411-419, 2010.

[54] M. Buhrmester, T. Kwang, and S.D. Gosling, “Amazon’s mechanical turk: A new source of inexpensive, yet high-quality, data?,” Perspectives on Psychological Science, vol.6, no.1, pp.3-5, 2011.

[55] D.J. Hauser and N. Schwarz, “Attentive turkers: Mturk participants perform better on online attention checks than do subject pool participants,” Behavior Research Methods, vol.48, no.1, pp.400-407, 2016.

[56] L. Litman, J. Robinson, and T. Abberbock, “Turkprime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences,” Behavior Research Methods, vol.49, no.2, pp.433-442, 2017.

[57] J. Chandler, C. Rosenzweig, A.J. Moss, J. Robinson, and L. Litman, “Online panels in social science research: Expanding sampling methods beyond mechanical turk,” Behavior Research Methods, vol.51, no.5, pp.2022-2038, 2019.

[58] F.G.W.C. Paas and J.J.G. Van Merriënboer, “Instructional control of cognitive load in the training of complex cognitive tasks,” Educational Psychology Review, vol.6, no.4, pp.351-371, 1994.

[59] S. Kalyuga, P. Chandler, and J. Sweller, “Managing split-attention and redundancy in multimedia instruction,” Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, vol.13, no.4, pp.351-371, 1999.

[60] K.E. DeLeeuw and R.E. Mayer, “A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load,” Journal of Educational Psychology, vol.100, no.1, pp.223-234, 2008.

[61] A.E. Gimino, Factors that influence students’ investment of mental effort in academic tasks: A validation and exploratory study, University of Southern California, 2000.

[62] F. Paas, J.E. Tuovinen, H. Tabbers, and P.W. Van Gerven, “Cognitive load measurement as a means to advance cognitive load theory,” Educational Psychologist, vol.38, no.1, pp.63-71, 2003.

[63] R. Brünken, J.L. Plass, and D. Leutner, “Direct measurement of cognitive load in multimedia learning,” Educational Psychologist, vol.38, no.1, pp.53-61, 2003.

[64] J. Schüler, “Arousal of flow experience in a learning setting and its effects on exam performance and affect,” Zeitschrift Fur Padagogische Psychologie, vol.21, no.3/4, pp.217-227, 2007.

[65] F. Rheinberg, Y. Manig, R. Kliegl, S. Engeser, and R. Vollmeyer, “Flow bei der Arbeit, doch Glück in der Freizeit,” Zeitschrift für Arbeits- und Organisationspsychologie A&O, vol.51, no.3, pp.105-115, 2007.

[66] W.W. Burchett, A.R. Ellis, S.W. Harrar, and A.C. Bathke, “Nonparametric inference for multivariate data: The r package npmv,” Journal of Statistical Software, vol.76, no.4, 2017.

[67] Y. Benjamini and Y. Hochberg, “Controlling the false discovery rate: a practical and powerful approach to multiple testing,” Journal of the Royal statistical society: series B (Methodological), vol.57, no.1, pp.289-300, 1995.

[68] J.S. Cole, D.A. Bergin, and T.A. Whittaker, “Predicting student achievement for low stakes tests with effort and task value,” Contemporary Educational Psychology, vol.33, no.4, pp.609-624, 2008.

Footnotes

1.  https://git.io/JyeDr

Authors

Nawras KHUDHUR
  Hiroshima University

Aryo PINANDITO
  Universitas Brawijaya

Yusuke HAYASHI
  Hiroshima University

Tsukasa HIRASHIMA
  Hiroshima University

Keyword

concept map,  recomposition,  reading comprehension,  cognitive load,  flow state,  partial solution