Web based educational hypertext systems provide features which go beyond conventional learning material. There is a high amount of directly available information. The learners are more independent in terms of time and space restrictions. Moreover, the information can be adapted to the learner's needs and be easily updated [1]. In the field of foreign language learning new technologies are used more and more in addition to and in contrast to traditional classroom settings. For grammar learning in particular there are some web based tools available that refer to a wide range of real examples and that provide specific feedback to the learners. This is the case with the Chemnitz InternetGrammar [6] which combines the demonstration of grammatical rules, illustrated with real examples from a variety of domains, and exercises.
The specificity of learners' processing goal is an essential factor in learning with educational hypertext [2, 9]. There is evidence that in some cases the processing goal dominates over hypertext design [10, 5, but see 8]: Wiley [10] demonstrated that in learning historic events, the combination of processing goal and design determine learning. In Foltz's [5] study, the processing goal determined learners' acquired knowledge and navigation. In contrast, Shapiro [8] found that prior knowledge and structure of a given overview overshadows the effect of learning goals on performance when learning biology. Nevertheless, in [7] it is proposed that different hypertexts are appropriate for different processing goals.
For most studies [e.g. 3, 4, 5] the processing goal is subordinate to hypertext design. Additionally, learning is usually subordinate to navigation within hypertext systems. Nevertheless, there is a reported trade-off between better navigation and understanding of details shown by learners searching for specific information in the text [5, 3] and steady navigation and higher comprehension of general knowledge shown by learners reading the text without a specific task [4].
This study examined the effects of processing tasks on the acquisition of knowledge. Goal specificity was implemented by a specific processing task of searching for details within a hypertext and by a unspecific reading task. To measure comprehension of the hypertext, participants were asked to answer questions about the content. The design of the learning tool was constant for all conditions. According to Dee-Lucas' findings [3], it would be expected that searchers would answer fewer questions than readers because of the readers' greater general knowledge. In contrast, searchers would answer questions in more detail. Additionally, only searchers should show varying levels of performance depending on the type of tests and the test order.
Forty students of the Chemnitz University of Technology (age: M = 23.1; SD = 5.7; 80% female) took part in this study for course credit. All of them were native German speakers. Their English performance was at intermediate level, so that they could understand the chapter written in English and learn from its content. All students were familiar with using computers and Internet but had no programming experience.
In order to investigate learning with educational hypertext the Present Perfect chapter of the Chemnitz InternetGrammar [6] was adapted to the experimental setting. Learning the Present Perfect is a challenging task for German students, since the linguistic rules for its usage are somewhat different in German and English.
The hypertext used was created with Hypercard 2.4.1 and contained 75 cards of rules, examples and exercises. The headings of the topics within the chapter were presented as a navigation aid on the left side of each card. Tabs above the text linked a card to the corresponding rules, examples or exercise cards. Additionally, two buttons were presented below the text leading to the previous and next cards, respectively. The study was conducted with iMacs (64 MB RAM).
Figure 1. A rules' card out of the Present perfect chapter with navigation bar on the left side, tabs above and buttons below.
Figure 2. An exercise card out of the Present perfect chapter with navigation bar on the left side, tabs above and buttons below. Pressing the "Check"-button after selecting a verb form within the exercises results in a "right" or "wrong" feedback message.
The global learning goal of all participants was to understand the central concepts described in the hypertext and to apply them afterwards. Additionally, half of the learners processed the hypertext searching for specific details and the other half processed the hypertext with an unspecific reading task. Learners of both groups were comparable in respect to prior knowledge and processed material. They differed in processing task and time of testing.
The comprehension of the content was measured by a multiple choice test (MC) with questions about single details and by a open-ended questionnaire asking about several concepts of the hypertext. Both tests contained twelve items each and were presented sequentially as paper and pencil tests. An example of a MC item is the identification of one situation out of three given situations where the present perfect cannot be used. In contrast, in the open-ended questions learners had to enumerate all mentioned situations were the present perfect cannot be used.
The order of MC test and open-ended questions was balanced between participants, so that half of the learners answered multiple choice items first (MC first condition) and the others answered open-ended questions first (open-ended question first condition). In the following analysis, first and second performed test (MC and open-ended questions pooled) will be compared and for MC and open-ended questions separately first and second performed test will be compared. All questions dealt with the main ideas of the chapter, and required neither inferences about the content nor connections between details belonging to different cards. Nevertheless, questions could not be answered by copying and pasting because they dealt mostly with a summary of a card's content instead of asking for specific phrases within the text. The question order within a test was suitable for guiding searchers on an optimal path through the chapter and to focus their attention on the most central details within the hypertext.
Before processing the chapter participants performed a prior knowledge test and before and after processing an application test. All learners worked on the chapter for thirty minutes and navigated freely within the hypertext. The searching group answered MC and open-ended questions while reading. The reading group processed the hypertext without a specific task and answered the same questions afterwards. The study was conducted in group-sessions of up to six subjects at time. A session lasted about one hour.
Searchers answered approximately 6 out of 12 MC items and 8 out of 12 open-ended questions. In contrast, readers answered an average of 12 out of 12 MC and 10 out of 12 open-ended questions (F(1, 72) = 39.54, p = 0.0001).
The number of answers was different for first and second performed test for searchers (10, 4), but not for readers (11, 11) (F(1, 72) = 17.95, p = 0.0001). Newman-Keuls test revealed significant differences for all four combinations of the test positions (MC first and second, open-ended questions first and second) for searchers (all p = 0.01), but not for readers.
The largest difference was found for MC questions. Searchers in the MC first condition gave approximately 10 answers. In contrast, searchers in the open-ended question first condition gave approximately 3 answers on MC items.
The number of answers by searchers and readers did not differ for the first performed test. For the second test, searchers answered fewer MC and open-ended questions (3, 6), than readers (11, 12).
Searchers performed comparably well for both test conditions in the first test. In the second performed test searchers answered significantly fewer questions than in the first test, especially fewer MC items. This pattern can be explained by the fact that answering open-ended questions first consumes more time than answering MC items first. Therefore, less time remains for answering MC items afterwards. This stands to reason, if searchers answer open-ended questions in more detail than readers do.
In contrast, readers performed both test conditions at a relatively high level in respect to the number of answers. There was no significant difference between the number of answers for MC and open-ended questions nor for both test conditions.
Figure 3. Number of answers for MC and open-ended questions given by searchers and readers. Light columns symbolize MC items and dark columns symbolize open-ended questions. The number of answers is shown by pairs for first and second performed tests.
Moreover, there was a difference in the pattern of missing data. Searchers tended to answer all given questions starting with the first question and finishing with the last question. So missing data occur clustered at the end of a given test. In contrast, for readers there was no accumulation of unanswered questions at the end of the tests. Instead, readers tended to skip highly complex questions. So missing data occurred mostly in isolated open-ended questions requiring more details.
In contrast to the varying number of answered questions, both groups, searchers and readers, attained an average overall score of 5 points out of 12 with regard to the correctness and completeness of their answers.
For the first performed test, searchers attained higher scores than readers for MC (9 vs. 7, p = 0.03), but not for open-ended questions (5 vs. 3, p = 0.23). For the second performed test, the effects were reverse: Searchers attained lower scores than readers for MC (2 vs. 7, p = 0.001), but not for open-ended questions (4 vs. 3, p = 0.71).
Searchers' performance decreased from the first performed test to the second test in MC items (9, 2, p = 0.001) but not in open-ended questions (5, 4, p = 0.95). For the first test, searchers attained higher scores for MC (9) than for open-ended questions (5) (p = 0.001). In contrast, there was no difference between MC (2) and open-ended questions (4) for the second performed test (p = 0.12) for searchers. These results are heavily interpretable because of the lower number of answered questions for searchers in the second performed test.
Readers achieved 7 points out of 12 for MC items and 3 points out of 12 for open-ended questions for both the first and second performed tests. They showed no difference in performance over time (p = 0.90) and they attained a higher score for MC items than for open-ended questions (p = 0.001). That pattern is compatible with the general-but-not-detailed knowledge predicted for this group.
Searchers' performance varied from test to test in contrast to the steady behavior of readers. For the MC items searchers performed better in the first test than in the second test. For open-ended questions there was almost no difference between first and second performed test for searchers and between both groups.
Table 1. Scores for MC and open-ended questions for searchers and readers, separately for first and second performed tests. There were maximally 12 points attainable.
| Learners | Questions | First performed test | Second performed test | Mean for both tests |
|---|---|---|---|---|
| Searchers | MC | 8.8 | 1.6 | 5.2 |
| open-ended | 4.7 | 5.3 | 4.1 | |
| Readers | MC | 6.7 | 6.8 | 6.7 |
| open-ended | 3.1 | 3.2 | 3.1 | |
| total | 5.8 | 3.8 | 4.8 |
The number of answers and, in part, the attained scores indicated that readers answered more questions resulting in slightly higher scores than searchers. In order to evaluate the grade given for the answers, it is necessary to compare the scores relative to the number of answered questions. For the following analysis scores for each learner and test was divided by the corresponding number of answers and was multiplied by 12 in order to get comparable grades for correct answers on MC items and complete answers on open-ended questions. Doing so the picture reverses:
Searchers answered open-ended questions in more detail and more MC items correctly than readers (7 vs. 5, out of 12) (F(1, 65) = 14.67, p = 0.001). Neither groups' performance changed from first to second performed test for MC and open-ended questions (for searchers: p = 0.87; for readers: p = 0.97).
Searchers answered MC items and open-ended questions equally well (7, 7). In contrast, the readers' relative grade of answers was higher for MC items than for open-ended questions (7, 4) for all the test orders.
There was no difference between searchers' and readers' performance for MC items (8, 7). However, searchers answered open-ended questions in more detail than readers (6, 4).
Generally speaking, results show that searchers answered questions better than readers. This effect is especially obvious for open-ended questions requiring enumeration of several details. This corresponds with the hypothesis that searchers with specific processing task demonstrate a more detailed comprehension of the hypertext. In contrast, performance of readers' was lower than the performance of searchers' especially for open-ended questions. This corresponds with the hypothesis that readers with unspecific processing task acquire more general but not detailed knowledge.
Figure 4. Grade of answers for MC and open-ended questions given by searchers and readers measured by overall scores relative to the number of answers. Light columns symbolize MC items and dark columns symbolize open-ended questions. The scores are shown in pairs for first and second performed tests.
The reported trade-off between specific and unspecific processing tasks for learning with educational hypertext was also present in this study. Readers answered more questions than searchers, demonstrating a better comprehension of the general knowledge. In contrast, searchers answered questions in more detail, illustrated by higher relative scores, especially for open-ended questions. Therefore, searchers demonstrated more detailed comprehension than readers.
Moreover, searchers differed from readers especially in the second performed test: They demonstrated higher comprehension, but answered fewer questions. That implies that only searchers profit from processing the chapter in later tests requiring detailed answers. At the same time they run into trouble collecting extensively details within a limited period of time.
The results have two implications. First, the reported effect of goal specificity was verifiable for the first time without simultaneously manipulating design of the used hypertext. Therefore, it could be more efficient to adapt the guidance through the hypertext system provided by processing task to the overall learning goal instead of designing different hypertexts for each condition. In fact, processing tasks can guide the learner through the hypertext acting as a navigation aid. The order of questions within a test given to searchers eases decisions where to go next and creates relations between text nodes. Given a meaningful question order, learner can navigate freely and nevertheless be guided through the learning environment.
For adapting processing tasks to learning goals, there are two distinctive guidelines: In order to get a broader overview it is suitable to present an unspecific reading instruction with facultative additional information for highly complex details. In contrast, for finding specific information the proper instruction and presented material should be specific and frugal.
Second, the difference between learners with different processing tasks is usually explained by their familiarity with the learning system and the improved navigation shown by those learners with specific instruction. However, the difference could also be caused by the varying knowledge organization between the groups. If readers with unspecific processing goals acquire a high level of general knowledge without a deeper understanding of the main details, they will answer all factual questions at a relatively high level saving processing time for answering all questions. Conversely, if searchers with unspecific processing tasks acquire a high amount of detailed and disintegrated knowledge, they will reproduce more details than readers especially for complex questions. That holds as long as there is no need for inferences and knowledge transfer. Answering questions in detail takes time. So searchers could produce the paradoxical pattern with lower scores in spite of their more detailed knowledge and better navigation performance because of a lack of time. So, their numbers of answers were reduced, but not the quality of their answers. In contrast, readers differed in the quality of answers, but not in the number of answers. Therefore, taking the organization of learners' potential knowledge representations into consideration when designing learning tools is worthwhile.
[1] Brusilovsky, P. Adaptive and Intelligent Technologies for Web-based Education. Künstliche Intelligenz, 4/99, 19-25.
[2] Chen, C., and Rada, R. Interacting with Hypertext: A Meta-Analysis of Experimental Studies. Human-Computer-Interaction, 11 (1996), 125-156.
[3] Dee-Lucas, D. Effects of Overview Structure on Study Strategies and Text Representations for Instructional Hypertext. In: Rouet, J.F., Levonen, J.J., Dillon, A.P., and Spiro, R.J. (eds.). Hypertext and Cognition, 73-107. Erlbaum, Hillsdale, NJ., 1996.
[4] Dee-Lucas, D., and Larkin, J.H. Hypertext Segmentation and Goal Compatibility: Effects on Study Strategies and Learning. Journal of Educational Multimedia and Hypermedia, 8 (1999), 279-313.
[5] Foltz, P.W. Comprehension, Coherence, and Strategies in Hypertext and Linear Text. In: Rouet, J.F., Levonen, J.J., Dillon, A.P., and Spiro, R.J. (eds.). Hypertext and Cognition, 109-136. Erlbaum, Hillsdale, NJ., 1996.
[6] Hahn, A., Reich, S., and Schmied, J. Adaptive and Intelligent Technologies for Web-based Education. Corpus Linguistics and Linguistic Theory, Vol. 33, (2000), 131-140.
[7] Naumann, A., Waniek, J., and Krems, J.F. Knowledge Acquisition, Navigation and Eye Movements from Text and Hypertext. In: Reips, U.D., and Bosnjak, M. (eds.). Dimensions of Internet Science, 293-304. Papst, Lengerich, 2001.
[8] Shapiro, A.M. The effect of Interactive Overviews on the development of conceptual structure in novices learning from hypermedia. Journal of Educational Multimedia and Hypermedia, 9 (1999), 57-78.
[9] Unz, D.C., and Hesse, F.W. The use of hypertext for learning. Journal of Educational Computing Research, 20 (1999), 279-295.
[10] Wiley, J. Supporting Understanding through Task and Browser Design. in Proceedings of the Twenty-third annual Conference of the Cognitive Science Society (Edinburgh, Scotland, 2001), Erlbaum, Hillsdale, NJ., 1164-1169.