Date: Fri, 18 Mar 2005 15:16:51 -0500 From: Uma Pappuswamy Subject: Use and effect of declarative information in user instructions
AUTHOR: Karreman, Joyce TITLE: Use and Effect of Declarative Information in User Instructions SERIES: Utrecht Studies in Language and Communication 18 PUBLISHER: Rodopi YEAR: 2004
Umarani Pappuswamy, Research Associate, Learning Research and Development Center, University of Pittsburgh, USA.
BOOK'S PURPOSE AND CONTENTS
The book 'Use and effect of declarative information in user instructions' is a PhD thesis by Joyce Karreman, promoted by Michaël Steehouder and Peter Jan Schellens at the University of Twente, Netherlands. This study investigates the effects of different information types in instructions for use.
The work of Ummelen (1997) serves as the starting point for many researchers studying reading and learning from texts and Karreman is not an exception. According to Ummelen (1997), information in instructive texts can be either procedural or declarative. Information types that are directly related to the functioning of the product, for example, "actions" that must be performed to get a product working, conditions for actions and results from these actions are "procedural" whereas all other (explanatory) information about the internal workings of the device, trouble-shooting tips etc. are "declarative". Previous studies have shown that adding declarative information in an instructive text helps to improve the task performance (Kieras and Bovair 1984; Smith and Goodman 1984; Payne 1988) in a number of ways. The types of declarative information used in these studies were: information about how a system works; metaphorical and information about the interface (p.43). Karreman extends research in this area with the goal 'to investigate whether particular types of declarative information lead to specific positive effects during the process of learning to work with a device' (p.3).
The study is presented in the form of five chapters.
Chapter 1 presents an overview of the literature on using instructive texts. It describes two general theories on reading text: Van Dijk and Kintsch (1983) and ACT* theory of Anderson (1983). Van Dijk and Kintsch (1983) distinguish three models of mental representation, viz., surface representation, propositional representation and situational model. After a careful analysis of these three representations of text comprehension, the author concludes that this theory cannot be directly applied for describing the process of operating devices by using instructive texts. However, the author finds the notion 'situational model' to be more useful to the topic of her research. Karreman discusses ACT* theory on the description of procedural learning in detail and finds that it is also not an appropriate theory for explaining effects of instructive texts. In the next three sections, the author provides various other theoretical constructs that were essentially outgrowths of the two reading theories mentioned above. Kieras and Bovair (1984) assume that instructive texts are read as in Van Dijk and Kinstsch (1983). They also provide a characterization of how suitable declarative representations are formed based on the instructions read (which was not clear in Anderson's ACT* theory). Karreman discusses their model of procedure acquisition and some experimental results that threw light on the effect of declarative information on task performance in depth. Payne (1988, 1991 and 1993) criticized Kieras and Bovair (1984)'s model and Karreman finds some of his comments to be a valuable contribution to the topic of effects of declarative information in texts (p.22). Lastly, the author discusses Dixon (1987, 1993)'s model of hierarchical planning framework with the central assumption "mental plans have a hierarchical structure, in which the task is described in general terms at the top of the hierarchy and specified more precisely at lower levels" (Dixon 1993, p.377).
The author also presents a comparison of all the five theoretical constructs in the latter part of the chapter with a reference to the analytical data from experiments concerning declarative information in instructive texts. The experiments are about the effects of adding declarative information to procedural instructions; the effects of declarative information compared with the effects of examples or results of practicing. The chapter concludes with a table that neatly summarizes the results of these experiments.
Chapter 2 describes an experiment which differs from Ummelen (1997) in three aspects. Firstly, it tests if Ummelen's results could be extended to another new system. Secondly, unlike Ummelen whose experiments focused on the effects of declarative information in general, Karreman investigates two particular types of declarative information: system and utilization. System information is about the internal working of the product whereas utilization information is about the reason, circumstances under which a particular function of the system can be used. Thirdly, Karreman's experiment uses 'reading times' in addition to Ummelen's 'using times' (p.47). The methods, designs and results of the investigation are discussed in detail. Justifications for selecting Ummelen's 'Click and Read' method over the others are also provided. The results of this experiment showed that system and utilization information are read during the orientation phase but unfortunately they did not have any effect on the task performance and knowledge. Hence, Karreman devised a follow-up experiment that uses a combination of 'Click and Read' and 'Think Aloud' methods. The results of this experiment also leave the question of absence of any effect of declarative information on task performance.
Chapter 3 discusses another experiment and a follow-up study both of which are designed differently from the studies described in chapter 2. The goal here is to investigate the possible effects of cognitive information on cognitive load, confidence and the appreciation of the instructions and the device. The results demonstrate that declarative information in user instructions does not affect the appreciation of the instructions or the device but has a negative effect on the cognitive load and the confidence of the users. This experiment also showed that declarative information does not have clear effects on task performance. Liberal use of figures and tables makes the ideas clear and interesting.
Chapter 4 investigates the effect of system information on the "transfer of knowledge". The author presents a brief overview of previous research on the theory of 'structure mapping' that discusses the role of analogy and similarity in human cognition along with a review of some empirical studies on this concept. Karreman prefers to use the terms 'structural similarities and superficial similarities' (p.135) for her study to investigate if system information about one device will result in a higher degree of transfer when learning to operate structurally similar devices. She conducts three experiments which are extensions to the series of experiments conducted by O'Reilly and Dixon (1999, 2001 and 2002). Each experiment is based on previous research on problem solving and transferring knowledge to new tasks. The main finding is that order of devices affected the degree of transfer (p. 167). Karreman reports the observations of these experiments at length.
Chapter 5, the final chapter, summarizes the results of her experiments, conclusions and discussions. It also provides answers for the three research questions raised in the previous chapters. The questions are: 1. Are system and utilization information read? 2. Do system and utilization information affect knowledge, cognitive load, confidence and appreciation? 3. Do system and utilization information affect task performance?
Her main conclusions are that declarative information has a weak effect on task performance and had some unexpected effects on user's confidence in performing a task. She outlines some directions for future research in this topic.
Lastly, the author presents a neat compilation of the references running into seven pages.
This book is a good starting point for researchers interested in information in instructional texts. The author provides a good presentation of various methods and techniques used in investigating the use and effect of declarative information. The language is clear and effective and the text material is supported by tables and graphics wherever needed. The chapters are arranged in such a manner that the findings of one chapter serve as a stepping stone for the next one. Most of the experiments reported in the chapters 2, 3 and 4 have been also previously published by the author and her co-authors Steehouder and Dixon (see the book for references). As pointed out by the author herself in many places, her findings provoke several interesting hypotheses that need further investigation from a cognitive point of view.
Though it was not within the scope of the study, the author could have written a short section on potential guidelines as how to write useful and effective instructive texts in general and what kind of system and utilization information should be included in an instructive text. Such guidelines could be of great help to user-manual writers.
The book has a minor flaw in typesetting which changes the meaning of the sentence the author intended to convey: on page 132, in line 22, the word "cannot" should have been " can not" ('These users cannot only execute tasks that are explicitly formulated ... ; they can also ..."). Besides this, there are other minor presentation issues, for example, repetition of hypotheses (shared between two experiments) on p. 152 could have been avoided by simply providing reference to the list given on p. 142. The section 4.2 , titled 'First Experiment', goes on to describe O'Reilly and Dixon's experiments before she sets hers own hypotheses for the desired experiment (p. 138-142). This spoilt the readability of the text at least for this reviewer who feels that this could be avoided by moving this portion of the text to the subsection on literature review on p. 137 and Karreman could have provided justifications for her motive to build upon O'Reilly and Dixon's hypotheses. The book also lacks a subject- index for reference purposes to concepts, techniques and domain-specific vocabulary mentioned in the book.
Overall, this book should appeal to those interested in research on learning from texts. It could also serve as a course material for students designing experiments on information texts.
Dixon, P. (1987). The processing of organizational and component step information in written directions. Journal of Memory and Language, 26, 24- 35.
Dixon, P., et al. (1993). Effects of sentence form on the construction of mental plans from procedural discourse. Canadian Journal of Experimental Psychology, 47, 375-400.
Kieras, D. E. and Bovair, S. (1984). The role of a mental model in learning to operate a device. Cognitive Science, 8, 255-273.
O'Reilly, T., and Dixon, P. (1999). Procedures are only skin deep: The effects of surface content and surface appearance on the transfer of prior knowledge in complex device operation. Proceedings of the Cognitive Science Society Conference, 21, 486-489, Hillsdale, NJ:Erlbaum.
Payne, S. J. (1988). Metaphorical instruction and the early learning of an abbreviated-command computer system. Acta Psychologica, 69, 207-230.
Payne, S. J. (1993). Memory for mental models of spatial descriptions: An episodic-construction-trace hypothesis. Memory and Cognition, 21, 591-603.
Smith, E. E. and Goodman, L. (1984). Understanding written instructions: The role of an explanatory schema. Cognition and Instruction, 1, 359-396.
Ummelen, N. ( 1997). Procedural und declarative information in software manuals. Effects on information use, task performance and knowledge. Amsterdam/Atlanta: Rodopi.
Van Dijk , T. and Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic press.
(For other references mentioned in this review, please see Karreman's book).
ABOUT THE REVIEWER:
ABOUT THE REVIEWER
Dr. Umarani Pappuswamy is a research associate at LRDC, University of Pittsburgh, USA. She has a Ph.D. in Linguistics with specialization in computational linguistics. Her main areas of research interests are: computational semantics, intelligent tutoring systems, machine learning, discourse analysis, typology, and corpus linguistics.