Date: Fri, 02 Dec 2005 08:26:51 +0700 From: Richard Watson Todd Subject: Research in Applied Linguistics: Becoming a Discerning Consumer
AUTHOR: Perry, Fred L. Jr. TITLE: Research in Applied Linguistics SUBTITLE: Becoming a Discerning Consumer PUBLISHER: Lawrence Erlbaum Associates YEAR: 2005
Richard Watson Todd, Department of Language Studies, King Mongkut's University of Technology Thonburi, Thailand
SUMMARY OF CONTENTS
This book is designed as a text for courses on research methods on Masters-level TESOL and applied linguistics programmes. It claims to provide ''a solid introduction to the foundations of research methods'', but, in contrast to most texts in the same area, it aims to help MA students read, understand and criticise research rather than conduct their own research. Thus, the purpose of the book is for readers to become the ''discerning consumers'' of research of the book's title.
In order to achieve the goal of enabling readers of the book to ''read research reports from beginning to end with a level of understanding that can be used to address both theoretical and practical issues'', the book starts with two introductory chapters examining the nature of research and finding research before moving on to the main part of the book which proceeds through the different sections of a typical research article. There are also three appendices on writing literature reviews, statistics and journals which publish applied linguistics research, together with a glossary. I will briefly look at each of the chapters in turn.
Part 1: Fundamentals for discerning consumers
Chapter 1: Understanding the nature of research The first chapter starts by justifying the purpose of the book, namely, that readers should become ''discerning consumers'' of research. It then moves to discussing the nature of research which allows many of the key terms concerning research (e.g. construct, hypothesis) to be introduced. In common with all of the chapters, in addition to the explanatory text, there are a couple of short exercises for readers to attempt, a list of the key terms introduced in the chapter (most of which can be found in the glossary), and suggestions for further reading.
Chapter 2: How to locate research This chapter gives advice on how to locate and obtain examples of primary research. It includes a substantial section on how to use the ERIC database, but also considers looking at literature reviews and position papers.
Part 2: The major components of published research
Chapter 3: Understanding the framework of a primary research article In this chapter, the reader is taken briefly through the main components of a research article, namely, title, abstract, introduction, methodology, results, and discussion/conclusion.
Chapter 4: Understanding where data come from: the sample This chapter is devoted to the issue of how the research sample is selected, and contrasts the information-rich sampling paradigm with the representative sampling paradigm. Guidelines are given for evaluating the suitability of the sampling strategies used and the generalisability of the sampling.
Chapter 5: Understanding research designs Starting with ways of classifying research designs, this chapter introduces three continua for classifying research: basic - applied, qualitative - quantitative, and exploratory - confirmatory. In the discussion of these continua, three qualitative research paradigms, case studies, ethnography and protocol analysis, are briefly discussed as illustrations. The chapter then moves on to how 'what' and 'why' research questions can be answered through different research designs. The chapter finishes with an in-depth discussion of all the various threats to internal validity, largely as they relate to conducting experimental research.
Chapter 6: Understanding data gathering In the first half of this chapter, a variety of data collection procedures are discussed. The emphasis here is on collecting data from people through observation, questionnaires and tests (as opposed to collecting texts to analyse), and on the potential pitfalls in data collection. The second half of the chapter concerns the reliability and validity of data gathering procedures, focusing especially on the statistics available for evaluating these issues.
Chapter 7: Understanding research results Divided into sections on verbal and numerical data, the chapter starts with a discussion of 17 ''tactics'' for checking the quality of verbal data. The second half of the chapter which concerns numerical data focuses on statistics, especially on the meaning of statistical significance as it applies to generalising results from a sample to a population.
Chapter 8: Discerning discussions and conclusions: completing the picture Much shorter than the preceding chapters, the final chapter gives 7 guiding questions for evaluating the discussion/conclusion section of a research study, and looks at how these can be applied to two example articles.
Appendix A: Constructing a literature review This appendix reviews chapter 2 on locating research, gives guidelines on how to summarise a research study, and presents a template for writing a research review.
Appendix B: Going to the next level of statistics The second appendix provides more details of statistics to add to the discussion in chapter 7. Particular emphasis is given to the various types of ANOVA and controlling for Type II errors.
Appendix C: Journals related to applied linguistics The final appendix contains a useful, if not comprehensive (e.g. no reference to 'Prospect' or 'Journal of English for Academic Purposes'), list of research journals in applied linguistics with URLs and purposes of the journals.
Since 'Research in Applied Linguistics' is designed to be used as a text for MA courses on research methods, how well does it fulfill this purpose? To start with, let us look at what the text does not do. As a book devoted to how to read research, it does not really prepare readers for conducting their own research, although many of the points in the book are applicable. Even within the limits of reading research, the book does not give coverage to some paradigms. There is nothing on the less formal research approaches, such as action research, and a similar dearth of information on text-based research paradigms including genre analysis, policy research and corpus approaches. Furthermore, although there are short sections on qualitative research, the book places a much heavier emphasis on quantitative approaches. 'Research in Applied Linguistics', therefore, is really a book on how to read and understand serious academic research, primarily focusing on quantitative research paradigms.
So, within this narrower field, how does the book stand up? First, the book is replete with useful information about how research should be conducted and reported, some of which, such as the various threats to internal validity, I was not familiar with. This information is generally presented clearly and is well illustrated with examples from sample research studies.
There is, however, a general feeling that the book is nitpicking about research procedures and overlooking the bigger picture. For example, the suggestions for summarising research in Appendix A include an 8- point template. These points include: the hypothesis being tested (in addition to the research question); the size, characteristics and methods of choosing the sample; and the observational, independent, dependent and moderating variables in the study. Surely, experienced readers of research focus on much bigger issues than these points (maybe, the appropriateness and usefulness of the topic or the innovativeness of the methodology) when reading research, and only consider the research technicalities when they are sure that the research is worth spending time reading.
'Research in Applied Linguistics', then, is a book focusing on the details of research. This is an important topic well worth a full-length publication like the one under review. Even at the level of details, however, there are a few problems. For example, on page 83 a supposed example of exploratory research using a quantitative research design is given. The author of the book argues that this is exploratory research since it did not state a hypothesis. However, the research concerned whether bilingualism influences the learning of a third language - which clearly implies a hypothesis even if it is not explicitly stated. The research is therefore not exploratory and so does not illustrate the point the author wishes to make. Another problem, on page 130, is that the author states that correlation coefficients range from 0 to +1 (not -1 to +1). Also, the discussion of the meaning of statistical significance is restricted to whether findings from a sample can be generalised to a population. Not all research using inferential statistics is trying to generalise, and significance can also indicate the probability that the results obtained were not due to chance. In a book of this length, however, these problems are minor annoyances.
To summarise my views on the book, therefore, despite its broad title, it is quite a narrow book. If you are looking for a clear text explicating the details of reading and understanding academic quantitative research, this book serves that purpose well. I would guess, however, that most MA courses on research methods have broader goals than this, and so the book may serve best as a useful supplementary text for certain portions of a course.
My own view as a tutor on an MA course in research methods, however, is perhaps not the most important. The book is designed for Masters students to use. In addition to my own critical evaluation, therefore, I asked one of my students (Nguyen Quang Tuyen), who is currently studying the research methods course at my university, to also read and comment on the book. Below is a summary of his review of the book.
Firstly, on the positive side, Tuyen says that, ''as an MA student, I have 2 major concerns regarding research: how to evaluate and how to conduct research. Therefore, I highly value this book because it focuses on the first, which is different from most traditional works in the field.'' He also states that the websites, lists of journals and frequent examples through the text are useful, and that the language is suitable for his level as a non-native speaker. Furthermore, he found the format of the chapters helpful, since they are organised from general to specific, give clear definitions, list key words, illustrate points with examples of published research, and contain exercises to reinforce learning.
On the downside, he points out that the book binding is not strong enough (a point I can concur with since several pages were loose when he returned the book to me). For the content, he would have preferred more cross-references, especially for where the same research study is used as an illustration in different chapters, he found the lack of any support or feedback for completing the exercises meant that he was uncertain about his answers, and he found some points to be unnecessarily over-complicated. Like me, he was left wondering why action research was not dealt with in the book. Concerning the style, his expectations had been raised in the author's introduction where humour is promised, but he was disappointed that the ''humour was not as much as expected''.
To conclude Tuyen's review, he states, ''in short, the strengths outweigh the limitations, so I wish to combine this book with the traditional ones as both the abilities to do and to evaluate research are interrelated and support each other.''
It can be seen that there are some similarities between Tuyen's and my own reviews, and yet the emphases in the reviews are very different. Apart from highlighting the fact that all book reviews are primarily subjective, our differences seem to indicate that our goals in reading the book (Tuyen was looking for new information, whereas I was viewing the book as a potential coursebook), our concerns with what we want from the book (Tuyen was pleased to find a book focusing on reading research, whereas I would have preferred general coverage of most issues in research), and the ways in which we read the book (Tuyen worked through all the exercises, whereas I skipped them) all had a major impact on our reviews.
ABOUT THE REVIEWER:
ABOUT THE REVIEWER
Richard Watson Todd has worked at King Mongkut's University of Technology Thonburi in Bangkok for over ten years, where his work includes teaching a Masters course on research methods. He has a PhD from the University of Liverpool and has published widely, including books, research papers and newspaper articles.