This book "fills the unquestionable need for a comprehensive and up-to-date handbook on the fast-developing field of pragmatics" and "includes contributions from many of the principal figures in a wide variety of fields of pragmatic research as well as some up-and-coming pragmatists."
Review of Research Methods in Second Language Acquisition
EDITORS: Mackey, Alison and Gass, Susan M. TITLE: Research Methods in Second Language Acquisition SUBTITLE: A Practical Guide PUBLISHER: Wiley-Blackwell YEAR: 2011
Pamela M. Wesely, Foreign Language and ESL Education, Department of Teaching and Learning, University of Iowa
SUMMARY Following on their 2005 book, ''Second Language Research: Methodology and Design'', this edited volume by Mackey and Gass offers an updated perspective on the approaches to research in second language acquisition (SLA). Importantly, this volume is intended “as a guide for students as they design research projects,” as a collection of chapters that give what the authors call “basic background” (p. 1). As such, the volume as a whole seeks to characterize current SLA research and to offer concrete suggestions as to how to follow widely-accepted procedures for research.
The short introductory chapter establishes the general goals of the volume and gives an overview of each subsequent chapter. The chapters in the volume are organized into two different parts: Part 1, Data Types (consisting of nine chapters); and Part 2, Data Coding, Analysis, and Replication (consisting of five chapters).
The introduction also describes the five different elements present in each chapter: (1) the basic background to the area of research; (2) a “how-to” section consisting of a practical, step-by-step guide to the method it covers, often with a reference to actual studies; (3) project ideas and resources, including actual readings; (4) brief summaries of studies that reflect the area of research, presented in shaded “Study Boxes” that contain subsections like Background, Research Questions, Method, and Results or similar; (5) study questions.
Chapter 2, “How to Use Foreign and Second Language Learner Corpora,” by Sylviane Granger, offers a precise definition of learner corpus research (LCR), depicting it as currently shifting from an orientation in corpus linguistics to one more in SLA research, and focused on computerized databases of written or spoken texts. The chapter reviews the main stages in LCR, also taking into consideration the challenges of that type of research. Four sample studies are summarized. The project idea section lists multiple online resources, and the study questions focus on definitional understanding.
Tanya Ionin, in Chapter 3, “Formal Theory-Based Methodologies,” focuses on grammatical acceptability judgment tasks and truth-value judgment tasks (with related picture-matching tasks). Five sample studies utilizing these tasks and addressing formal theory are provided. The author offers a thoughtful sequence for designing and implementing such tasks in the framework of a study. Additionally, she provides print resources and study questions focusing on research design issues.
Chapter 4, “Instructed Second Language Acquisition,” by Shawn Loewen and Jenefer Philp, tackles a broad topic of study in SLA. Limiting themselves to addressing research on general L2 classroom instruction, they focus on three substantively different research methods: observations, non-interventionist quasi-experiments, and interventionist quasi-experiments. Action research is mentioned but not described procedurally as are the other three. The “how-to” for each of the three methods focuses on the selection of a topic (research question), setting, and instruments, and continues through coding. Seven sample studies are summarized, and the authors provide a short list of published resources and study questions that focus on the theoretical as well as the practical aspects of this topic.
The foremost expert on individual differences in SLA today, Zoltán Dörnyei, works with Kata Csizér to address “How to Design and Analyze Surveys in Second Language Acquisition Research” in Chapter 5. Focusing on survey research in “characteristics, opinions, attitudes, and intended behaviors” of a group (p. 74), they offer a brief background of the topic, and then a substantial overview of the processes of creating a questionnaire, sampling, administering, analyzing, and reporting the survey results. One sample study is provided. The authors, in the “Project Ideas and Resources” section, provide one entire questionnaire as well as some additional print references; the study questions subsequently ask the reader to reflect on the provided questionnaire as well as additional application-oriented topics.
Patricia A. Duff focuses on qualitative case studies of individual learners in Chapter 6, “How to Carry Out Case Study Research.” Starting with a background section that acknowledges the difficulty in identifying case studies as just another data type (the author also calls it “a type of research design and written report that highlights cases,” [p. 95]) Duff spends some time situating case studies historically within SLA research. An understandable set of steps for conducting case study research, from identifying a conceptual framework to defending the case study report, follows. The project ideas and resources provide additional information about the topic. Three study examples are provided, as well as a set of complex and interesting study questions.
Chapter 7, “How to Use Psycholinguistic Methodologies for Comprehension and Production,” by Kim McDonough and Pavel Trofimovich, addresses four types of tasks in characterizing this area of SLA research: self-paced reading tasks, self-paced listening tasks, picture-word interference tasks, and sentence preamble tasks. In all four tasks, the authors describe the task, suggest theoretical frameworks, and offer methodological considerations. Four example studies are offered, one for each type of task described. A short list of further readings and definitional and application-oriented study questions are provided.
In the eighth chapter, “How to Research Second Language Writing,” Charlene Polio takes on a considerably broad topic in SLA research. The author summarizes twelve examples of studies on L2 writing that use different research methods in a table. She then discusses each in turn, grouping some together as either analysis of writers’ texts or analysis of the writing process. She offers a brief and open-ended review of the steps of conducting an experimental writing study. Three example studies are described, and although study questions do not appear in this chapter, a substantial set of project ideas and resources are provided.
As with the previous chapter, Chapter 9, “How to Do Research on Second Language Reading,” by Keiko Koda, addresses a vast topic in SLA research. Koda tackles the topic somewhat differently from Polio, explicitly adopting one view of reading, as a “psycholinguistic process involved in reconstructing the message intended by the author based on visually encoded information” (p. 158). In narrowing her focus to one particular theoretical and methodological orientation, she then represents the basic background in some detail, followed by sections about hypothesis formulation, empirical testing, data construction and implementation, and pedagogical interpretations. The author contextualizes all of her explanations of the research process in current studies in the field. Four studies are offered as specific examples; selected further readings, and some detailed study questions are provided as well.
Debra A. Friedman’s Chapter 10, on “How to Collect and Analyze Qualitative Data,” focuses on ethnography and conversation analysis in SLA research, which she identifies as two of the three qualitative research traditions that are widely accepted in applied linguistics (other than case studies, addressed in Chapter 6). The author provides a very readable summary of background information about both traditions, followed by a summary of the process for conducting each type of study, including suggestions for conducting data analysis. Two studies are summarized as examples, and substantial and helpful project ideas and study questions round out the chapter.
Part II focuses more precisely on Data Coding, Analysis, and Replication.
Chapter 11, “Coding Second Language Data Validly and Reliably,” by Andrea Révész, narrows the focus from the overall design of a study to one specific stage of the research process. The emphasis here is on “top-down, theory- and instrument-driven coding methods” (p. 203). In her explanation, the author advises the reader on important issues related to validity and reliability in coding at all points, shifting from selecting and preparing data, to the coding itself (including coder selection and reliability), to the reporting of coding procedures. Two model studies are reviewed. A short set of provocative study questions and project ideas are included at the end.
A counterpart to Chapter 11 is provided in the twelfth chapter, entitled “Coding Qualitative Data,” and written by Melissa Baralt. The chapter exclusively examines computer-assisted qualitative data analysis software (CAQDAS), independent of theory-building and other aspects of qualitative research methodology. With the aid of screenshots and other visuals, the author reviews the process of programming and evaluating the results from using NVivo, stopping at the point of the process where the results are presented to another audience. Three studies that used NVivo are presented as examples, and a short list of resources and study questions are provided.
Jenifer Larson-Hall offers an overview of another stage of the research process in SLA in Chapter 13, “How to Run Statistical Analyses.” As she acknowledges, the chapter is “of necessity a brief survey,” (p. 245), where she focuses on what she identifies as the most basic types of inferential statistical tests in SLA: one-way ANOVA, correlation, the chi-square test, and the t-test. After a short background section, Larson-Hall proceeds through the tests, providing an explanation of when it is used and how to conduct it, followed by sections on its results, effect sizes, reporting, and graphics. Three sample studies are provided, as are multiple useful links to websites, and several study questions.
In Chapter 14, “How to do a Meta-Analysis,” Luke Plonsky and Frederick L. Oswald give a strong overview of meta-analysis in SLA research. They review the specific actions required to craft careful and precise calculations, also while addressing the theory and content knowledge needed in designing a meta-analysis. The authors offer guidance in interpreting results, and include two sample studies, including one by Plonsky. Their lengthy list of study questions is remarkable in its depth and breadth, as is their very logically labeled list of resources.
The final chapter of the text, Chapter 15, “Why, When, and How to Replicate Research,” was written by Rebecka Abbuhl. She provides a basic background that offers the reader examples of different types of replications, including literal/exact replications, approximate/partial replications, and conceptual replications. In the framework of a five-step procedure for conducting replications, the author smartly plots out the pitfalls as well as the advantages to this type of research. She concludes with some intriguing ideas for other projects, as well as important study questions.
EVALUATION This volume has numerous strengths. It achieves what a single-authored (or double-authored) text on research in SLA can never do: it provides a variety of perspectives that truly reflect the diverse and multifaceted research base of SLA. The individual authors are well-known researchers in their areas, and their advice for their readers, ostensibly novice scholars, is always precise and accurate, and sometimes remarkably insightful. There were many, many moments as I read this book when I wished I had had access to it earlier in my career.
Indeed, the editors underestimate the diverse uses for this book. Beyond the volume’s potential as an introduction to research methods in SLA, this book can serve as an important reference volume for any researcher in SLA. Researchers at every stage must craft rationales for and explanations of their research methods, and this volume contains many chapters that could buttress these rationales quite well.
This leads to a consideration of the book’s depth and breadth. The depth is represented by how well each chapter meets the goal of “providing a practical, step-by-step guide to the method it covers” (p. 1). Many chapters, as indicated above, do just that, by offering specific steps and guidelines, as well as warnings about pitfalls, in the method of interest. Some topics are so vast, however, that this “step-by-step” goal is simply not attainable. Some authors resolve this problem by narrowing the definition of the topic. Two notable instances of this are Koda’s focus on reading as a psycholinguistic process in Chapter 9, and Baralt’s emphasis in Chapter 12 on using NVivo only for qualitative coding. In these cases and others, it can be made more explicit, either through the title or through a more extensive acknowledgement of other perspectives, that the authors are selecting one perspective rather than giving a definitive overview of the topic. Other authors resolve the issue of having a very broad topic area by attempting to address the major sub-categories of the topic each in turn. This includes Loewen and Philp’s Chapter 4 that addresses four types of research on instructed second language acquisition, Polio’s Chapter 8 examining twelve different ways to research second language writing, and Chapter 13 by Larson-Hall, which covers four different common statistical tests. Although this prevents the chapter authors from meeting the editors’ stated goals, it is a reasonable solution to a difficult challenge.
Beyond these considerations, I commend the editors for getting a diverse set of 14 different authors to conform to one template for writing about a given topic. Some authors excel in some specific sections. For instance, some (like Dörnyei and Csizér in Chapter 5, and Duff in Chapter 6) offer truly insightful descriptions of the analysis and reporting stages, while others only address that stage briefly. Other authors give comprehensive and wide-ranging lists of well-organized project ideas, resources, and study questions (notably, Plonsky and Oswald in Chapter 14), where others only offer a few definitional study questions and a short list of resources.
An individual’s preferences for what should be included in this volume will vary; I myself would have appreciated a chapter specifically on how to conduct research on individual differences (e.g. Dörnyei, 2005, 2006) and pragmatics and interlanguage (e.g. Bardovi-Harlig, 2006; Kasper & Rose, 1999). More notable omissions are chapters addressing manual qualitative coding (e.g. Miles & Huberman 2005) and mixed methods research (e.g. Creswell & Plano Clark, 2011; Dörnyei, 2007; Teddlie & Tashakkori, 2009).
Finally, in regard to the volume’s breadth, the editors deserve praise for striking a satisfactory balance between qualitative and quantitative approaches to SLA research. As such, the volume might have benefited from either a more substantial introductory chapter or a conclusion chapter. There, the editors could have offered a more comprehensive overview of the approaches to research that have traditionally been associated with SLA research, beyond a simple mention in the first sentence of the introduction. For instance, in the table of contents and as chapter topics, the following are all presented as “data types,” among others: learner corpora, instructed second language acquisition, case studies, qualitative data, and second language writing. An overview might have helped guide the reader to a better understanding of how these areas interrelate and overlap.
This volume ultimately deserves strong praise for its wide-ranging representation of current topics and procedures in SLA research. Although intended for students, it guides all scholars to a better understanding of how we study our field and is an important addition to the library of all SLA researchers, novice or seasoned.
REFERENCES Bardovi-Harlig, K. 2006. Interlanguage development: Main routes and individual paths. AILA Review 19. 69-82.
Creswell, J.W. & Plano Clark, V.L. 2011. Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage Publications.
Dörnyei, Z. 2005. The psychology of the language learner: Individual differences in second language acquisition. New Jersey: Lawrence Erlbaum.
Dörnyei, Z. 2006. Individual differences in second language acquisition. AILA Review 19. 42-68.
Dörnyei, Z. 2007. Research methods in applied linguistics: quantitative, qualitative and mixed methodologies. Oxford: Oxford University Press.
Kasper, G. & Rose, K.R.R. 1999. Pragmatics and SLA. Annual Review of Applied Linguistics 19. 81-104.
Mackey, A., & Gass, S.M. 2005. Second language research: Methodology and design. Mahwah, NJ: Lawrence Erlbaum.
Miles, M.B., & Huberman, A.M. 1994. Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: SAGE Publications.
Teddlie, C. & Tashakkori, A. 2009. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage Publications.
ABOUT THE REVIEWER
ABOUT THE REVIEWER:
Pamela M. Wesely, PhD, is an Assistant Professor of Foreign Language and
English as a Second Language (ESL) Education in the Department of Teaching
and Learning in the College of Education at the University of Iowa, and an
affiliate faculty member of the SLA interdisciplinary doctoral program.
Her current research interests focus on foreign language and ESL in the
K-12 traditional and immersion/bilingual settings, and include L2 learner,
teacher, and parent beliefs and attitudes, the instruction and learning of
culture and intercultural competence, and the use of social media in the L2
classroom. She specializes in mixed methods research methodology.