Purpose – The purpose of this paper is first, to discuss the theoretical assumptions, qualities, problems and myopia of the dominating quantitative and qualitative approaches; second, to describe the methodological lessons that the authors learned while conducting a series of longitudinal studies on the use and usefulness of a specialized balanced scorecard; and third, to encourage researchers to actually use multiple methods and sources of data to address the very many accounting phenomena that are not fully understood.
Design/methodology/approach – This paper is an opinion piece based on the authors’ experience conducting a series of longitudinal mixed method studies.
Findings – The authors suggest that in many studies, using a mixed method approach provides the best opportunity for addressing research questions.
Originality/value – This paper provides encouragement to those who may wish to bridge the authors’ ideological gaps and to those who are actively trying to do so.
In this paper, we respond to the special call by Qualitative Research in Accounting and Management for articles exploring the ways in which qualitative and quantitative research methods can be combined to advance our understanding of accounting, management and organizations. The purposes of this paper are to:
. discuss the theoretical assumptions, qualities, problems and myopia of the dominating quantitative and qualitative approaches;
. describe the methodological lessons that we learned while conducting a series of longitudinal studies on the use and usefulness of a specialized balanced scorecard (BSC) (Malina and Selto, 2001, 2004, Malina et al., 2007); and
. encourage researchers to actually use multiple methods and sources of data to address the very many accounting phenomena that we still do not understand well.
Zimmerman (2001) famously argued that accounting research is firmly wedded to economic theory, and by that many have inferred that he refers to classical microeconomic theory. Others (Hopwood, 2002; Ittner and Larcker, 2002; Luft and Shields, 2002; and Lukka and Mouritsen, 2002) responded vociferously that Zimmerman had overlooked the shortcomings of classic economic theory and the contributions of other behavioral theories to accounting research, such as social and cognitive psychology and sociology. We do not wish to continue this debate, but we observe that many accounting research papers currently seem devoid of axiomatic theory (unlike the heyday of the 1970s and 1980s) and are concerned with documenting empirical regularities. These efforts can be considered basic, pre-theory science, for which more-relevant tools seem to hold great promise for progress.
We echo Euske et al. (2010) and deeply regret that accounting researchers have separated into methodological camps that do not communicate well to refine or modify our incomplete theories and knowledge of practice. Furthermore, we observe that the methodological camps are divided on the nature of the data which are worthy of rigorous examination. In accounting, research and researchers are stereotyped as either “number crunchers” or “navel gazers” (a.k.a. hard or squishy). We, as the accounting research community, have to face the fact that both numbers and words convey meaning and both are needed if we are to understand the world. Gherardi and Turner (1987) suggest that the issue is one of knowing when it is useful to count and when it is “difficult or inappropriate to count at all”, when data are non-standardized and we have no clear rules for saying what is variation and what is error.
We think that the divides between quantitative versus qualitative methods and economic versus other behavioral theories are not constructive toward understanding accounting phenomena. We suggest that in many studies, using a mixed method approach provides the best opportunity for addressing research questions. We hope this paper provides encouragement to those that may wish to bridge accounting’s ideological gaps and comfort to those that are actively trying to do so.
II. Inherent value in quantitative and qualitative methods
Positivism searches for empirical truths. Positivism is not about confronting “things themselves” because direct observation of a phenomenon is subjective and hence not reliable. Likewise, a phenomenologically based methodology also does not create an immediate interpretation of phenomena to the level of concepts, theories or statements of fact. Concepts, for example, are not constructed by a direct confrontation with the phenomena but similarly by analyzing the data collected. For example, qualitative method might construct a phenomenological platform in the form of interview data (similar to a quantitative researcher’s statistical dataset), which is used to construct concepts or reflect on extant theory. This epistemic platform should openly and transparently bridge theory and phenomena, as does a publicly available statistical dataset, to dispel some of the concerns about bias, vagueness, imprecision and distortion of direct observations. If the phenomenological grounding is insufficient, then the evidence and data are likely to be misleading. In statistics, this means that the data are not representative or are missing observations of key variables. In qualitative method, this means that the interview material does not adequately cover the field of study (Nørreklit et al., 2007).
Many accounting qualitative studies use word counts or frequency for content analysis. This assumes that we all use the same language and that there is an optimal length of message. If the number of occurrences is greater than a certain cutoff, for example, then the data are alleged to be saying something of importance. If the count does not meet the cutoff, then the data are not saying anything. Such cutoffs meet many accounting researchers’ comfort in making binary classifications, but this approach seems simplistic. Bliss et al. (1983) tell us that a word or phrase does not contain its meaning as a bucket contains water, but has the meaning, it does by being a choice made about its importance in a given context. Qualitative data analysis should bring meaning and understanding to the research question. This comes from the human judgment of context and is, therefore, subjective. Historically, we, as accountants, have not been visibly comfortable with judgment in academic research, although choice of research questions, methods and interpretations surely are subjective judgments. However, the European tradition seems to be more inclined to use qualitative methods than the American tradition (Panozzo, 1997; Bhimani, 2002).
Mixed method research employs both approaches iteratively or simultaneously to create a research outcome stronger than either method individually. Overall, combined quantitative and qualitative methods enable exploring more complex aspects and relations of the human and social world. Some of these aspects and relationships may be analyzed quantitatively and qualitatively. Ambiguity is not a matter of qualitative method versus quantitative method, but whether the underlying and revealed concepts are valid representations of the phenomenon. In both quantitative and qualitative methods, concepts can be imprecise and open to interpretation. Salomon (1991) argues that the issue is not quantitative versus qualitative methods at all, but whether one is taking an “analytic” approach to understanding a few controlled variables, or a “systemic” approach to understanding the interaction of variables in a complex environment. Firestone (1987) suggests that quantitative studies persuade the reader through de-emphasizing individual judgment and stressing the use of established procedures, leading to results that are generalizable to populations. However, qualitative research persuades through rich description and strategic comparison across cases, thereby overcoming the “abstraction inherent in quantitative studies” and permitting generalization to theory (Yin, 2004). Qualitative research typically answers research questions that address “how” and “why” whereas quantitative research typically addresses “how often” and “how many”. The above reflections give reason to suggest that a mix of quantitative and qualitative methods can be fruitful for obtaining profoundly new empirical insights. As quantitative methods need valid conceptual grounding, qualitative methods are probably always a necessity to understand social phenomena. It should be noted that Einstein’s mathematical work was conceptual, which later was investigated more empirically. Whether qualitative to quantitative research is staged iteratively from one to the other or simultaneously, it seems clear to us that both methods can contribute to understanding accounting phenomena.
III. An example of mixed method research
The aim of this paper is to highlight the lessons we learned while conducting a multi-year field study of a specialized BSC. Our research project began in 1999 when the BSC was just beginning to hit the academic literature. We gained access to a Fortune 500 company that had recently implemented a BSC for its distribution channel. At the time, the business community and consultants advocated the use of the BSC, but very few academic studies had been conducted on either its use or usefulness. The main focus of attention for both practitioners and academics was the novelty that the BSC expressed an organization’s strategy via a combination of both financial and non-financial measures of performance. Most academic research focused on verifying one or more putatively causal links between contemporaneous or leading non-financial measures and lagging financial measures (Ittner and Larcker, 1998).
We set out to examine the process and impact of managing an organization with non-financial performance measures, specifically in the context of a performance measurement model such as the BSC. We used both qualitative and quantitative methods and various theories to answer our research questions and to understand the phenomenon of an enduring, evolving BSC. The following paragraphs provide a brief summary of the papers as well as how qualitative and quantitative methods were utilized in the studies.
The first paper (Malina and Selto, 2001) reports evidence on the effectiveness of the BSC as a strategy communication and management-control device. We conducted 14 semi-structured interviews and used company-provided archival qualitative and quantitative BSC data. Based on business communication and management control theory, we used computer-aided qualitative data analysis to model the use and assess the communication and control effectiveness of the BSC. We relied almost exclusively on a qualitative method that cross-tabulated both a priori and in vivo coding of interview transcripts. We conclude that this specific BSC, as designed and implemented, is an effective device for controlling corporate strategy. The results also indicate disagreement and tension between top and middle management regarding the appropriateness of aspects of the BSC as a communication, control and evaluation mechanism.