Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.
Here we go. Three new quotes from the same ole’ article for a brand-new story. This time, I asked AI to suggest six (6) page numbers. I chose three (3) and then chose quotes on each of those three pages and arranged them into a story.
(1) “A theme might be given considerable space in some data items, and little or none in others, or it might appear in relatively little of the data set. So, researcher judgement is necessary to determine what a theme is. Our initial guidance around this is that you need to retain some flexibility, and rigid rules really do not work” (Braun & Clarke, 2006, p. 82).
The theme that circles during this LDA challenge is rigor. I once read a LinkedIn post from Victoria Clarke asking for a systematized definition of rigor within the qualitative research literature, which to me suggests that there might not be one. Or that the term is underdefined.
One aspect of rigor that bumps up against the black box of reflexivity is the flexibility of what constitutes a theme. With quotes like these, I appreciate the important tensions that Braun and Clarke eloquently name and refuse to resolve. That refusal to resolve the tension between rigor — as demanding comprehensive, systematic, transparent analysis — and reflexivity — which demands intuition, discernment, and storytelling skills beyond what researchers can empirically systematize — makes this paper fascinating and tragic. Fascination comes from the mixed register of rigor with reflexive creativity. Tragedy comes from the endless hermeneutic circle of reflexivity; telling the story of data cannot help but impose coherence onto non-linear, recursive movements by a research apparatus (i.e. the researching self) that fixes epistemological commitments like nitrogen in academic soil. Reflexivity is a storied process.
Hindsight is 20/20 and reading Braun and Clarke 2006 alongside 20 years of later publications, we can see how many researchers recontextualized their six-step starting point as a checklist — a checklist that gives critical ammunition to those of a postpositivist bent. It became easy to read Braun and Clarke (2006) as a model to follow, rather than a jumping-off point into a method that refuses its own codification. Rigid rules do not work. You need to retain some flexibility. That makes a lot of researchers, including myself, very uncomfortable at times – especially in my own field of applied linguistics where social scientists impose postpositivist criteria on qualitative inquiry. This is a need to take seriously, as applied linguistics often stresses practical takeaways and social transformation that follow one’s research.
This marks an often-unexpressed takeaway from this article: qualitative research aims to answer questions that quantitative researchers cannot. (Reflexive) thematic analysis cannot abide by the same (post)positivist criteria expressed in quantitative research. In reflexive thematic analysis we cannot outsource rigor or credibility to numbers, statistics, or mathematics in the same way that quantitative analyses can. Or we can, but according to later papers such a practice loses what makes reflexive thematic analysis reflexive. Both the reading and the writing of (reflexive) qualitative analysis constitute interpretive acts. And “rigor,” or “trustworthiness” is therefore not a standardizable quality, but an interpretive act at the discretion of whomever reads it. There is no p-value that can satisfy a qualitative report. Yet despite their adamant refusal to treat reflexive TA as a methodological checklist, Braun and Clarke supply one on page 96.
Even reading these criteria, we see where ambiguity and discernment wheedle themselves in. What constitutes “appropriate level(s) of detail” in transcripts? What constitutes appropriate boundaries for a “data item”? How do I distinguish between an analytic, a narrative, or an illustrative extract? Even checklists require interpretation. Which is why Braun and Clarke are so adamant to point out this article as a starting point rather than a map or manual for thematic analysis. Because ultimately, a subjective sense of “fit” and “sense” undergirds good thematic analyses – the sense to tell a compelling story that fits the data. Alas, if only data were so yielding.
(2) Note that no data set is without contradiction, and a satisfactory thematic ‘map’ that you will eventually produce, an overall conceptualization of the data patterns, and relationships between themes, does not have to smooth out or ignore the tensions and inconsistencies within and across data items. It is important to retain accounts that depart from the dominant story in the analysis, so do not ignore these in your coding (Braun & Clarke, 2006, p. 89).
Here we again see the beautiful paradox: Braun and Clarke are not shy to lay out their criteria for what constitutes a bad thematic analysis — one being “themes that have too much overlap.” Yet there is also a call for themes to connect in a storied fashion. Here the line between connection and overlap becomes blurry. And it raises an important question: what is a systematic, inconsistent thematic analysis? Or is it the analysis that is systematic and the themes that are inconsistent? If the analytical process is non-linear and recursive, it can also be unpredictable. Unpredictability and serendipity often define analytic turning points, yet they are also the most difficult to represent in a way that sounds rigorous and systematic.
Braun and Clarke argue that a satisfactory thematic map need not smooth out tensions or inconsistencies. However, when we start to write the “story” of our analysis, oftentimes it is the inconsistencies that we smooth out for the sake of the reader. One can only be so transparent, and one can only write so many tangents before the story loses coherence — particularly on an 8,000-word budget. I read this very often in applied linguistics papers – the enumeration of beautiful themes before emphasizing that these themes overlap, co-occur, and dynamically interact (most analyses don’t go beyond this vague claim in the discussion sections). This is where I get stuck within the logic of the theme. The theme is a noun. Yet scholars now conceptualize language as an activity. And when I aim to translate nominal themes into an analysis of activity, I struggle to tell a coherent and concrete story. I just arrive at a list of themes coupled with a caveat that the themes are entangled, overlapping, and take different shapes with different participants.
The closer I read Braun and Clarke (2006), the farther it retreats from the ideal of a bulletproof thematic analysis. Bulletproof was never the goal. Without shared, uniform standards, any thematic analysis, no matter how rigorous, is vulnerable to the analyst who defines rigor differently. Because, like analysts, reviewers also carry unstated epistemological assumptions. And when those assumptions come into contact, some reviewers are more generous than others. A willingness to embrace ambiguity and inconsistency, while also demanding transparency and systematic analysis, will ultimately produce residues of luck regarding which analyses move on to review and publication. In other words, peer review often resembles vibe-checking more than it does a systematic audit of a piece of research in qualitative inquiry. Or, as Pratt (2008[1]) reported in an article about qualitative research in organization studies, 40% of reviewers look for papers that are well-written and make strong contributions to theory or practice rather than papers that have rigorous methods. We teach scholars to be systematic, but it does not mean we teach them to write well. And nowadays I wonder if good writing can take one further than rigorous analysis ever could. Again, the ideal is to have both. But when tensions arise, which direction do you commit? Because we all have commitments.
(3) What is important is choosing a method that is appropriate to your research question, rather than falling victim to ‘methodolatry’, where you are committed to method rather than topic/content or research questions (Holloway and Todres, 2003) (Braun & Clarke, 2006, p. 97).
Here I start to fall behind. If I understand what Braun and Clarke mean by “methodolatry,” they speak of researchers’ tendency to force-fit data into a method of choice, rather than allowing the research question to guide one’s method of choice. To that I ask: are researchers method-agnostic? Do themes emerge from data? And part of reflexivity is not just one’s identity, life capital, and relationship to participants. It also includes one’s implicit epistemological commitments. These epistemological commitments need not be one-track mind. I would argue that everybody has a methodological toolkit, or an instrumentarium, as Rampton, Maybin, and Roberts (2015) calls it. And many of us mix and match elements from different methods in a way that suits our research questions.
But I can already see a particular misunderstanding coming out of this quote: researchers are people too, people with proclivities toward certain types of data and certain types of analysis of data. Perhaps the question is not, “What is the best method for the research question?” Often interest in a community or phenomenon precedes a research question – and such interests often load epistemological assumptions before research questions arise at all. Research questions are easy to change to fit a method. The reverse is not often true.
That brings me back to something that Braun and Clarke stress about writing. Namely, that writing never stops. It never finishes. It never even really starts; it co-occurs throughout the research process. The research report is not a siloed-off final step as their Step 6 insinuates. Rather, writing is a continuous process, endlessly unfolding one finds and refines the story told through the data.
This is not something I hear about often. There seems to be an implicit assumption that people follow research reporting conventions to “neutrally” present the results of their coding and analysis. But when we ask if the method fits the research question, another important question is, “How do you like to write?” Or of we like writing in the first place. If the answer is yes, then writing becomes the method, as Elizabeth St. Pierre argues. If the answer is no, it is worth asking: Do researchers perform the conventions of a transparent analysis? And if not, where are the cracks? And as a reader, how do I locate those cracks?
This “need to locate cracks” from reviewers who all have unique qualitative methodological fingerprints creates a sticky situation for peer review. Which is also why — and I know this is an unpopular opinion — learning how to write well can be more important than doing thorough data analysis. Because a good story tops a boring story — rigor be damned. Of course, we would love to have the best of both — a top-notch rigorous story. But with 8,000 words and publish-or-perish breathing down our necks, who has the time? Or the space?
[1] Pratt, M. G. (2008). Fitting oval pegs into round holes: Tensions in evaluating and publishing qualitative research in top-tier North American journals. Organizational research methods, 11(3), 481-509.
Leave a Reply