I was searching through tweets using the #CAQDAS hashtag the other day and came across one that sent me reeling. And not in a good way. I’ve since been pondering why it prompted such a strong reaction, which might not have happened had I not been to the recent ICQI conference.
The language of determinism and constructivism in CAQDAS discourses
At the opening plenary of the Digital Tools stream at ICQI, Kristi Jackson, of Queri, Inc. highlighted that critics of CAQDAS often frame their positions in the language of determinism, whereas advocates use the language of constructivism. She noted that to determinist critics “the software limits personal agency by standardizing processes” – whereas to constructivist advocates “the software expands options and promotes diversity” [i].
Spot on.
Using analogies from American urban planning history, Kristi highlighted the ‘boundary work’[ii] CAQDAS critics and advocates both do in ‘other-izing’ one another. She mentioned two particularly important points:
that although as CAQDAS enthusiasts we may be constructivist in the way we think about the role of CAQDAS and the way we talk about digital tools, CAQDAS critics also use constructivist language in defining their own, non-CAQDAS work;
that CAQDAS enthusiasts often use unflattering descriptions to criticise those who chose not to use CAQDAS, such as ‘luddites’, ‘rigid’, ‘behind the times’, ‘wildly uninformed about the capabilities of the software’.
I really enjoyed Kristi’s presentation because it challenges us as a community of practice to think more critically about the ways we talk about the use of CAQDAS and think about and research the relationship between technology and methodology. I’m not doing justice to Kristi’s work here, so I urge you to read her article which is currently being prepared by the Digital Tools SIG. (I’ll link to it from here when it’s available).
If I hadn’t heard Kristi’s presentation I don’t think I would have stopped to think about my reaction to the tweet that sent me reeling, and I wouldn’t now be questioning my default standpoint qua CAQDAS through writing this blog-post.
The tweet that got me reeling
It was posted on the 7th June at 20.50 by @GT_SummerSchool and showed a picture of slides being presented at the International Summer School on Grounded Theory Methods that took place between 6-10 June 2016 in Pisa, Italy.
The slides listed ‘pros’ and ‘cons’ of using CAQDAS packages. I’m always interested in why people choose not to use dedicated CAQDAS packages, so it was only natural to zoom in and check it out.
I don’t know who was presenting, but the pros and cons were listed as follows:
Pros
Help in organizing and managing data
Help in looking at complex relationship in the data
Help in storing additional data. Memos and annotations
Help in building complex conceptualisations
Most of all helps in coding, categorizing and retrieving
Cons
Time-consuming in learning and practising
Eventually, produce “distance” amongst analyst and data – social worlds
Detach data from contexts in which they were collected
Favour increasing homogeneity in methods of data analysis
Availability and choice of software (price, complexity etc.)
The first and last of the listed Cons I would certainly agree are issues. Definitely, it’s time-consuming to learn to harness CAQDAS packages powerfully – that’s one of the key observations behind the work Nick and I have been doing, in collaboration with many colleagues, to develop Five-Level QDA as a method of learning and teaching CAQDAS packages. Availability is also a big issue, especially for those working or studying in institutions that have invested in a site licence for only one product. But when this isn’t the case there’s actually a lot of choice. There are at least 8 or 9 dedicated CAQDAS packages that have specialised tools. You can read independent reviews of them at the CAQDAS Networking Project website. Also, there is a range of non-dedicated software programs that are used to facilitate qualitative and mixed methods analysis – for example Word, Excel, Onenote or Scrivener.
It’s the three Cons in the middle of the list – distancing, decontextualising, and homogenising – that raised my temperature: because I don’t believe they’re true! I’ve heard these sorts of criticisms – and others besides – many, many times over the years. When CAQDAS packages first became available they were to varying degrees clunky and provided limited functionality. But things have changed as technology has developed, so part of the reason I reacted strongly was because I was thinking ‘same old, same old’.
But then I stopped to think – was I guilty of what Kristi warned against? Was I being deterministic in my thinking in critiquing the ‘other’?
We have to listen to the ‘other’ viewpoint
Maybe I have a vested interest in researchers using CAQDAS – seeing as I have built my career on teaching and supporting their use. That’s true. But that’s not why I got upset. I got upset because I think it’s irresponsible to promulgate inaccurate and un-evidenced criticisms. I truly believe that researchers have the right to make their own choices about whether to use CAQDAS packages or not. And I believe there are some good reasons not to use them.
There are also some very good alternative digital tools that many researchers successfully use. At my CAQDAS workshops I make it clear that just like it’s possible to do good-quality analysis without using CAQDAS packages, it’s also possible to do bad-quality analysis with CAQDAS packages. This of course speaks to the role and capabilities of these tools. CAQDAS packages do not and cannot do any analysis, or take interpretive control away from the analyst - although that may change as intelligent computing advances in future years.
What they do offer is a range of tools that can be harnessed to undertake a range of different analytic strategies. Some analytic strategies are not yet well enough supported but I don’t think that constitutes “homogenisation of methods”. If others do then let’s see up-to-date evidence for that point of view and then let’s discuss it. Then we can move the debate forwards and encourage the developers to provide finer tools.
The point is that the choices students and researchers make about whether to use CAQDAS at all, and if they do, which package to choose, should be informed. Encouraging critical reflection about the role and use of CAQDAS packages is very important. And I’d argue, as Kristi does, that we need more of it. But let’s make sure those critical reflections are based on the reality of what is possible when using CAQDAS packages. Otherwise, we risk unfairly skewing the mindsets of the next generation of researchers qua the role of digital tools.
What upsets me isn’t that people criticise CAQDAS or don’t use it. What upsets me is when I see criticisms which are based on misinformation. As researchers and teachers, we have a responsibility to the next generation to engage in open discussions about the role of CAQDAS, and that includes advocates being sensitive to the concerns of critics. As Kristi made very clear, thinking about the language we use is part of that – whether we are criticising CAQDAS, advocating its use, or indeed criticising the criticisms.
Of course, as a qualitative researcher, I am very conscious about the dangers of interpreting out of context. I wasn’t there. I don’t know what was said whilst the slides were being shown. It might be that the presenter was refuting the oft-seen criticisms of the use of CAQDAS listed in the Cons column rather than suggesting them to be accurate or valid reasons not to use CAQDAS packages. I’d love to know, actually, so if anyone who attended the @GT_SummerSchool International Summer School on Grounded Theory Methods reads this, I’d be really interested to start talking.
Practising what I preach
In the meantime, in order to practice what I preach, our next blog posts will take each of the three Cons that riled me in turn and illustrate why we believe them not to be true. And that will involve evidencing the claims I made above about what CAQDAS does and does not do. Because, just as it’s not fair to make categorical disparagements without backing them up, it’s not fair to categorically promote CAQDAS without backing it up.
[i]“Determinism vs. Constructivism: the polarising discourse regarding digital tools for qualitative research and how it threatens our scholarship”, Paper presented by Kristi Jackson (www.queri.com) at The International Congress of Qualitative Inquiry in Urbana-Champaign, Illinois as part of the Digital Tools for Qualitative Research SIG. Paper #4 in the Critical Plenary. Critical Plenary Title: The Construction and Use of Digital Tools for Qualitative Research: Challenges on the Horizon. Friday, May 10, 2016: 8:00am – 9:20am
[ii] Gieryn, T. F. (1983). Boundary-work and the demarcation of science from non-science: Strains and interests in professional interests of scientists. American Sociological Review, 48.
781-795. Gieryn, T. F. (1999). Cultural boundaries of science: Credibility on the line. Chicago: The University of Chicago Press.