Using ATLAS.ti to Facilitate Data Analysis for a Systematic Review of Leadership Competencies in the Completion of a Doctoral Dissertation

The author used ATLAS.ti to conduct a systematic review of the literature on leadership competencies in fields undergoing rapid change to complete his dissertation. Studies were imported to ATLAS.ti for first, second and third stage analysis which led to the creation of final themes and concepts. The use of ATLAS.ti for coding encouraged a cyclical and iterative approach to data analysis that would have been difficult to accomplish through note cards, word processing, or spreadsheet applications. ATLAS.ti assisted with using meta-ethnography as the means of synthesizing both qualitative and quantitative research. ATLAS.ti provided the ability to make chains of multiple codes and linking of quotations to create networks, which was vital to third stage coding. These network diagrams were qualitative visual representations of the data and greatly assisted with third stage coding. Concepts, themes, and patterns emerged from the network diagrams. Also useful at this stage for identifying patterns in the data was mapping of co-occurring codes. ATLAS.ti allows co-occurring codes to be retrieved and visualized through network and mapping tools. Finally, ATLAS.ti was used to create hyperlinked maps of quotations which assisted with the analysis of proposition three of this dissertation.


USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
rating scientific knowledge in the content and process of making decisions" (p. 3). EBM involves a fourstep process: Use of the best available scientific findings; gathering and attending to organizational facts, indicators and metrics in a systematic fashion to increase their reliability and usefulness; on-going use of critical, reflective judgment and decision aids in order to reduce bias and improve decision quality; and consideration of ethical issues including the short and long term impact of decisions on stakeholders. (Rousseau, 2012, pp. 4-5).

Evidence-Based Librarianship
Much like evidence-based management, evidence-based librarianship (EBL) is a relatively recent phenomenon. The movement began in health libraries as the evidence-based medicine trend quickly affected the libraries associated with hospitals and medical/nursing/public health schools. The majority of evidence-based information science is still authored by health librarians. The two major figures in the area are Jonathan Eldredge, in the United States, and Andrew Booth, in Great Britain. Eldredge (2006) defined evidence-based librarianship as "a process for integrating the best available scientifically-generated evidence into making important decisions, it seeks to combine the use of the best available research evidence with a pragmatic perspective developed from working experiences in librarianship" (p. 342). Booth and Brice (2004) defined EBL as "an approach to information science that promotes the collection, interpretation, and integration of valid, important and applicable user-reported, librarian-observed, and research-derived evidence; the best available evidence, moderated by user needs and preferences, is applied to improve the quality of professional judgments" (p. 7).
Unlike some other disciplines, EBL, because it has grown out of the social sciences, makes use of both quantitative and qualitative research in its systematic review. "Librarianship is in the unique position of being able to conduct and use the different types of studies available" (Crumley & Koufogiannakis, 2002, p. 65).

Systematic Review
Systematic reviews are not new and did not arise from evidence-based medicine. The first systematic reviews were produced in the fields of psychology and education (Littell & Corcoran, 2010). Later, systematic reviews became part of the evidence-based medicine phenomenon and grew greatly in popularity.
The creation of the Cochrane Collaboration in 1993 to gather data on health care interventions greatly assisted with the acceptance of systematic review (Littell & Corcoran, 2010). Systematic reviews can be defined as "carefully organized, comprehensive and transparent studies of previous research on a particular topic that follow written protocols that specify their central objectives, concepts, and methods in advance" (p. 313).
There are many different methodologies for conducting systematic reviews. However, all systematic reviews "include procedures for identifying, critically appraising, synthesizing, and presenting results of

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
previous studies" (Littell & Corcoran, 2010, p. 313). One of the strengths of systematic review is that bias and error can be minimized through following proper procedures. Another advantage is that the well-documented stages of a systematic review create transparency and allow for readers to evaluate or reproduce the research conducted. The key features of systematic reviews, then, are: an explicit research question to be addressed; transparency of methods used for searching for studies; exhaustive searches, which look for unpublished as well as published studies; clear criteria for assessing the quality of studies; clear criteria for including or excluding studies based on the scope of the review and quality assessment; joint reviewing to reduce bias; and a clear statement of the findings of the review. (Evans & Benefield, 2001, p. 529).

Research Process
The research process involves steps used to conduct a systematic review on a particular topic area. The steps utilized were those outlined in the Cochrane Collaboration's Reviewer's Handbook and expanded upon by Booker (2010) as she was the first scholar to combine qualitative and quantitative studies in a meta-ethnographic review. These steps are: 1. Identify the topic of interest 2. Select studies relevant to the topic 3. Read the studies 4. Determine how the studies are related 5. Translate the studies into one another 6. Synthesize translations 7. Express the synthesis ATLAS.ti was heavily used in steps four through seven of this systematic review research process.

Administrative Systems
One of the more mundane but important tasks while conducting a systematic review is keeping track of included and excluded articles and creating a reference list.
Since systematic reviews aim to be a comprehensive syntheses of research literature, it is vital that reviewers keep track of the reports they find and are able to present a full account of how each report is dealt with as the review progresses" (EPPI-Centre, 2010, p. 5).
Systematic reviewers may be dealing with hundreds if not thousands of references over the course of a review. Creating a proper audit trail is vital for updating the review and for future researchers who may wish to recreate the review process.
Retrieved articles were first saved into folders in the various vendor databases. After a preliminary screening process, relevant articles were loaded into the Qiqqa PDF management software tool. Reasons for exclusion were noted, most articles eliminated at this stage failed to meet one or more aspects of the dissertation's inclusion/exclusion criteria. Qiqqa was initially used to de-duplicate articles. Because many databases were searched, duplication of results was a strong possibility. As Harris (2005) pointed out, the amount of duplication depends on "the scope and dates the database covers, the journals it indexes, and

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
the indexing terms it employs" (p. 84). Qiqqa was next used to create a list of references in APA style.
Studies were imported from Qiqqa to ATLAS.ti for first, second and third stage analysis which led to the creation of final themes and concepts.
The use of software for coding encouraged a cyclical and iterative approach to data analysis that would have been difficult to accomplish through note cards, word processing, or spreadsheet applications. The structure and functionality of ATLAS.ti does not promote a linear progression of tasks but rather a dynamic and emergent approach to qualitative research (Silver & Lewins, 2014). The ATLAS.ti coding process is not hierarchical but rather inductive. The software provides the "ability to express relationships between codes, concepts, and themes in a range of different ways, and often these cannot be represented in a hierarchical list" (Silver & Lewins, 2014, p. 210). Key to this approach is the creation of network diagrams which were used extensively in the author's dissertation.

Data Synthesis Approach
The data synthesis approach to the systematic review was interpretive. An integrative approach was impossible because heterogeneity of articles made such an approach very difficult if not impossible. A Cochrane-type review of quantitative studies of high methodological rigor was also not supported by the lack of such studies in the library and management literature. The interpretive approach was first suggested by Noblit and Hare (1988). As explained by Dixon-Woods, Agarwal, Jones, Young and Sutton (2005), interpretive reviews "achieve synthesis through subsuming the concepts identified in primary studies into a higher-order theoretical structure" (p. 46). Articles are read and re-read to find similar terms and concepts. Most importantly, "the defining characteristic of an interpretive synthesis is its concern with the development of concepts, and with the development and specification of theories that integrate those concepts" (p. 46). The main product of this process is not data but theory. While Noblit and Hare (1988) proposed interpretive synthesis for qualitative research Dixon-Woods et al. (2005) believe that it could also be used to synthesize quantitative research. The added value of such a framework "lies in its ability to examine systematically a much wider literature than is usually included in systematic reviews to draw conclusions and suggest recommendations for policy and practice" (Oliver, Harden, Rees, Shepherd, Brunton, Garcia & Oakley, 2005, p. 440).

Meta-Ethnography
Meta-ethnography was first proposed by Noblit and Hare (1988) as a means of synthesizing qualitative research. It was an off-shoot of ethnography which was popular with anthropological researchers. Metaethnography involves three major strategies: reciprocal translation analysis (RTA), refutational synthesis, and lines of argument synthesis (LOA). In RTA "the key metaphors, themes, or concepts are identified, and an attempt is made to translate these into each other" (Dixon-Woods et al., 2005, p. 48). In refutational synthesis, the "contradictions between the reports are characterized, and possible refutations are examined and an attempt made to explain them" (p. 48). LOA "involves building a general interpreta-

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
tion grounded in the findings of the separate studies" (p. 48). Meta-ethnography has many advantagesthe primary one being that it preserves the interpretive properties of primary qualitative data. Although it was created for synthesizing qualitative research, meta-ethnography has also been used to synthesize quantitative data as well. Booker (2010) used meta-ethnography to synthesize both qualitative and quantitative data on racial or ethnic matching of clients with clinicians.
Meta-ethnography involves a three-step process. In the first step, studies are analyzed "for common and recurring concepts" (Britten, Campbell, Pope, Donovan, Morgan & Pill, 2002, p. 211). This step was accomplished through original first-stage coding using the qualitative data analysis software ATLAS.ti. Although ATLAS.ti provides a text search tool for counting the number of instances a term appears in an article, it was not used in this research. Using such a tool merely results in quantitative counting of qualitative research. As Silver and Lewins (2014) pointed out, the work of much qualitative research is about unpacking the meaning of texts in terms of respondents' experiences, attitudes, and beliefs rather than concentrating explicitly on the language used; the overuse of text search tools in an interpretive approach may be a red herring, perhaps over-simplifying or skirting over complex undercurrents and nuance. (p. 150) Using a simple text search tool will also miss alternate uses of a term. Instead, all 51 articles were painstakingly coded on a line by line basis.

Getting Started In ATLAS.ti
The first step in using ATLAS.ti to conduct a systematic review is to load into the software all the documents that are to be included in the systematic review. ATLAS.ti accepts many different file formats including text files, PDFs, images, Excel tables, as well as audio and video files. Since the research method was a systematic review, all documents uploaded into ATLAS.ti were PDFs of peer-reviewed journal articles. In total, 51 articles in PDF format were uploaded into ATLAS.ti.
Loading ("assigning") documents into ATLAS.ti is a pain free process. In fact, if you have all the documents in a single folder, they can all be assigned at once. Assigned documents in ATLAS.ti are called Primary Documents (or P-Docs). After opening the program, you simply select Project / Add Documents from the menu. After assigning all your documents, make sure to create a copy bundle file to save all your documents as a single project. Figure 1: A coded PDF document in ATLAS.ti

ATLAS.ti First Stage Coding
Open or in vivo coding was used in the first stage of analysis as it allowed small segments of data to be considered in detail and compared with one another (Silver & Lewins, 2014). First stage codes could represent theoretical concepts, or be completely practical and descriptive, or merely represent ideas that were interesting or needed to be considered at a later stage of analysis. Any line of data which could be important or relevant was coded in this first stage of analysis. First stage coding results in mere indexing -a "first attempt at systematically gathering together segments that are instances of the aspects of interest to the research" (Silver & Lewins, 2014, p. 189).
In ATLAS.ti you can simply highlight a section of text for use as in vivo code. This is very useful at the beginning stages of coding. ATLAS.ti has a maximum character length of 40 characters for an in vivo code; however, longer codes are possible by dragging and dropping the text using the Code Manager (Friese, 2014). This was not an issue as none of the in vivo codes exceeded this limit. As coding continues, certain codes will become heavily used, and it is easier to select from the code list presented in the Code Manager. Simply drag and drop the desired code near the highlighted text. This will associate the highlighted text with that particular code. Proper concept building even in first stage coding will assist with 6

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
second and third stage coding. Too many in vivo codes will make later data analysis efforts a nightmare.
Friese (2014) suggests not exceeding 300 in-vivo codes during first stage coding.

ATLAS.ti Second Stage Coding
In the second stage of analysis, similar codes are grouped together, merged into higher order categories, and continually compared and revisited (Silver & Lewins, 2014). Britten et al. (2002) suggest that Schutz's theory of first and second order constructs be utilized in this stage. "Schutz used the term first order construct to refer to the everyday understandings of ordinary people and the term second order construct to refer to the constructs of the social sciences" (p. 211). Printing out the list of first stage codes assisted with the second stage coding process. Terms of ordinary usage were converted to more usable terms that were prevalent in the evidence base. Codes which were similar or related were merged.
Little-used codes which were dead ends or irrelevant were also eliminated during second stage coding.
The merge code function of ATLAS.ti is extremely useful during second stage coding.
In reviewing first stage codes it often becomes apparent that two or more codes have different names but essentially represent the same concept. To merge codes in ATLAS.ti, one selects the desired code,

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
then right clicks and selects Merge Codes from the menu. All available codes will pop-up. Select the code to be merged and click OK. This will merge the two codes into one code name. It is important to note that all existing comments will be kept under the merged code as will associated text.

ATLAS.ti Third Stage Coding
Selective coding was used during the third stage of analysis. During this stage the data and the codes were revisited, searching for themes, concepts, and relationships (Silver & Lewins, 2014). ATLAS.ti provided the ability to make chains of multiple codes and linking of quotations to create networks which was vital to third stage coding. These network diagrams were qualitative visual representations of the data and greatly assisted with third stage coding. Concepts, themes, and patterns emerged from the network diagrams. Also useful at this stage for identifying patterns in the data was mapping of co-occurring codes. ATLAS.ti allows co-occurring codes to be retrieved and visualized through network and mapping tools. Finally, ATLAS.ti was used to create hyperlinked maps of quotations which assisted with the analysis of proposition three of the dissertation. Once third stage coding was completed, lines of arguments could be expressed for these overarching themes and patterns.

Creating Network Diagrams In ATLAS.ti
The network diagrams that can be created in ATLAS.ti are very useful during third stage coding. Codes can be dragged and dropped and moved around to various locations assisting the researcher in finding patterns and relationships in the data. The result is essentially a concept map that "aid creativity, help in detailing the entire structure of an idea or LOA, and enhance metacognition in the form of elucidating and thinking about knowledge" (Friese, 2014, p. 219). Network diagrams are also helpful for the reader.
Many learners are visual learners and a concept map often can assist with their comprehension of a topic. For the purpose of the systematic review the author wished to discuss five propositions presented in the dissertation. The first two propositions which detailed the relationships of transformational leadership and emotional intelligence to various competencies were more easily explained using the network diagram function of ATLAS.ti.

Figure 3: Network Diagram of Transformational Leadership Competencies in ATLAS.ti
Creating network diagrams in ATLAS.ti is relatively simple but time consuming as one works with all the various permutations that are possible in a diagram. Codes are added and subtracted. Codes are shifted and moved into various positions. Finally, the relationship between the codes must be determined. Although the author did not utilize the color coding feature it also is extremely useful in helping to find patterns and relationships in the data.

Creating Hyperlink Diagrams In ATLAS.ti
Relationships in the data can also be expressed through the creation of hyperlinked quotations in ATLAS.ti. Rather than using a code, an entire quotation can be used to show patterns or themes in the data. Hyperlinking between quotations is a "more flexible way to track varied routes through the data" (Silver & Lewins, 2014, p. 153). There are many similarities between creating a hyperlinked quotation diagram and creating a network diagram view. However, in certain situations it may be more useful to create a hyperlink diagram. This type of diagram works well for relating a smaller number of concepts.
When creating a hyperlink diagram one quotation, the core concept, is the starting point for the diagram. You then link other quotations to this core concept. The relationships can be displayed in various ways such as a star or a chain. The author used a hyperlink diagram to show the relationship of change leadership to various methods of initiating a change initiative in a hyperlinked diagram. Because there were fewer concepts than in the two initial dissertation propositions this type of diagram worked well.
The quotations were able to explain and describe the various change techniques and their use in change leadership.

Creating Co-occurrence Coefficient Tables In ATLAS.ti
Another method of examining patterns or relationships in the data is through the use of Code Co-occurrence tables in ATLAS.ti. The frequency of co-occurrence determines the strength of the relationship between various codes. ATLAS.ti presents two numbers in a code co-occurrence table. The first number is the total number of co-occurrences. The more important number is the second number which presents the c-coefficient. This number is similar to a correlation coefficient in statistics, without obtaining a p-value. It is a method of performing quantitative analysis on qualitative data. The value of the c-coefficient is between zero and one. The closer the number is to one, the stronger the relationship between the codes. Creating code co-efficient tables can be a daunting task if the researcher has many codes and is not sure which ones are closely related. It is best to wait until relationships between concepts have been clearly established before creating code co-efficient tables. The researcher should be prepared for surprises as concepts which appeared to be closely related may not be born out through strong c-coefficient scores.
Depending on the way the data is coded, a score higher than one is also possible. ATLAS.ti marks such instances with a red circle. The researcher then needs to double check the codings. The CODINGS ANALYZER under the TOOLS menu can help to identify redundant codings.
A Code Coefficient

Conclusion
The final step in conducting a systematic review is to write the report, or as here a dissertation. The writing stage is the longest stage in producing a systematic review. However, the judicious use of qualitative data analysis software like ATLAS.ti can greatly assist with this step. While coding, creating diagrams and code co-occurrence tables took about three weeks, this early work greatly reduced the time spent on writing the analysis and discussion chapter of the dissertation. ATLAS.ti also was instrumental in the writing of the "Conclusions" and "Implications" chapters by assisting with the creation of leadership megacompetencies and suggestions for models for implementing change in academic libraries. The time spent on data analysis in ATLAS.ti greatly reduced the writing effort for these two chapters. Trying to organize 13

USING ATLAS.TI TO FACILITATE DATA ANALYSIS FOR A SYSTEMATIC REVIEW OF LEADERSHIP COMPETENCIES IN THE COMPLETION OF A DOCTORAL DISSERTATION
one's thoughts through the use of note cards would have been a herculean task. The data analysis abilities of ATLAS.ti greatly simplified the writing process-an advantage of CAQDAS that is not always pointed out.