Methods
- Methods Building off Research Questions.
- Research Site: Archive of Our Own.
- Computational Data Collection Methods: AO3 Data.
- Qualitative Data Collection Methods: Interview Data.
- Computational Data Analysis Methods.
- Member Checking.
- Qualitative Data Analysis: Qualitative Coding.
In Cushman, Falconer, and Juzwik’s (2018) editor’s note in their final issue of the Research in the Teaching of English, they advocate for a “pluriversality in methodology” that engages the generalizability of corpus-based research and much more contextualized qualitative research. They argue, “corpus studies offers both a generalizable and particularized perspective on learning. For all they offer, though, they provide fewer insights into the social workings of literacies in terms of power, value, structures, and ideologies” (p. 535). Cushman, Falconer, and Juzwik’s critique informs this study. The methods used in this project draw on both qualitative and computational methods found in digital humanities and writing studies. This mixed methods approach was chosen because they provide complementary perspectives on patterns across a fandom as well as individuals’ choices that either mirror or resist these patterns, which a large dataset analysis alone cannot provide.
By triangulating both the results from a large corpus analysis along with interview data from individual writers in the corpus, I describe fan uptakes and generic conventions. I rely on the interviewed fans’ descriptions of their rhetorical choices and generic actions and larger patterns in the genres based on computational analysis. Finally, I draw comparisons between the computational analysis and the fans authors’ interviews to trace case studies based on particular themes. The larger patterns from the quantitative data demonstrate normative conventions within each fandom, while the qualitative data reveals how individuals either conform to or resist these normative conventions.
To read more about the feminist ethics that drove my research, everything from data collection to publishing about fan authors, see the “Research Ethics” portion of this site.
Methods Building off Research Questions
The four research questions that animate this study were answered by analyzing fans’ tagging practices, and interviewing fans about their rhetorical choices, and tracing individual and collective genres and uptakes based on this data.
RQ1How can the resistance to or the reinforcement of cultural ideologies be traced in fanfiction genres?
To answer this question, I used Natural Language Processing (NLP), word embedding models, and metadata visualizations based on fanfiction metadata. The data I collected are uptake artifacts (Dryer, 2016), or the fanfiction texts themselves. I collected these fanfiction texts and metadata from Archive of Our Own, one of the current largest fanfiction publishing platforms in 2020 with over five million fanfictions published. I incorporated computational analysis methods to explore patterns across the fanfiction texts and their metadata. I examined tagging practices; linguistic patterns that revolve around power, normativity, and identity; and patterns that mirror or push against the source text.
Documentation about the computational process can be found in the “Computational Essays” portion of the CFT. Results can be found across the CFT, but especially in the “Fandoms by Numbers” section along with several of the “Critical Fan Case Studies."
RQ2How can critical fandom practices be traced in fans’ uptakes of the original cultural material and fanfiction genres?
To answer this question, I conducted six discourse-based interviews of fan authors who published in the fanfiction corpus, asking them to reflect on particular choices as well as larger community expectations and the ideologies woven within the expectations. I chose writers who demonstrated critical fan uptakes in their fanfictions, whether this approach was in their tagging choices or the linguistic patterns in their texts. I then qualitatively coded these interviews to capture patterns around the writing process, fans’ understandings of generic conventions and expected uptakes in fandoms, fans’ commentary on the canon text, and fans’ analysis of ideologies within fan communities. I framed data analysis around uptake enactment (Dryer, 2016), or the action of responding to one genre with another expected genre. These expectations are determined contextually.
In fan communities, there are expected fan uptakes within the community that can be traced through fanfiction, discussions, and other forms of fan writing. Full description of this process and the codes can be found in the “Qualitative Coding” section of the CFT. Interview transcripts and further results can be found in the “Interviews with Critical Fans” portion of the CFT.
RQ3How can mixed methods help reveal the complex and contextual interactions in fanfiction texts between individual authors’ uptakes and fan genres and politics?
To answer this question, I triangulated data from the above two questions to trace case studies from each fandom. I examine larger trends across fandoms using both the metadata and the actual fanfiction. I also bring in fan authors’ voices to demonstrate how they interpret these patterns, how these patterns are tied to fanfiction genre conventions, and how their individual uptakes resist or reinscribe these conventions. For The Legend of Korra, I trace how fans resist heteronormativity through their critical uptakes even before the show does. For Game of Thrones, I demonstrate how fans reinscribe the white supremacy found in the show. I also show how a few fans, including one who I interviewed, critically uptake one of the few Black characters on the show as a response to this white supremacy. I also examine how one fan enacts a critical disruptake (Dryer, 2016) to critique GOT’s and the general racism while also inviting more fanfictions about Black characters.
These pieces can be found in the “Critical Fan Case Studies” section under “Missandei Deserves Better” and “Ship is Canon!”
RQ4How can critical fan pedagogies be made useable and accessible for instructors and fans alike?
The answer to this question resides more in the digital publication format of the dissertation, rather than specific data or analysis. The CFT is built to be an open-access, accessible, and public project for fans, scholars, and instructors alike. The CFT invites fans who are interested in implementing critical fan practices into their fandom participations and instructors who are interested in integrating critical fan pedagogies into their classrooms. As I built the CFT, I followed User Experience (UX) best practices guidelines, including accessibility, documentation, search engine optimization (SEO), and sustainability.
Research Site: Archive of Our Own
One of the largest and most popular websites to publish fanfiction is Archive of Our Own (AO3). Since 2009, when the AO3 beta first launched, over seven million works have been published. While there are other fan publishing websites out there such as Wattpad, I focus on AO3 for several reasons. First, AO3 received a Hugo Award for Best Related Work, meaning every work published on AO3 received this Hugo Award (Romano, 2019). Second, AO3 belongs to the Organization of Transformative Works (OTW), a larger organization dedicated to protecting, preserving, and strengthening fan cultures. OTW has several projects, including Transformative Works and Cultures — a peer-reviewed open-access journal for fan scholars to share their research — and Fanlore, a large fandom wiki. Third, AO3 is well-known for its information system that was designed by and for fans (Dalton, 2012; Fiesler, Morrison, Bruckman, 2016).
Fan studies scholars study sites like AO3 for various purposes, including examining the structure of its information system as an example of feminist human-computer interaction (Fiesler, Morrison, & Buckman, 2016), tracing how fans use tagging practices to shape community (Dym, Brubaker, & Fiesler, 2018; Messina, 2019; Price & Robinson, 2021), and its relationship with academic scholarship through the OTW. In fact, several members on the current OTW board are academics — such as Kristina Busse — who write fan scholarship and are members of fan communities.
The first reason I chose AO3 is the actual design of its information system is one that reflects community values. Fiesler, Morrison, and Buckman (2016) build upon Bardzell’s (2010) concept of “feminist Human-Computer Interaction (HCI)” to demonstrate how AO3 is an example of feminist HCI. By merging feminist praxis and user experience, Bardzell (2010) defines Feminist HCI as a:
Constellation of qualities—all of them appearing together in a critical mass—that I argue characterizes feminist interaction. The qualities I propose as a starting point are as follows: pluralism, participation, advocacy, ecology, embodiment, and self-disclosure (p. 1305).
Fiesler, Morrison, and Buckman (2016) use Bardzell’s proposed Feminist HCI qualities to analyze AO3 as an exemplar Feminist HCI platform. AO3, they argue, is built upon a particular value system, and therefore the literal system itself reifies those values. These values include diversity, inclusivity, agency, and empowerment; the system builds upon these values with similar qualities to those Bezdell proposes in Feminist HCI. AO3 is already committed to feminist values. These values, too, are interwoven across the back-end design and fanfiction authors’ and readers’ approaches to using the platform.
AO3’s tagging system also provides ways to trace and define communities. I trace how fans use tagging and metadata practices to create, shape, and reshape their communities. Gursoy, Wickett, & Feinberg (2018) refer to AO3 as a “user-generated metadata ecosystem” in which fanfiction authors both self-select tags and validate other fans’ tags. Price and Robinson (2016) provide a framework for analyzing AO3 community practices from an Information Systems perspective, specifically examining the practices fan community members incorporate to “consume, create, remediate, disseminate, promote, describe, access, and preserve” cultural products (p. 650). They use the Delphi study method to capture fans’ perspectives of online fan community and genre practices. However, they do not engage with the actual metadata from AO3. Other studies conduct analyses of actual metadata and tagging practices used in AO3 and other online fandom spaces (Dalton, 2012; Dym, Brubaker, & Fiesler, 2018; Gursoy, Wickett, & Feinberg, 2018; Messina, 2019; Price & Robinson, 2021; Pianzola, Acerbi, and Rebora, 2020). For instance, Messina (2019) also demonstrates how particular fan communities resist heteronormativity both in their metadata practices and their actual fanfiction texts. Pianzola, Acerbi, and Rebora (2020) use multiple points of metadata, including Kudos and comments, to examine accumulation —rising amounts— and improvement —better reader engagement— in an AO3 fandom.
In addition to fan tagging, AO3 has “tag-wranglers,” who are volunteer fans who regulate metadata tags that authors choose for their fanfictions to make sure their fanfics can be discovered. They preserve more unique tags, but regulate more popular tags, such as “Angst,” as well as relationship and character names. For example, in GOT, the character tag “Missandei” may be used. However, because the tagging system for fan authors is free form with suggested tags, fans may misspell or use the wrong character tag, making their fanfic harder to discover. The tag wrangler’s job, then, is to go through tags and check for differences to then standardize certain tags.
AO3’s design and fans’ metadata practices allow for researchers to trace fans’ community formation and reshaping practices. AO3 users often taken on multiple roles: the reader, the writer, the critic, and the community-builder. AO3 users use metadata practices to both find and carve out community, discover fanfictions that they want to read, and/or reach potential readers using metadata for their fanfictions.
Why The Legend of Korra & Game of Thrones?
Fan studies scholars describe fanfiction as transformative writing practices in which fans carve out their interpretations and reimaginings of the material; usually those writing fanfiction are women, people of color, and queer people (Russ, 1985; Lamb & Veith, 1986; Jenkins, 1992 & 2008; Summers, 2010; carrington, 2013; Potts, 2015; Thomas & Stornaiuolo, 2016). In Textual Poachers, Jenkins theorizes why fans choose to participate in particular fandoms:
Fans have chosen these media products from the total range of available texts precisely because they seem to hold special potential as vehicle for expressing the fans’ pre-existing social commitments and cultural interests; there is already some degree of compatibility between the ideological construction of the text and the ideological commitments of the fans. (p 30)
Jenkins acknowledges that fans’ “pre-existing social commitments and cultural interests” impacts the source texts fans invest in.
Within AO3, I focus on two fandom communities — The Legend of Korra (TLOK) and Game of Thrones (GOT). I chose these two fandom communities because of the “ideological construction of the text[s],” are stark contrasts of one another, which provides more breadth and better comparisons when studying fans’ uptakes through computational analysis and interviews. The cultural ideologies in TLOK challenge white supremacist, heteronormative, misogynist, and ableist tropes, demonstrating how to represent all types of bodies and identities. Meanwhile, GOT has been heavily critiqued for its racism and misogyny, particularly the constant violence done upon women and characters of color (Florini, 2019). What makes GOT a fascinating case study in opposition with TLOK is that GOT has been claimed as revolutionary by breaking generic boundaries in speculative fiction and television shows, but the ideologies embedded in the show — unlike TLOK — are the complete opposite of revolutionary.
TLOK is a young adult show on Nickelodeon that aired from 2012–2014; TLOK is a sequel to the popular children’s show Avatar: The Last Airbender. What makes this show special is that the creators of the show were already resisting harmful dominant ideologies, such as ableism and white supremacy. TLOK already breaks generic conventions and demonstrates critical genre practices that subvert systems of power and oppression, especially around its representations of diverse races and sexualities. Ravynn Stringfield (2020) in her beautiful piece on the meaning of Korra’s character for her, writes:
Few people in Korra’s life on screen or in the Avatar fan base extend her the courtesy of receiving her exactly as she is: flawed and struggling. And that is a direct result of the way we as a society view powerful women of color, particularly Black women. Korra is not Black—Water Tribe members are based in part on Intuit cultures—but Black women can see so much of their struggle in how she moves through the world.
While Korra is not Black like Stringfield herself, Stringfield sees herself in Korra. TLOK was not set on creating a perfect woman of color hero, but rather a “flawed and struggling” hero who must be careful “in how she moves through the world.”
As Ebony Elizabeth Thomas (2019) and so many other cultural and fan studies scholars have argued, mainstream popular culture’s representation of people of color is limited. The “imagination gap” Thomas (2019) describes — or the ways in which science and speculative fiction often represent the same groups of (White) people — demonstrates creators’ and producers’ lack of imagination in character representation because characters of color or other marginalized characters may be “unlikeable” to the larger public. The public, in this case, is coded as white, cisgender viewers, completely ignoring the viewers of color, queer viewers, viewers from other marginalized groups, and White viewers who are invested in combating white supremacy.
TLOK bridges this imagination gap by having the main character in a cartoon fantasy series be a powerful and vulnerable woman of color — Korra — who in the end is confirmed bisexual when she begins a romantic relationship Asami, another woman. The show already invites fans to challenge dominant ideologies around gender, race, and sexuality: the ‘happy ending’ is not heteronormative, but rather suggestive of an adventure to come, a grand vacation, and a new love that breaks boundaries.
While TLOK is a demonstration of creators pushing against exclusive cultural ideologies, GOT — an HBO series which aired from 2011–2019 — seems to thrive in racist and misogynistic portrayals of women and characters of color. The show is based on George R. R. Martin’s Game of Thrones series, which has yet to publish its final books. The series finale recently aired on May 19, 2019, wrapping up the long, bloody, and violent story with more violence and blood. Currently, in early 2021, a Change.org petition for a season 8 rewrite has over 1.8 million signatures. Of course, season 8 will not be rewritten — but this is exactly why fanfiction exists. GOT depicts people of color as nomadic savages, a White woman as their savior, women being often raped or losing control over their bodies, queer folks being punished for existing, and other harmful stereotypes that perpetuate dominant ideologies.
There are some parts of GOT, like most shows, that challenge harmful ideologies. For instance, Sansa — one of the main White women leads — has an incredible character arc from being obsessed with the idea of marrying a prince to leading an entire nation without a partner. In the early season, Sansa recites stories about young princes saving damsels in distress. She performs within the expectations of her gender role, learning how to craft and other feminine hobbies in hopes of marrying a prince. Her hopes resonate with young girls across America and beyond who are taught our value is always placed in men and romance. Yet, her desires lead to violence, as she almost marries a prince who murders her father and abuses her. She realizes her dreams do not align with reality, and she grows to become independent and powerful. Her feminist arc resonates with me and other feminist-minded viewers, as also shown in the staggering number of Sansa-centered fanfictions published on AO3.
While Sansa’s arc may be read as feminist, the show still reifies white supremacy, demonstrating that feminism and antiracism are not inherently interlocked. Missandei and Grey Worm are some of the few consistent characters of color on the show. They start a romance, but tragically — and unsurprisingly — Missandei is killed off in the last season, preventing Missandei and Grey Worm from having the same happy-ish ending that so many White characters are granted. As Ebony Elizabeth Thomas (2018) argues, “We [Black people] always die in other people's fantasies. (Other people seem to fantasize about Black death. Endlessly. Often needlessly.).” Missandei’s death mainly feels like a plot device to further one of the White main characters’ character arch. None of the time in the show is dedicated to Grey Worm’s grief after losing the love of his life; instead the show creators chose to show his grief by having him murder (White) characters during a war scene, feeding into dangerous narrative that (White) people should fear people of color. The choice to kill Missandei is one that not only broke viewers’ hearts, but reminded viewers that Black characters are expendable, especially in epic (White) fantasies; fanfiction writers show, though, that Missandei and other characters (people) of color are not expendable. As Thomas (2019) says, “Missandei, like us, lives in the wake.”
By examining both TLOK and GOT, patterns across each fandom reveal fans’ political and ideological commitments as well as how these commitments seep into or inform fans’ uptakes. Comparing fandoms from very different source texts provides a method for better understanding how fans construct their communities, critique or reinscribe dominant ideologies, and how individual fans critically uptake both fan genres and the source text genres.
Computational Data Collection Methods: AO3 Data
Textual data and the metadata data were collected from Archive of Our Own using a third party web scraping Python script created by Jingyi Li and @ssterman. I collected the data with the permission of Transformative Works and Cultures. Each fanfiction has its textual data, which is the actual story published, and metadata about the fanfiction. Metadata are data about data, or in this case the information about each fanfiction. The visualizations in the “Fandom by Numbers” section, for example, are mainly using the metadata from TLOK and GOT fanfictions.
Table 2.1 describes the data collected using this web-scraper and is broken into three columns: the name of the data, how this data was created, and the description of these data. The data creator is either an automatically-assigned data, such as the publication date or the number of comments on a fanfic, or the fan author, which is data written or selected by the fan author. I have also included the column “data type” that explains what the data looks like and if the data is standardized or not.
Data Name | Data Creator | Data Type | Description |
---|---|---|---|
Publication date | Automatic | Date, standardized. | The date that the fanfiction was published. There is also a “last updated” date, but I do not use this column for this project. |
Relationship categories | Fan author | List of words, standardized. | Six categories of relationship types based on gender. Fan authors can only select out of these six categories: female/female, female/male, male/male, gen (no pairings), other, or multiple. |
Relationships | Fan author | List of words, community-standardized by sometimes unique. | Fan authors choose which characters appear in relationships together and what types of relationships. Romantic pairings have a slash (/) between characters’ names, while platonic pairings have an ampersand (&) between characters’ names. Fan authors are given the option to choose from popular relationships that other fan authors use, or they can create their own. |
Characters | Fan author | List of words, community-standardized by sometimes unique. | Fan authors choose which characters appear in their fanfic. Fan authors are given the option to choose from popular characters’ names or they can write their own in. This choice allows for fan authors to either follow the controlled vocabulary determined by the community or choose to step outside of community norms. |
Additional Tags | Fan author | List of words, community-standardized by sometimes unique. | Fan authors choose their own tags to signal more information about their fanfic. There are tags that are often used across AO3, such as “angst” or “fluff;” and there are tags often used in particular fandoms, such as “korrasami freeform” or “ R Plus L Equals J.” Fans can also write their own unique additional tags. |
Body | Fan author | Text, unique. | The entire body of fanfiction text. |
For the approximately 30,000 fanfics I collected for this project, each contains all of this data. Some fanfic authors choose not to include certain metadata, but for the most part, all the fanfics in this corpus have this metadata and the actual fanfic texts. The data is not perfect, as there are some data (relationships, characters, and additional tags) that are not perfectly standardized. For example, fanfiction authors may have tagged Korra and Asami’s relationship as “Korra/Asami,” “Korra/Asami Sato,” or “Korrasami.” For the most part, AO3’s tag wranglers adjust deviations to standardize the metadata. I also attempted to standardize some of the deviant data and combined the results. I preserved novel tags, as often fans will use “Additional Tags” in particular to directly address their audience or potentially critique something in the fandom.
Qualitative data collection methods: Interview data
As Kurtyka (2015) argues, centering emotion is crucial when analyzing individuals’ genre performances and uptakes because the genres that “stick” and the genres we want to participate in are often tied to our emotional responses and positionalities. Fandoms are spaces that are carved out based on affective responses to a source text. Whether someone loves or hates a source text, their affective response drives them to a particular fandom. Affective responses and performing this affective response is tied to an individual’s positionality and aspects of their identity, as well.
Importantly, I wanted to incorporate fans’ voices and provide a space for them to share their perspectives, motivations, process, and analysis of the canon and fandom. I chose specific fanfiction authors from either the GOT or TLOK fandom based on their tagging practices and the ideological values replicated in their works. Their perspectives of both their own writing process as well as larger community genre conventions provides an important lens into the three different cases. The TLOK fan authors speak to the second and third cases, while the GOT fan authors speak to the first and third cases. Without their important voices, this project would have overlooked the enactors of fan genres and uptakes: the fans!
For the interview recruitment process, I wanted to have an equal number of fan authors from both TLOK and GOT. In order to find potential interview participants, I analyzed the larger TLOK and GOT fanfiction corpora I collected to narrow down my options. Specifically, I chose pieces that reflected commitments to anti-racism, disability representation, and representations of queer sexualities/genders.
For GOT, I searched for GOT fanfiction that used tags such as “Missandei deserves better,” “pov character of color,” “genderfluid character,” and “alternate universe - race changes.” For TLOK, I chose TLOK fan authors who wrote stories about Korrasami and found texts that explicitly mentioned gender oppression or trans characters. More information can be found in the “Ship is Canon!” case study.
I reached out to TLOK and GOT fan authors by leaving a comment on their AO3 stories, as this is the only way to currently communicate on AO3. I invited them to fill out a consent form to have their fanfiction used in my dissertation and asked if they were interested in a further interview ( see the interview recruitment forms). I used a purposive sampling approach to select the fan authors to interview based on their different gender, sexuality, and racial/ethnic backgrounds as well as the fanfiction texts they wrote. I was particularly interested in how fans saw themselves in either the canon text, the fandom community, or the fanfiction they wrote.
Once fan authors agreed to be interviewed, I forwarded them interview questions so they could prepare their answers for our video conference. These interview questions (see the interview questions template here) were separated into three sections:
- History: The fan authors’ self-identification and relationship with fanfiction,
- Discourse-based interview questions: The fan authors describe their process writing the TLOK/GOT fanfic,
- Corpus analysis: The fan authors’ interpretation of larger patterns within the fan community based on preliminary corpus analysis results.
In the discourse-based interview portion of the interview, I designed questions based on discourse-based interview (DBI) methods, in which interview participants talk through their writing choices (Lancaster, 2016). I provided fan authors with a set of more general questions to talk through their writing process, and then pointed to specific moments in their texts and asked them to reflect on those moments. For the corpus analysis portion of the interview, I built the questions based on Lancaster’s (2016) DBI approach in which he incorporates large corpus analysis within the interviews, asking writers to reflect on their own choices in comparison with larger trends in a corpus of similar writing. For each author, I created tables and visualizations of trends across the fandom they wrote within and asked them to reflect on these patterns. For instance, I showed Aria results from a word embedding model and asked her to talk through her interpretation of these results.
Each interview took about one hour and I audio-recorded the interview to then have it transcribed. Each interview participant received a gift card and I paid for a transcription service, funded by the NULab Seedling Grant. I then qualitatively coded the interviews, which you can read more about the process in “Qualitative Coding and Codebook,” based on my theoretical framework and patterns I observed during the interviews and in my listening after the interviews.
Computational Data Analysis Methods
To examine the AO3 data, I used computational text analysis methods (Natural Language Processing and word embedding models), data visualizations, and network analysis. I use computational text analysis methods on the actual fanfiction texts; I use network analysis and visualizations on the corpora metadata. To get a full understand of the methods I use, I invite you to look at the “Computational Essays,” which break down my data transformation process and document each step I took to transform, analyze, and visualize data.
Computational Text Analysis
I use the term “computational analysis” instead of “distant reading,” a one common term for analyzing a large corpus of humanistic data. Distant reading, as Lauren Klein (2018) argues is a term coined both by well-known sexual assailant Franco Moretti and structurally continues to erase gender and race from analysis. Klein argues distant reading can be reclaimed by researchers to actively center race and gender; she points to the need for using more diverse corpora as well as better models. I use the term “computational analysis” not only to avoid the messy history of distant reading, but also recognize that there is nothing distant about this analysis. Even though the I trace larger trends and patterns across the corpora, my understanding of and relationship to this corpus are far from distant—they are intimate. Examining larger patterns across a corpus still requires contextual knowledge about said corpus.
Computational analysis is also broad enough of a term to invite all different types of analysis, mixing metadata and data together for analysis. I specifically pair Natural Language Processing, including machine learning methods like word embedding models. I turn to Klein (2020), Quinn (2020), and Nelson (Forthcoming) who use similar methods to break down the close/distant reading binaries. For each author, they are invested in transcending notions of scale, instead using their findings to inform other methods of analysis. Nelson (Forthcoming) argues that machine learning can allow for more intersectional research. She creates word embedding models with race and gender variables encoded; along with these models, she enacts “a complementary close reading” to “place…the word vectors back into their full discursive context to provide both qualitative validity and depth to the analysis” (p. 11). Quinn (2020) also uses topic modeling, moving back and forth between the model findings to demonstrate how these findings also manifest on the textual level. Klein (2020) also uses topic modeling and statistical analysis to “surface” the hidden labor in publishing. Each scholar moves between the results from their models to the texts to better situate how different writers’ positionalities impact the discursive space of the corpora with which they work.
I specifically use Natural Language Processing (NLP). NLP merges linguistics and computer and data science to trace linguistic patterns across a corpus. NLP is particularly popular in cultural analytics and writing analytics, both disciplines which trace linguist patterns within particular cultural and community contexts (Aull, 2017; Klein, 2020; Quinn, 2020; Soni, Klein, & Einstein, 2021; Nelson, Forthcoming). NLP is one of the most well-used forms of analyzing textual data and offers several different approaches for analysis, such as basic word and nGram counts to parts-of-speech analysis to more advanced machine learning techniques like word embedding models.
Because I approach this project with a variety of data—from the data collected on AO3 to fan author interviews to individual fanfiction texts—I incorporate a variety of methods. I use nGrams to measure which characters are more frequently used, comparing the results to the “Character” Tags. I also use concordances to reveal the context within which fanfiction authors use particular words. Finally, I use word embedding models to trace changes across time to unveil how fans’ linguistic practices mirror or resist dominant ideologies, especially around representations of race, gender, and sexuality.
Data Visualizations
Data visualizations are an important aspect of this project. As Engebretsen and Kennedy (2020) define data visualizations as “discursive resources used in the dissemination of statistical information and often numerical data” as well as “abstractions and reductions of the world, the result of human choices, social conventions, and technological processes and affordances, relating to generating, filtering, analysing, selecting, visualizing, and presenting data” (p. 22). Data visualizations are models designed to select, visualize, and present data. More importantly, too, data visualizations are forms of storytelling and composing that can be “make data accessible to publics” (p. 20). One of the goals of the CFT is to make findings and analysis accessible to all different audiences, including fans. To include visualizations, especially interactive visualizations, then invites different types of readers to learn and explore.
The “Fandoms by Numbers” storyboards are a series of visualizations paired with an analysis of The Legend of Korra and Game of Thrones fanfiction metadata. I used Python to transform the data, which you can read more about in the “Computational Essays.” These essays include explanations for why I chose particular metadata as well as explorations of the data transformations and findings. Then, I fed that transformed data into Tableau to create the interactive visualizations for both TLOK and GOT.
One of the affordances of working with fanfiction about television is tracing changes across time within the corpora and comparing these changes to both the source text and other times within the corpora. For instance, one visualization traces character tags used in GOT across all 8 seasons, showing which characters stayed consistently popular and which characters fell out of favor. This temporal analysis shows how trends change across time, especially how these trends are impacted by the source text as new material from each season is released.
Along with visualizations—mainly bar graphs and line graphs—about metadata, I also include a network analysis graph of “Additional Tags” in TLOK. Network analysis “makes it possible, with relative ease and speed, to measure the relationships between many entities in multiple ways, allowing a rich, multidimensional reading of complex systems never possible before” (Ahnert et. al, 2020, p. 5). This network analysis visualization of the Additional Tags show the sub-communities fans create by citing the generic forms they are working within and providing information about the content that appears in their fanfictions to reach their audience.
Member Checking
For the AO3 large datasets, member checking came in two ways: first, with my own experience as a fanfiction author and reader, and second with the fans’ who I interviewed interpretations of the data. Because I have been involved in fanfiction communities since I was a pre-teenager, I could tell which general trends and patterns reflected my own experience in the fandom. For instance, in the network graph in the “Fandom by Numbers” homepage, the results not only reflected my experience in TLOK fandom on AO3, but larger AO3 trends, such as “Angst” and “Fluff” being centered. During the interviews, I also invited fans to analyze some of the findings from the data. Most just agreed that the data reflected their own experiences in the fandom. Some pointed out surprising findings, but then explained how they had seen those patterns. To read about each of the fan author interviewee’s interpretations of the data, visit the 'Interviews with Fans' section and read their individual interview transcripts; the discussions about data are usually towards the bottom of each transcript.
To member-check the findings and analysis of the interviews, I invited every fanfiction author who was mentioned in the CFT — whether I interviewed them or mentioned their work — to look over their transcripts, my qualitative coding, and my analysis. I sent a series of emails, inviting them to look at different aspects of the CFT. Some interviewees did, while others did not. One person even caught a mistake in one of the case studies, pointing out I used the word “Meereen” when I meant “Essos” in a GOT reference. For every update with the project, I email the fanfiction authors and invite their thoughts. In future iterations of this project, I hope to include inter-rater reliability for qualitative coding.
Finally, I offer a “User Experience” feedback survey that invites feedback from CFT users. Even though I am not technically studying UX, one of the goals of the toolkit is to provide an open-access, easy-to-navigate experience. This means I want to receive any feedback from everyday users, whether they are fans, were mentioned in the CFT, or are scholars.
Qualitative Data Analysis: Qualitative Coding
Please visit the "Qualitative Coding: Description and Process" portion of the CFT to learn more about my qualitative data analysis methods.