(Re)Inventing the Screen and New Media

Lev Manovich’s (2001) The Language of New Media and Janet Murray’s (2011) Inventing the Medium provide an understanding of computation and new media and their effects on contemporary culture. Manovich’s (2001) five defining characteristics of new media are numerical representation, modularity, automation, variability, and transcoding. These characteristics are envisioned as a direct result of new technologies such as the digital computer, the gramophone, the photographic plate, or the film stock. Manovich wrote that the result of these new material forms are “graphics, moving images, sounds, shapes, spaces, and texts that have become computable; that is, they comprise simply another set of computer data” (p. 20). Similarly, Murray (2011) defined the affordances of the digital environment as procedural (following rules), participatory (allowing for human interaction), encyclopedic (storing large amounts of information), and spatial (structuring information so that it is navigable). From these lines of thought, we are led to believe that computation and digital environments are an exclusive coupling, with effects seen in our field through a focus on computers and composition.

We align with the premise that composition is a kind of data visualization that bears traces and threads of the lines and relations that come to produce them. As Tim Ingold (2016) wrote, “it was not enough to focus only on the lines themselves, or the hands that produced them. I had also to consider the relation between lines and the surfaces on which they are drawn” (p. 2). In this, Ingold asked that we also consider what else it is that is represented because what is capable of being represented is the result of what is disciplined by the surface or, in the case of this conversation, the screen. For example, in Manovich’s (2001) new media heuristic, data visualization is almost exclusively seen in terms of strategies that use existing or new custom-designed software to collect, process, analyze, and visualize data.

Beyond data traces, media also leave other traces. Murray (2011) conceptualized the digital affordances of new media, yet at the same time, Murray used analogies of social interactions that occur outside of new media environments as metaphors for digital environments. Contrary to this, Jay David Bolter and Richard Grusin (1999) and Angela Haas (2007) have asserted that digital media is not something uniquely of its own. Bolter and Grusin (1999) wrote that “no medium today, and certainly no single media event, seems to do its cultural work in isolation from other media, any more than it works in isolation from other social and economic forces” (p. 15). Media are practices that bear traces of other media and other practices. Moreover, Haas (2007) extended this claim to the example of Indigenous craft and technologies: Haas drew a similarity between Hypertext Markup Language (HMTL) as a strand of binary code strung together and the visual rhetoric of the Wampum Treaty Belt in terms of its pattern and bead work as also strung together. In both scenarios, there is a reliance on following procedures and on the artificial intelligence of the surface or the screen to communicate information from the composer to the reader. Despite this, Murray’s insights are still useful in helping us to consider the ways in which old media behave similarly to new media, particularly in terms of its capacity for procedural rhetoric. Thus, even at the stage of conceptualization itself, new media bear traces of all that come to produce it, and the visualization of data itself is a function of the surface, the screen.

We propose here a set of understandings about data and data visualization that push against the overdetermination of the electronic screen in the processing and representation of data.

We argue that 1) data is coded into made things, and 2) those made things are storage media that allow access to a structure of information.

Mai Nou

When I was 13 years old, my mother taught me how to make traditional Hmong patterns through cross-stitching. My mother was pregnant with my baby sister and was using this time to make my sister and me each a proper Hmong garment, which would be worn during New Year festivals, weddings—and, later, at our funeral. The spirits would not recognize us without them. Although it was a serious project, I came to love this time as ua paj ntaub—measuring each fiber through touch, counting each step, calculating each move, checking for balanced shapes, colors, and lines. I relished seeing my expertise show up on the textile and being able to undo mistakes. Most of all, I enjoyed the presence of my mother—her scent, her voice. And each time I pick up a needle, I pick up on this moment and our time together.

My mother had not seen her family since we immigrated to the United States. Her family remained in the Ban Vinai refugee camp for over a decade. She was finally able to make the trip to go see them when my sister was 3 years old—but it was to bury her father. Amidst it all, she purchased me an indigo silk skirt, a gold necklace, and a pink comb. So, when I started making this project, these memories unknowingly materialized into me using saliva to thread my needle, into the color choices, into the initial design choices, and into the subsequent design choices. I found myself inside the stories my mother told as we composed our work. I was inside a history of how I know how to do this and in a place with my hands in motion.

Krystin

In my office, a bright blue knitted square sits draped over my chair—two threads twined together, one soft and satiny, one glittery and rough. It is supposed to be a garter stitch, but there are holes all throughout it, dropped stitches and switched-up tension. It looks and feels as if a child had made it for their first swatch, but this was the last thing my grandmother knitted before she was diagnosed with Alzheimer’s, when her hands, eyes, and mind were too far gone to knit anymore. She came from a family of textile workers and was known for being the best knitter in town. We had been working on a blanket together, to stitch together and hand down to my children. When she died, I found the swatch, a prick of a memory of our shared skill.

Mamaw's last swatch surprised me, but it informed me of things that couldn't be said in words. This misshapen thing materialized her—her sickness, health, knowledge, all meeting as one. It aggregated medical data: her dementia, the way her memory fails her body and makes her forget this fundamental movement of weaving yarn into loops along a line. It wove together sentiment data: the sweat and tears in the fibers. It shared geographic data: the places it has traveled to be with ones who are and are no longer here. When I sit with her swatch now, after her death, I hold a body in ever-slowing movement. It collects her hands, her lost and found memories. It follows a protocol that is itself data, the re-programming of a body–mind. It gathers her life together with her death and continues to reflect back the ways she lives on through the things she made.