BETWEEN LINES
Design, Speculation, Envisioning, & Ethics?
SPECULATIVE DESIGN — PUBLICATION DESIGN — FILMMAKING — DESIGN RESEARCH
Post Mortem: Speculative Design Filmmaking
This project combines traditional filmmaking practices with interaction, and product design in order to explore new methods of envisioning and rapidly prototyping possible futures on film, allowing designers to understand and communicate the impact of their work.
Using film, designers can communicate ethical concerns and worries in a compelling, easily digestible form. This project explores the way ordinary experiences with future technologies can communicate broader ethical and technological concerns that may be missed in other forms of risk analysis.
How do designers think about the future?
The future is inherently unstable. Many disciplines prioritize their resources best, orienting themselves to capitalize on the most likely outcome of future events—their speculation taking the form of analysis and trend forecasting. On the other hand, designers are often tasked with realizing a specific vision of the future (their vision or, quite often, their organization leaders' vision). Designers introduce objects, products, and services into the wild, often without thinking through the ethical ramifications. These artifacts often permanently change society (see the AIGA's writing on ontological design).
Small design teams often make countless small decisions while creating a product or service. While some firms have adopted participatory approaches, methods for collaborative futuring remain few and far between. With this thesis project, I simultaneously wanted to expand and iterate on existing methods of critical design speculation (leveraging my background in film production) to question an increasingly relevant and pressing topic: location data usage in recommender systems.
The cone of possible futures refined by Anthony Dunne and Fiona Raby for situating speculative designs along axis of distance and preference.
How We Think About Location Data?
While location data may seem like a niche topic for interaction design research, designers often draw on the power of location-based recommender systems without knowing it.
At their most basic level, location-based recommendation systems are recommender systems that incorporate location information from a mobile device into algorithms to provide users with more relevant recommendations. These are increasingly complex recommendation systems behind the stable interfaces of the Google and Apple Maps, Uber and Lyft apps.
These systems are dynamic, often morphing, considering current and past locations, browsing, and purchasing activity across the web to inform increasingly contextual recommendations — and navigating our bodies through space to get there. If you read the headlines, you know this data is being used for some controversial, if not downright dystopian, stuff:
ICE to pursue privacy approvals related to controversial location data
Automakers Are Sharing Consumers’ Driving Behavior With Insurance Companies
FCC fines AT&T, Sprint, T-Mobile, and Verizon nearly $200 million for sharing access to users’ location data.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored and commercial surveillance systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I asked myself:
How can we use design to anticipate concerning and contentious future scenarios of location-based recommendations?
A still from Routine Repetitions.
Detailed Scenario Development.
Countless questions arose about how to develop the films. In creating scenarios, we decided on a method requiring a strong conceptual composition of critiques. Knowing we would produce multiple films, the team decided they should work together in concert. Each film would address distinct, scenario-specific critical themes without overlap—such as gentrification and corporate-state collaborations in what became Avoidable Inconveniences.
The films were also designed to share common overarching critiques, such as paternalism, surveillance capitalism, and reductions in agency. To achieve a diverse array of films meeting both criteria—distinct scenarios with shared themes—a significant part of the project early on involved internal debates and generating broader, high-level critiques. Details of the plot, specific scenes, and the intricate workings of the technology were not emphasized. Our films were shaped by three permeable categories: Surveillance Capitalism, Agency, and Technological Mediation, inspired by current events, themes in fictional films, and research reports. We drew on prior work and current events to inform the films' design and grounded them with subtle references to key events. For instance, December 26, 4:36 pm, heavily draws on HCI research addressing emotional sensitivity and recommender systems.
Alternatively, system overlays in Avoidable Inconveniences reference recent motions by the Seattle council for CCTV and "shot spotters" in lower-income neighborhoods. In addition to this real-world inspiration, we drew broadly from literature in other academic areas, particularly surveillance studies, cultural critiques of capitalism, and critical technology studies. Key texts included The Relevance of Algorithms, The Filter Bubble, and The Age of Surveillance Capitalism.
With the established conceptual framework—focused on critique, possible implications, and related technologies—we developed detailed scenarios. We initially created twenty scenarios, each with a logline (a one- to two-sentence description of the scenario, setting, and characters) specifying the critiques and trends they addressed. We included a three-frame storyboard for each scenario, using ShotDeck to pull visually and thematically relevant stills from existing films. This approach helped us align the concept, form of communication, and visual tone, combining mood and storyboards to flesh out each idea.These scenarios were then critiqued by designers, fellow researchers, and a subject matter expert. This feedback was crucial in narrowing down the final four scenarios that would be adapted into short films. Our colleagues and experts noted that some scenarios offered "richer" sources of tension or addressed key concerns in their research, guiding us in selecting which concepts to develop further. Scenarios lacking clarity, focus, or critical components were cut. In the downselection process, we balanced the subject matter, the manifestation of location-based recommenders, and the critiques involved. We diagrammed alternative versions through rapid iteration, considering a range of positive and negative implications. We developed detailed plots, scenes, narratives, and character arcs with the refined scenarios. The final four scenarios were Avoidable Inconveniences, December 26th, 4:36 pm, Evidence of Insurability, and Routine Repetitions. With these finalized, we moved into pre-production.
The original 20 scenarios. Each scenario contains three images conveying plot and tone through careful image selection.
Aesthetic and Look Development.
The film's look development process took place during and after the scenarios were created. As a team, we agreed that to achieve high-fidelity fiction films, the production quality needed to meet a certain visual standard. This was partly determined by reviewing stills from related films on platforms like ShotDeck and various commercial concept videos. To achieve this, we adopted a Kodak 2383 Print Film Emulation look, an industry-standard film stock, and filmed in 6K for flexibility in re-framing and stabilization during post-production. Given our selected scenarios and tight timelines, we opted to shoot with a single 35mm f1.4 lens.
A cross section of the four Between Lines films.
Making the Films
Once the scenarios were locked in, I moved on to the pre-production phase. Due to the project's condensed timeline and the number of films I aimed to create, I began a rolling production schedule. For each of the four films, I would write a script on Monday, create a storyboard on Tuesday, and send it to my talent on Wednesday. I would assemble a skeleton crew, including a gaffer, onset sound, and myself (camera operating and directing). Each week (often Thursday and Friday), I would assemble a rough cut of what had been shot the week before, doing a rapid color grade and making other minor adjustments. Occasionally, if I were not able to find help, I would record motion without sound (MOS) adding all audio in post-production. While I sacrificed greatly on lighting quality, camera stabilization, and a layer of refinement that might otherwise have been there, this approach allowed me to be incredibly flexible (shooting alternate versions, coming up with shots on the fly, etc.) in a more documentary style.
Once the films were edited (often a fast one to two-day process), I began to approach screen replacements. Artistically, I was opposed to text overlays on screen. While they provide greater flexibility, I prefer the naturalism and realism of showing UI on-device. This was debated heavily throughout due to the time on-screen text and UI graphics would save. However, I decided that the extra effort would pay off, not breaking the illusion of an alternate reality. To this end, each screen was replaced using Davinci Resolve’s built-in Fusion compositing tools. While I was initially able to replace all of the phone screens, due to some of them looking poorly (often due to difficult tracks and complex rotoscoping work), I pivoted to taking the interface I’d created for the screen replacements and using them in a series of reshoots. I reshot nearly every scene from Avoidable Inconveniences and Evidence of Insurability. In the end, 50% of the interfaces were replaced in post production as originally intended, and 50% were reshot with stand-in actors.
Behind the scenes filming Avoidable Inconveniences.
Pre-Piloting and Piloting
After completing the initial versions of the films, I conducted a preliminary screening with two colleagues who specialize in data and privacy. This was less a traditional pilot and more akin to a test screening aimed at evaluating the clarity of the narrative, the critique being applied, and the depiction of algorithms within each film. The feedback was critical: the narratives lacked clarity, leading to misinterpretations of the plots; the role of recommendation systems was obscured; and the interfaces failed to effectively convey their intended messages.
In response, I revised each film, focusing on strengthening the narratives through strategic edits—including shortening the duration—to enhance narrative clarity. I also reshot and updated the user interfaces, ensuring each was of higher fidelity and more effectively communicated within the story's context.
Upon completing these revisions, I conducted another pilot with a subject matter expert in location data, privacy, and security. The revised films elicited fewer questions about the narratives and the roles of recommendation systems in the story. Instead, they sparked visceral reactions, indicating a high level of engagement with the themes presented. This feedback was encouraging as I proceeded to engage with my main study participants.
Onset filming Evidence of Insurability.
Participants + Recruiting + Study Design
Given the nature of the project and the constraints of a small team and condensed timeline, we took a pragmatic approach to recruiting, targeting two key participant groups: tech industry professionals and HCI researchers. Our focus was on recruiting 'experts,' defined as individuals with technical knowledge of recommender systems and an understanding of how and why these systems are developed. We were particularly interested in participants with insights into internal team dynamics and the defense of design decisions, aligning with our goal to explore perceptions of personal agency within these environments. Due to the project's condensed nature, we turned to peer schools within the university, reaching out to people with whom we had no prior relationships. Four participants were recruited via word of mouth; the remaining four were referred via snowball sampling.
Our participant pool consisted of four professional developers, one professional designer, one PhD student studying recommender systems, one PhD student studying surveillance, one PhD student studying predictive policing, one professional artist whose previous work focused on surveillance, one community activist engaged in philanthropy and nonprofits, and one assistant city attorney dealing with public data, culminating in a total of eleven interviews. Our participant's domain knowledge and expertise allowed us to utilize participants in two roles: evaluators of feasibility and plausibility and collaborators in the act of speculating, critiquing, and understanding possible futures.
Our study protocol consisted of a brief introduction describing the project's critical nature and informing participants of any key definitions they might not know (often simply a brief explanation of speculative and critical design). We would then begin with the films. Each film would be introduced with two pieces of information: a one-sentence plot overview and a one-sentence description of topics. We would then play the film. Once the film had finished, we would wait for initial participant reactions or prompt them for their first thoughts, then work with questions from our discussion guide. We would spend roughly 20 minutes on each film. We would then ask our closing questions, asking participants to reflect across all of the videos. The interviews would last between 120 to 180 minutes. The films would always be shown in the order seen in.
The study protocol was updated twice throughout the study. These times were after the second and fourth interviews. Our first iteration added additional questions, targeting areas of discussion we felt the team had overlooked. The second iteration of the study included adding two new components. The first item was a scenario summarizing the plot of the first film that would be read to the participant before viewing. The second item was a short animated video from a car insurer describing their RightTrack program (a similar concept to the service in Evidence of Insurability).
Once the interviews were completed, the recordings were transcribed using Dovetails' transcription feature. Author 1 conducted each round of analysis. While reading the transcripts, they inductively used thematic coding to identify thematic groups in participant responses. Quotes with these codes were extracted from the transcripts and grouped into themes, with a short paragraph summarizing them. We drew on various frameworks encompassing feminist and Marxist theory, finding alignment in Mark Fisher's Capitalist Realism and, later, Dencik's reinterpretation and definition of Surveillance Realism.
A spread from the final publication highlighting a selection of participant quotes grouped thematically.
Dissemination
With the project largely completed at this point, I was interested in how this work might live on. I wondered how the findings and insights developed during my research might find find relevance for designers and other researchers working in industry or academia. These were two of the driving questions behind the project site and the thesis publication. In a short 1 week sprint, I developed the Between Lines project site on Webflow. The design and interactions went beyond what was possible with vanilla Webflow, so I turned to ChatGPT 3.5 and 4 to help generate custom code. While not perfect (there are still a few annoying bugs remaining), I am happy with what I was able to accomplish with only a basic understanding of HTML and CSS. The thesis publication, which is covered on the Between Lines: On Location & Recommendation project page grew out of a similar mindset. I aimed to create a standalone publication that could exist without the short films. Additionally, I wanted to challenge myself—I had no formal visual communication training, nor publication design experience prior to my time at UW. It'd always been illusive. I taught myself the fundamentals of publication design to create a bespoke 212 page coptic bound book. The publication and fabrication process was completed in 2 weeks. Additionally, I have continued to work, formalizing my findings and documentation for publication at CHI 2025 and DIS 2025 conferences. Even if rejected, 10 peer reviewers will have throughly engaged with the work—a new form of guerrilla dissemination. These ideal were carried into the design of the installation at the Jacob Lawrence Gallery, in which the publication and videos were presented to a wide ranging public audience comprising industry professionals and everyday people.
Between Lines installed at the Jacob Lawrence Gallery.
An overview of Between Lines, a critical speculative design project, completed as a portion of requirement for my Masters Degree in Interaction Design at the University of Washington
Interaction Design
Wyatt Olson
Visual Design
Wyatt Olson
Burke Smithers
Filmmaking
Wyatt Olson
Laura Le
Rachael Winkler
Anne Winkler
Mary Olson
Nelly Quezada
Teresa McDade
Kenneth Nguyen
AK McDade
Mark Olson
Publication Design
Wyatt Olson
Installation Design
Wyatt Olson
Fabrication
Wyatt Olson
Laura Le
Mark Olson