Hindsight: Similarity-Based Analytics for Mars Rover Drive Retrieval
Luke Fiorante (Harvard University), Leslie Liu (Carnegie Mellon University), Adam Zhengyuan Xu (ArtCenter College of Design), Xianmei Lei (Jet Propulsion Laboratory), Darwin Chiu (Jet Propulsion Laboratory), Krys Blackwood (Jet Propulsion Lab), Maggie Hendrie (Art Center College of Design), Scott Davidoff (Space Science Institute), Santiago V Lombeyda (California Institute of Technology), Hillary Mushkin (California Institute of Technology (Caltech))
Jump to Discussion
Presentation
- Session
- Lost in Dimensions
- Time
- Thursday, Nov 12, 15:27 – 15:36 (US/Eastern) · session 15:00 – 16:30
- Room
- Hall Essex north
Links
Sign in to access the Paper PDF.
- Supplemental Material(opens in new tab)
Abstract
While Mars rover operators plan drives across hazardous Martian terrain and diagnose unexpected faults, the necessary information is distributed across separate systems and often reconstructed through manual correlation and memory. To address this challenge, we partnered with Mars rover operators at the NASA Jet Propulsion Laboratory to introduce Hindsight, a visual analytics system that unifies previously disparate rover drive data into a single workspace for search, comparison, and investigation. This paper presents a design study of the Hindsight application. The partnership revealed that operators reason about drives as holistic spatiotemporal episodes rather than discrete parameters. By externalizing operator intuition into an explicit visual query process, we argue that Hindsight transforms analysis into a structured, shareable workflow. Preliminary feedback from operators suggests Hindsight supports their ability to correlate terrain, telemetry, and fault events within a single workspace.
For Practitioners
The most directly interested practitioners are spacecraft and planetary rover operators: the mission engineers and rover planners who plan traverses and diagnose faults from telemetry, imagery, and event logs. More broadly, the paper speaks to practitioners who reason about complex operational archives by analogy to remembered episodes rather than by parameter thresholds, including operators in other high-risk, time-constrained domains such as aerospace operations and diagnostic settings where a current case is understood against historically similar ones. Practitioners can apply two transferable lessons. Methodologically, our formative design probe shows how to surface the retrieval representation experts actually reason with before committing to an interface. At the system level, the paper offers a pattern for archives split across single-purpose tools: pairing similarity-based search with parametric filtering, and unifying heterogeneous data streams under a shared temporal reference so any moment of interest can be inspected across all modalities at once.