Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics System (Joint AR) Product



Journal Title

Journal ISSN

Volume Title


2023 International Conference on Environmental Systems


Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate and tight choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. Additionally, EVA operations, aside from the Apollo Lunar surface missions, have predominately focused on maintenance and construction tasks where success criteria are clearly measurable. However, future exploration missions expect to enable crew to carry out scientific objectives in increasingly Earth-independent ways. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. This paper highlights the project's experience with a product-focused management style and use-case centered systems engineering approach to iteratively design, build, and test. The Joint AR product features were defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, graphics engine software and analog test beds. We outline the defining architectural design decisions, including safety criticality considerations, suit mounting interfaces, computer architectures, and partnership contracting mechanisms. The outcomes of these studies, architecture decisions, and management requirements result in a recommended design which is the Joint AR product. We discuss the evolution, development of these system components, and what work remains. We hope to share a unified understanding of various design decisions and how they impact the future of crew members' access to data during Lunar and Martian EVAs. This ongoing effort can enable a community-wide discovery process toward realizing necessary AR features and capabilities for future missions.


Paromita Mitra, NASA Johnson Space Center, USA
Matthew Miller, Jacobs/NASA Johnson Space Center, USA
Briana Krygier, NASA Johnson Space Center, USA
Sarosh Nandwani, NASA Johnson Space Center, USA
Matthew Noyes, NASA Johnson Space Center, USA
Vishnuvardhan Selvakumar, Guardians of Honor/NASA Johnson Space Center, USA
Amanda Smith, KBR Wyle/NASA Johnson Space Center, USA
Tyler Garrett, NASA Johnson Space Center, USA
ICES403: Extravehicular Activity: Space Suit and Surface Mobility Operations
The 52nd International Conference on Environmental Systems was held in Calgary, Canada, on 16 July 2023 through 20 July 2023.


augmented reality (AR), informatics, heads up display (HUD), radiation, graphics, engine, optics, optical, operations, spacesuit, helmet, mounted