Project Sidekick

Sidekick is a project developed by NASA and Microsoft to provide virtual aid for astronauts using the Microsoft HoloLens augmented-reality headset. As of December 2015, Sidekick is deployed at the International Space Station.

Functionality
Sidekick has two modes of operation. Remote Expert Mode uses the functionality of the Holographic Skype application—voice and video chat, real-time virtual annotation—to allow a ground operator and space crew member to collaborate directly over what the astronaut sees, with the ground operator able to see the crew member's view in 3D, provide interactive guidance, and draw annotations into the crew member's environment. In Procedure Mode, animated virtual illustrations display on top of objects as a crew member interacts with them. This mode can be used for guidance and instructional purposes in standalone scenarios. Applications include inventory management, in which the system recognizes an inventory item, and can display a path to where the item should be stored. Previously, crew members would rely primarily on printed instructions and voice-based communication when performing complex tasks. The capabilities provided by Sidekick have been promoted as potential enabling features allowing for reduced crew training requirements and increased efficiency in deep space missions in which communication delays can complicate difficult operations.

History
After having performed simulated reduced-gravity testing in its Weightless Wonder C9 aircraft, NASA attempted to launch a pair of HoloLens units aboard the SpaceX CRS-7 launch to the International Space Station on June 28, 2015, but the Falcon 9 rocket carrying the payload exploded at 2 minutes and 19 seconds into the flight. Sidekick was tested at the Aquarius laboratory from late July to early August 2015 as part of the two-week long NASA Extreme Environment Missions Operations 20 expedition, demonstrating its operation in tasks such as equipment checks and setup. HoloLens hardware was successfully delivered to the ISS in the Cygnus CRS OA-4 commercial resupply mission on December 9, 2015.