Author: Maja Stark

In TRANSIENT EXPOSURE, we work with the Azure Kinect, a sensor bar that contains advanced AI sensors, a depth sensor and a spatial microphone array. The device can track body movements (keyword body tracking) and save them as a data set. The AI combines all collected data with a rough skeleton model.

To enable our team members in Colombo, Sri Lanka, to easily make recordings with the Azure Kinect for Unity, a recorder is being developed in XR_Unites which currently includes the following features:

• Capture motion via the Kinect’s body tracking feature
• Transferring these movements to a human 3D model
• Save the motion as an animation of a 3D model in a separate file
• Simultaneously record audio via a microphone
• Saving the audio in a separate (.wav) file
• Specifying a concrete recording time
• Simultaneous playback of audio and animation
• Loading and deleting the saved recordings via a menu

TRANSIENT EXPOSURE is an artistic experiment with the Hololens 2, the latest Mixed Reality goggles from Microsoft, which some may already know from artistic works such as BLOOM: Open Space by Brian Eno and Peter Chilvers or Concrete Storm by Studio Shift (both 2018). Usually, it is used in industry – for example, in manual production by digitally overlaying additional information and instructions. So far, it has not been used much in art, which may also be due to its high price of over € 3,000 – however, it is sometimes also lent by Microsoft for artistic projects, for example in the two cases mentioned above. The Hololens 2 features 6 degrees of freedom, area- and space-, as well as gesture-, hand- and speech recognition. It is also suitable for people wearing glasses. In XR_Unites, the Mixed Reality application for TRANSIENT EXPOSURE is being developed with the Unity game engine.