The Immersive Arts include Extended Reality (XR) which is the umbrella term for Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) and spatial sound. These technologies immerse users in a fully digital environment through a headset or surrounding display and engage our two main senses: sight and hearing. In this blog we aim to clarify the role of spatial audio in an immersive space by describing each way of listening.
Immersive Audio
An immersive audio (IA) sequence is composed of a sequence of bitstreams that are self-decodable and composed of non-timed Metadata. They are followed by a sequence of IA Frames whose structure is defined by the Non-timed Metadata. Each IA audio bitstream can deliver three dimensional soundfields that can be used for multichannel sound playback and immersive experiences.
Binaural: This is where the user is fixed in one place and experiences a fully 360-degree soundscape through a specially-encoded stereo file that has to be experienced through headphones. It models the way sound reflects around the head and the construction of the ear. In fact, it is often recorded with a microphone that mimics the size and shape of a human head! As demonstrated in this video, you can hear in every direction, but the audio is not responsive to user input if you move your head, the audio doesn’t change accordingly. The industry refers to this as “head-locked” audio.
Ambisonics or 3D audio delivers a fully 360-degree soundscape that is responsive to a visual field. According to where you move your head, the audio changes to reflect that movement. In addition to the horizontal plane, it covers sound sources above and below the listener.
Surround sound audio in most cases relies on engineers mixing multiple audio channels (e.g. 5.1, 7.1) for playback on numerous speakers that surround the audience. You’ve probably heard surround sound in movie theaters where it’s presented by companies like DTS, THX and Dolby.
Spatial Audio with Dolby Atmos (in Apple Music): 3D audio technology that creates a more immersive soundscape with compatible Apple products by using dynamic head-tracking for a 'theatre-like' experience. With Spatial Audio, Apple believe that sounds can be placed “virtually anywhere” in the hemisphere. To experience this you need to use Airpod Pro, Airpods Max, Airpods 3rd Gen or the latest Beats headphones.
360 reality audio
A new immersive music experience that uses Sony‘s (object-based) 360 Spatial Sound technology. Individual sounds such as vocals, chorus, piano, guitar, bass and even sounds of the live audience can be placed in a 360 spherical soundfield, giving artists and creators a new way to express their creativity.
Please put on your headphones and listen to some of these audio experiences for yourself examples.