A good filmmaker never forgets about audio. Good audio can make or break a film, no matter how good it might look, how well your actors perform, if it sounds terrible people will notice, and they will care.
In this project I need more than just a good audio mix, with good levels and positioning. I need audio can can move with the viewer. As they turn their head, phone, screen, what ever viewing method they choose, the audio placement needs to move with them.
In traditional film a solid stereo, or a surround sound, mix will work. Each frame of your film has sound specifically designed to match with what is shown. If the source of a noise it to the left, the sound engineer can place that sound to the left of the mix and so on. However when viewing a 360° film the viewer can alter what is in that frame. If they turn around what was on their right is now on their left and visa-versa. This presented an issue, a new method of applying sound to the film would be needed.
This is where spatial audio comes in. This is where you can map sounds to a specific point in the film and when the viewer moves the audio remains locked to its source and will rotate along with the image. As a basic example, if you are watching a 360° video of a band and the drums are to your left, the sound will fill your left earphone, if you rotate so they are now in front of you, the audio will present balanced in both ears, and if you turn further so they are to your right the audio will emanated from there instead.
This is achieved with the use of a 8-channel ambiX audio file combined with the 360° footage. The file includes mapping, that link the sound to certain points within the video.
The tool I will use to achieve this is Facebook 360's Spatial Workstation. It has the plugin's required to produce the audio file in Reaper, or ProTools, and the required software to combine the audio and video before uploading it to YouTube to be viewed,