What is High Density Encoding?
CODEX High Density Encoding [HDE] is an encoding technique optimised for Bayer pattern images and provides bitexact data reduction of uncompressed camera ARRIRAW files from cameras like those in the ARRI ALEXA family. For example, the size of a single ARRIRAW OpenGate frame at 3.4K resolution is 11.5MB, meaning that an hour is 996.2GB. HDE can reduce this with up to a 2:1 ratio.
Initial Reaction
When HDE was first presented to key studio executives and post production partners it received a great reception, because everyone who saw it understood that it mitigates the need for increased storage and streamlines ARRIRAW workflows with lossless data compression. The reduced file size provides efficiencies all the way from dailies to VFX pulls to the final digital intermediate.
The benefits are obvious to studios like Marvel whose productions are international in scope and include large numbers of VFX shots or to Netflix, whose production slate continues to grow and grow and who demand the highest quality RAW content for their feature and original series acquisition.
"CODEX HIGH DENSITY ENCODING DOUBLES YOUR TRANSMISSION SPEEDS AND IMPROVES MANY ASPECTS OF YOUR UNCOMPRESSED RAW WORKFLOW PIPELINE."
Spider-Man: Far From Home
Spider-Man: Far From Home is a follow-up to Spider-Man: Homecoming. The script revolves around Peter Parker and his friends going on a European vacation. It was clear early on that there would be multiple production locations in Europe and the United States. Principal photography was based at Warner Bros. Leavesden Studios, just outside London with additional locations in and around New York, Prague and Venice. These location changes placed particular challenges on the production and post production teams. Like Spider-Man: Homecoming, this was a co-production between Marvel Studios and Sony Pictures and was directed by Jon Watts.
A common challenge on many projects, including Spider-Man: Far From Home, is moving files securely from one place to another. Every single frame captured to be sent back to Los Angeles, so the volume of data to be transferred was large. A simple workflow was set up by Sony and Marvel to meet this challenge, using Codex HDE to reduce the size of the ARRIRAW files, with no sacrifice to the integrity of the ARRIRAW images.
DIT Francesco Luigi Giardello (Jurassic World: Fallen Kingdom, Thor: The Dark World, Aladdin) and his team assisted cinematographer Matthew J. Lloyd (Daredevil, The Defenders) on Spider-Man: Far From Home. The primary camera used was the ARRI ALEXA Mini, shooting ARRIRAW Open Gate (3424 x 2202). The main unit had 3 Minis, and so did the 2nd Unit. Francesco and his team provided the ARRIRAW files on Codex Transfer Drives to the near-set lab. They were then cloned to a local NAS (network-attached storage) as HDE ARRIRAW files using CODEX Production Suite. The file size was reduced by an average of 40%. The smaller data footprint meant that the entire movie online (on a 200TB NAS) for the duration of the shoot. This was extremely useful – if production had to return to a roll or a shoot day for regrades or metadata adjustments, they had instant access to the files they needed rather than having to wait while material was retrieved from an LTO tape.
"INTEGRATION INTO OUR PRODUCTS WAS QUICK AND EASY AND THE SMALLER DATA FOOTPRINT IMPROVES THE EFFICIENCY OF OUR HIGH-PERFORMANCE WORKFLOWS, MAKING FOR FASTER RUSHES TURNAROUND."
Instead of shipping hard drives from London back to Marvel Studios in Los Angeles, the HDE ARRIRAW files were pushed via a 10GbE switch and Aspera. The smaller file size dramatically reduced the amount of data that had to be relayed from London to Los Angeles. Checksums ensured data integrity prior to Open EXR files being generated for VFX using Marvel’s VFX Plates lab built on CODEX Vault hardware.
Filmlight’s Daylight was used to generate dailies directly from the HDE ARRIRAW files – specifically MXF files for editorial and PIX-compatible files for viewing and review at Sony and Marvel. Filmlight rose to the challenge of supporting HDE deliverables by integrating High Density Encoding into both their Daylight and Baselight products. Filmlight’s integration of HDE into Baselight means that the HDE files can be used for the final digital intermediate grading.
CODEX HIGH DENSITY ENCODING SAVES ON STORAGE CAPACITY, TIME AND MONEY. IT IS A SOLUTION “THAT CHANGES EVERYTHING."
HDE Knowledge Base
Facts
- HDE stands for High Density Encoding.
- HDE reduces ARRIRAW and other bayer pattern file sizes by around 40%.
- When an ARRIRAW image is encoded as HDE, the file extension changes from .ari to .arx.
- The image essence is encoded, but the file header is otherwise identical.
- HDE encoding is lossless - when an HDE file is decoded, it is a bit-for-bit perfect match to the original file. HDE can be used on ARRIRAW images of any resolution.
Advantages
- Lossless reduction of the storage size of ARRIRAW images.
- All the original pixel values can be accessed.
- The reduction in size for ARRIRAW images drastically reduces storage costs and file transfer times, speeding up your workflow and saving you money.
Performance
- HDE provides fast encode/decode speed. (CPU dependent)
- ARRIRAW Open Gate 4.5K can be encoded comfortably at 24 fps on a modern MacBook Pro.
- ARRIRAW Open Gate 4.5K can be encoded at over 50 fps on a 10-core iMac Pro.
How to encode ARRIRAW HDE
