Let’s use the existing high quality 3D standards if we want 3D to return. We urgently need to switch to MPO and frame interleaved!

UPDATE 2: Sample of MKV video file using MK3D metadata. Non-3D devices will play them in 2d, but 3D devices and software will play them in 3D at full quality per eye! Jump to more details.

TABLE OF CONTENTS:

• Using better formats for 3D image distribution
• Let’s finally use proper 3D video formats. Period.

3D PHOTOS: Using high quality formats since XIX century. But we need to use better distribution methods

At the beginning of the Digital 3D era, 3D video was very difficult to use due to the power processing needed for video and not having good file standards to store, view and manage. Shortly after, new proper standards were developed, but people had learned to use the first temporary formats with serious compromises on the quality and ignored the new full quality 3D standards. Half-SBS was a temporary solution, both images were encoded on the same space that on a 2d version, that introduced three inconveniences, not usable for 2d users, less quality, and having to expand the image to make it usable.

Luckily, with 3D photos there were no limitations of power processing or storage space, so people started using Full-SBS, which quality of images doesn’t get reduced, just as all stereographers are doing in paper since the XIX century, but there’s still a problem with Full-SBS: when sharing the images to people without 3D screen, they will see two images, and that forces them to zoom in if they want to appreciate them better, so people not using 3D screens doesn’t like these images and won’t preserve in their library or share them even if they like it, and if they still want to save or share they need to do additional work to crop the images (or zoom in every time they want to watch them). Also, 3D devices and Apps doesn’t detect them as 3D files, so you have to manually activate the 3D every time, making them inconvenient.

Then, the Fuji W series 3D cameras came with a new format for 3D pictures: MPO. A new standard for photography was born (used also for other brands like Panasonic). As the MPO initials say (Multi Picture Object), this is not only useful for 3D photos, but it also can include more images (maybe even storing all the 4 views of Leia devices 4V images instead using depth maps, solving two problems in a row: depth problems with smoke/transparencies and losing the depth metadata when copying/sharing the file). At launch, the .mpo extension was hardly recognized by programs, but even then, you could force opening them and images loaded successfully. Now, more than 10 years later, MPO extension is recognized like any JPG on almost all programs and devices, not need to have a 3D device, you can open and send an MPO file even on famous messaging Apps on any device, and you will watch in the first stored image like it were another JPG file.

This means, that regular people even not interested in 3D, can browse, watch, and store MPO files, and share them like any other 2d file, but when they distribute or store the file, inside there’s still one image for each eye, so if someone with a 3D display receives an MPO, he/she can watch it in 3D no matter how many times were shared by 2d users. On the future, maybe even one of these 2d users will get a 3D display eventually, and then suddenly they will already have some 3D images that they were previously watching in 2d.

Click/Touch the below thumbnail to open an MPO image, your browser should show an apparently 2d image, but even if the browser doesn’t support 3D images, you can save it and open to watch in 3D on a 3D device or on an App/program supporting 3D images. Both 2d and 3D users can enjoy the image in their way; in the best version available for their display.

With MPO both 2d users and 3D users will see the best version of the image available for their screen. Plain or with depth. But JPG standard quality is very old. What if we use an even better standard to allow, not only more quality, but also more possibilities? Well, that standard already exists, it can store various versions of the same photo, videos and other complementary data to have more possibilities to do special things than if you use regular JPG. That is the HEIF format that is widely used by Apple on their devices: more quality in less space, motion (Live Photos) so you can have lots of retake/effects/editing possibilities after the photo is already taken, and other exciting things that you can do with them later in the same shutter press than if you shoot in regular JPG.

But HEIF files are not as widely supported as MPO, even with all his advantages, after 5 years the adoption is very basic, programs only support them for visualization, no editing or using the additional data and metadata (except for the iOS default Photo App, that is an example of possibilities, unfortunately almost nobody knows that power with an iPhone at their hands). The possibilities are endless, and without inconveniences except for the current lack of compatibility, but this will be solved over years, as modern operating systems and some programs/Apps are starting to recognize them. This means that HEIF will be a great replacement for MPO in the future.

3D VIDEO: We already have the proper formats and tools. Stop using sloppy solutions

What about video? With video, things are more complicated. Because the Half-SBS was so widely used, that some new devices instead supporting it for compatibility reasons, they rely on that format as the only way to see 3D videos, forgiving all the other better formats (Half-SBS is the format with the less quality). Some people like Top-Bottom (Over/Under) because they say there’s -slightly- more quality, but it has the same problems as Half-SBS and is not supported for every 3D display, so losing compatibility for only slightly more quality is not worth to even consider it as an alternative. Even Full-SBS is hardly supported by modern new devices. And is basically a better quality version of Half-SBS that only require a minimal change to the existing program routines applied to enlarge Half-SBS images, it should be an easy modification for the programmers/engineers of the Apps/Devices to add Full-SBS to their devices.

Youtube: For encoding files with 3D metadata for YouTube check the 4th update of this article.

But even Full-SBS is not the ideal format, it has the same inconveniences for 2d users than SBS photos. Fortunately, there’s a format that solve this: Frame interleaved, also known as Frame Sequential, and there are also video formats with dual channel that stores both images in full quality. Unfortunately, for some reason, new devices/Apps are not using them. It is the format used by 3D Blu-Rays. It preserves all quality on both eyes, so basically is twice the resolution than 2d, and when properly encoded it can be used by 2d users exactly like any other 2d video. Even better, because the frame interleaved format is stored as it were double frames per second, the compression algorithms only need to store changes between the two frames the same way any other movie is stored by calculating changes from key frames.

MK3D is a regular .mkv video file but with two views inside, one for each eye, and metadata indicating the 3D disposition*. MKV allows storing two video tracks at full quality in a single video file. Regular devices and players will play them normally; but 3D devices and programs will recognize the additional video stream and play the video in 3D at full quality. Same quality per eye than 2d. Just like MPO does.

You can use MKVToolNix GUI to encode 3D videos or pairs in this format, MKVmerge to join two video tracks*, or MakeMKV to extract 3D tracks from 3D Blu-Ray discs without quality loss. The file extension should be .mkv, otherwise the same file couldn’t be used for both 2d and 3D, therefore losing one of its major advantages.

 *MKV file container contains the StereoMode field on the metadata of the file, set to the right format

For that reason 3D Blu-Rays doesn’t need much more additional storage than 2d Blu-ray versions, the disc store the same full key frames than 2d version and only need to store additional frame changes for the other eye (yes, when you watch any movie, almost all the images you see are not stored, but calculated in real time from only a few stored key frames). And can solve also inherent differences of light/white balance between the lenses for each eye.

Frame-packing (not to be confused with Frame Sequential) is another way to send Frame interleaved/Frame Sequential to a 3D display, as it translates into frame interleaved through the HDMI cable to the display. Though it occupies double file size for storage than frame interleaved, so we recommend to use directly Frame Interleaved.

So, storing movies as frame interleaved have even more advantages than MPO for photo files, there’s no need to support a new format and the file size is not much bigger than 2d version, having the same full quality per eye than 2d version. The reason that this format is used only by 3D Blu-rays having that many advantages is a mystery to me. And to make things worse, instead of supporting the best format, many new devices only support the format with the worst quality. That makes me wonder how 3D have survived so many years. We could have only 1 version of a movie or video with both the 3D and 2d versions, and 2d users will not notice any difference and 3D users will get the full experience. This could also be applied to streaming services like YouTube and Prime Video/Netflix/Disney+/etc.

But wait… Yeah! I know! In the past I have been arguing that Vudu are discarding millions of 3D devices because they likely use frame interleaved format, and now I’m saying everybody should use frame interleaved. Why I changed my mind? Because now I see that the real problem is caused by the brands releasing 3D devices that only support the bad old format of the beginnings of Digital 3D era, instead of supporting the existing perfect 3D format (same quality per eye than 2d version). I was in a fight with the wrong enemies. Vudu is doing the right thing: offering great quality 3D movies in the best available format, but brands are making devices that are anchoring us to the past, I often see 3D enthusiasts replacing their 3DTV (some even 4K 3DTV) for a regular 4K TV, just because a lot of 3D content simply is encoded in bad formats and for them 4K seems too superior compared. Half-SBS should be unacceptable in the 4K era. Imagine streaming services offering 4K 3D frame interleaved content, that’s 4K per eye. And existing 4K 3DTVs and projectors will support it, doesn’t need a new standard, it already exists. You could enjoy today, right now, the same quality than 4K 2d but in 3D in your 4K 3DTV or 3D projector, the content providers just need to start using frame interleave.

We have the tools, but it seems brands and some users are locked in the past. We need to use the best 3D available, instead of using the compromise past solutions of the very beginning of Digital 3D era. Evolving to the right formats, 3D will have a future.

We have great 3D formats, but the few new 3D devices/services we receive, only use bad and inconvenient formats that impede the spread of 3D everywhere, when it could be spread even by 2d users.

If you know brands making devices relying only on old formats, suggest and urge them to embrace the future of 3D, instead of repeating the same limiting features that are blocking the possibilities and scope of 3D.

We have the power to influence brands when making new devices. Let’s use our voices to help them to make 3D ubiquitous.