Démos

The use of open standards like MPEG, demonstrates the ability to deploy such content and experience at large scale. Moreover, leveraging the compression efficiency of MPEG video codecs, the proposed solutions demonstrate viable use cases for streaming of volumetric assets and scenes. Integration of V3C into a future version of MPEG scene description standard will enable global scene compositing and more complex environment distribution.

This demonstration will enable viewers to access 2D video, dynamic point cloud content and multi-view plus depth content in a seamless interaction in both VR and AR context. It will show how such content can bring additional immersive experience even on a 2D screen such as a smartphone or a tablet, as well as on Handheld Mounted Display (HMD).

In the framework of the CORESA 2024 conference, this demonstration will highlight newly released standards and will help to engage discussions with researchers about future standard developments and multimedia signal processing tools required to enable such XR ecosystem and to improve the proposed experience. The V3C Immersive Platform has been recently released in 5G-MAG reference tools1 and is available for download and contributions. It comprises the native decoder plugin and a simple application for test, interoperability and enable the development of new demonstrations.

High production costs have hindered the widespread adoption of UHD broadcasting, with only a few special events being produced in UHD. Most 4K content comes from streaming platforms, and even then, availability is limited compared to non-4K options. As a result, UHD displays often rely on upscaling, which is constrained by device capabilities. High-quality upscaling offers a solution to expand 4K content by enabling content owners to convert existing HD libraries, reduce costs, and optimize their production pipelines, while maintaining control over the final UHD output. We propose to demonstrate a cloud-based AI-driven upscaling solution, based on a proprietary Generative Adversarial Networks (GANs) architecture able to significantly outperform traditional upscaling and deinterlacing methods while retaining low and scalable operational costs.

Convolutional Neural Networks (CNNs) are widely studied for image upscaling and generally outperform traditional methods. The simplest approach uses supervised learning with high- resolution images and their downscaled versions to train a generator CNN to restore missing details. However, this can produce soft, blurred results if the loss function isn't chosen carefully. Generative Adversarial Networks (GANs) address this by adding a Discriminator to assess the quality of upscaled images. As the Generator and Discriminator train together, they improve each other, with the Generator creating increasingly realistic upscales to fool the Discriminator. Several Super-Resolution solutions rely on GANs but target mainly still image upscaling. Most video solutions simply apply an analogous approach on a frame-by-frame basis, resulting in temporal inconsistencies such as shimmering and flickering. By referencing multiple fields or frames from the low-resolution video to generate each upscaled picture, the proposed solution not only significantly improves temporal stability, but is also able to capture more detail from the multiple references. Simultaneous deinterlacing and upscaling can also be done by the same CNN, with far better results than what is possible by combining conventional deinterlacing methods with conventional or Super-Resolution upscaling techniques. The fact the network was also trained to mitigate compression artifacts from previous encoding stages also contributes to the real-world performance of the proposed solution.

The inference process is computationally intensive but can be significantly sped up using GPU-accelerated virtual machines (VMs) in the cloud, reducing the need for acquiring expensive hardware and enabling scalable, cost-efficient operations. Currently designed for offline upscaling, content is uploaded and retrieved from secure storage. Further work on parallel task scheduling will allow us to offer a real-time cloud deployed solution in the future. The upscaled content quality rivals native UHD and is being trialed by several European broadcasters.

The demonstration will include technical details via a slide presentation and side-by-side video comparisons on a TV display to highlight the advantages of this real-world solution over current methods.

DEMO 8 - 3DGS / Video volumetrique (Orange)