Ultra Reality Extend: A Generative Display with Programmable Depth

Programming photons for true-depth immersion, productivity, & functionality.

Real footage as viewed with Brelyon Ultra Reality Extend. The recording camera lens alternates focus between the character at 0.7 m and the background at 2.5 m, showing the true optical depth provided by the display.

Imagine looking through a window and experiencing your visual content that moves towards you, wraps around you, and engages you with the same richness and depth as seeing the physical world. Or working with visual content that automatically understands your goals and makes learning curves easier to climb.

Brelyon Ultra Reality Extend is the very first generative display system, fluidly rendering virtual images at different depths and generating programmable content overlaid on your existing video streams, games, and data visualizations. The result is a dynamic, true-depth visual experience and a seamless window into a virtual world. But to really understand its impact, and how it goes beyond even state-of-the-art photorealism, you’ll need to understand display technology and its burgeoning transformation.

Not All Pixels Are the Same

Look in any direction and chances are you’ll see a screen. Pixels are everywhere–they’re cheap and of high quality. But simply increasing resolution and adding more pixels into your view isn’t necessarily better. They’ll just end up creating a logjam in your vision system, and the cluttered visual content will end up losing meaning. To be sure, a great display system indeed has all the right technical specifications–resolution, contrast, brightness–but it doesn’t lose the forest for the trees. Rather, it abstracts beyond the nuts and bolts, optimizes the experience holistically, and composes a harmonious, intuitive visual experience with intelligent interactivity.

Brelyon Ultra Reality Extend is up to the task. It crafts an end-to-end experience, from the basic image-forming photons to human perception and comfort. Here’s how.

Brelyon Ultra Reality Extend can show your existing input video stream at one monocular depth and automatically generate new content closer to you, with application in AR-enhanced productivity and real-time depth perception in your video games.

Power with an Extra Bit of Monocular Depth

Brelyon Ultra Reality Extend (pictured above) integrates AI computation with advanced optics to dynamically assign depth values to your input content and to render physically realistic imagery and information overlays. A descendant of Brelyon’s flagship product, Ultra Reality, it produces true multi-depth imagery that your eyes accurately focus to (we call this “monocular depth”): your eyes clearly see content when focusing at 70 cm (~28") and at 2.5 m (~100"), with the largest images stretching to 122", all viewable through a 30" window. Here is an example of the panoramic view Ultra Reality Extend generates, with images on the right captured by a camera focusing at different monocular depths:

Example photographs of content as viewed with Brelyon Ultra Reality Extend. The generative display’s monocular depth bit increases the visual bandwidth in alignment your perception of depth.

Ultra Reality Extend generalizes GPU-driven photorealism by introducing a computational bit of monocular depth to better match the visual experience of the real world. It’s like an 8K display (4K resolution per image x 1 monocular depth bit) with OLED-based curved 2D virtual imagery that produces depth content. The result is better eye comfort and enhanced productivity and immersion, not just for entertainment, but also for those data-critical visualization applications in, e.g., security/defense, healthcare, and manufacturing use cases.

Example applications of multifocal content provided by Ultra Reality Extend.

There’s even more to the story. As a generative display, Ultra Reality Extend automatically tailors the user experience according to a viewer’s goals, tasks, interests, and skills. We can lump these qualities into three use cases: entertainment, productivity, and functionality. These use cases aren’t strictly mutually exclusive and have a good deal of overlap, but they’ll let us highlight the main features.

In entertainment, an extra depth bit means more opportunity for programmable immersion. Ultra Reality Extend integrates Brelyon’s ARGEN Heart computational module into its real-time rendering pipeline. Now, turning 2D content into 3D content in real time is computationally expensive, but ARGEN Heart does this per-frame conversion using Ultra Reality Extend’s depth bit, allowing content on the horizon to physically look farther than objects in the foreground. See below for some validated examples. All these images are recorded by a physical camera that sweeps its focus while pointed at Ultra Reality Extend.

Imagine a superhero in a video game jumping toward you as your eyes really track its depth!

Note the color vibrancy, resolution, and wide field of view. In all of these cases, ARGEN Heart uses your existing video game feed to discriminate monocular depth and induce the same perceptive effects that you experience when seeing a real 3D scene. It works well on expansive, panoramic imagery and fast moving content:

At its core, ARGEN Heart is a transformer-based deep learning module with multifunctional capabilities, increasing the visual bandwidth of your content, going well beyond immersive entertainment and gaming. It automatically generates augmented-reality (AR) content on your existing feeds. A virtual instructor helping you train for a skill in real time, generative annotations or instructions in security operation centers or simulation training, an automatic content editor enriching your live streams— it’s all done for you on the fly, efficiently and seamlessly.

A generative display provides many cascaded benefits in security and training. For example, in mission-critical applications, a million-dollar decision is made in the blink of an eye. Ultra Reality Extend’s depth encoding to give the operator quick accessibility to information without requiring fixation on specific data. This unrivaled visual ergonomics simultaneously improves situational awareness and reduces cognitive load, which results in overall better decision making performance in security operations centers, upskilling tasks, and training exercises in flight simulators.

Ultra Reality Extend also accurately mimics HUD functionality and enables pilots to perceive the far environmental depth and the close cockpit depth accurately (avoiding negative training effects). And, similarly, operators of teleoperations setups can maneuver remote assets with angle accuracy that enhances situational awareness and avoids unwanted collisions.

Ultra Reality Extend generates an AR overlay on existing video streams, especially useful for mission-critical applications, simulation training, and teleoperations.

Ultra Reality Extend is versatile and modular. And whereas programmable depth represents the next generation of intelligent displays, other configurations are possible. Run two HDMI inputs for multiviewing in depth, or adjust the transparency of one depth layer to use it as a standalone virtual monitor. All these permutations render Ultra Reality Extend the world’s commercial multifocal generative display system. And, if you want the augmentation capabilities of ARGEN Heart on your own existing display, check out Brelyon Visual Engine as a standalone generative software tool.

A new category of immersive functionality, upskilling, and productivity.

Hear the full story of generative displays:

Conclusion

The visual experience is a core part of the human experience. And a generative display, marrying generative AI with optical hardware, is at the forefront. It adds life to visual content and enhances all your tasks, from the novel to the mundane. It understands your activities — playing a game, training or upskilling, monitoring a warehouse — and gives you a boost. The result is better immersion, improved focus, and a super human-computer interaction.