Pixel magic from Pixelworks – the X7 mobile visual processor
-
Loading... - Sussane Poulsen
- 07 Jul 2022
- 114 Views
- 0 Like
- 0 Comment
Motion estimation (ME) and motion compensation (MC) have been extensively used for video frame interpolation systems since the turn of the century. However, typical solutions are power-hungry and introduce interpolation errors. Pixelworks, a video alchemist skunkworks in San Jose, California, has been quietly developing a new back-end mobile processor that works for both movies and now video games. The result is an incredibly brilliant image, at super high resolutions and high frame rates, but with minimal latency that can actually improve battery life as well—it sounds almost as if they have broken the law of physics. But no, says their CEO, we just know those laws very well and how to work within them.
By using an expert-trained video motion estimation and compensation algorithm in silicon for video frame interpolation, the company has pushed the bounds limiting performance and quality. Through a novel adaptive compensation scheme trained by the company’s picture-quality experts (that the company doesn’t want to talk too much about), Pixelworks has integrated motion processing—things they’ve done for the film and TV industry in software—into a mobile gaming solution that is now possible in a smartphone.
This secret sauce, the expert training by their best engineers and picture-quality people, is differentiable such that the flow and estimation can be optimized jointly in real time, and very fast, with minimal latency. In addition to the smart MEMC, the design can be seamlessly adapted to several other video enhancement tasks such as super-resolution, denoising, and deblocking. Quantitative and qualitative evaluations and demonstrations have shown the X7 post processor outperforms state-of-the-art video frame interpolation and enhancement algorithms on a wide range of datasets. Compared to existing methods, Pixelworks’ magic is computationally efficient and able to generate more visually appealing results.
One segment that appreciates this is gaming, and as most people know, gaming has become a big business in units and dollars worldwide. It is especially big in China, where it is estimated there are 670 million gamers—20 percent of the world’s estimated 3.3 billion gamers. Most of those gamers play on a smartphone. Chinese gamers have adopted Genshin Impact from miHoYo as a benchmark. One of the elements in the game is weather. Weather is an exploration mechanic. While exploring the world, the weather can shift. Depending on the new weather conditions, special properties will be applied to the environment, to the characters, and to opponents.
Games like Genshin Impact and PC games like Cyberpunk 2077 that have been ported from the PC to smartphones challenge the phone’s performance vs. battery and heat operating parameters. Big games demand everything the phone’s processor has, and in the process, runs down the battery while heating up the phone.
Pixelworks saw that as an opportunity to apply its magic and introduced the X7 chip architecture, post processors that exploited some of the techniques the company used in the film and TV industry. The X7 chips created a new distributed computing/rendering chip market segment in mobile—a bit like what Nvidia and AMD have done in the PC and console segments.Phone manufacturers have referred to X7 as stand-alone display solution, a render accelerator, or a dual-chip gaming architecture.
MEMC and resolution scaling (RS) are very demanding on any platform and especially in a mobile device with limited cooling and power supply. The phone must deliver a high frame rate, with high color accuracy, high pixel resolution, no artifact or aliasing, and low latency—and accomplish all that while using very little power.
Pixelworks’ post processor has proven to be just what the doctor ordered: a unique technology for motion compensation and resolution scaling that off-loads the phone’s SoC. For gaming specifically, by taking care of the motion vectors and working with the game engine developers, the company’s X7 processor uses metadata, which guarantees the best gaming experiences. Read More...
