Together with 3D applications like ZBrush and Blender, Wacom Cintiq Pro’s precise pen on screen functionality has enabled artists to push the boundaries to create photorealistic assets within Unreal Engine.
Previously, digital assets were painstakingly designed and refined, manually, using 3D tools and digital clay. Teams of creators were needed to make each item, and the process was linear and took a large number of hours.
Now, creators are equipped with state-of-the-art tools that allow artists to feel like they’re creating in the real world. The level of accuracy and precision in digital creation has increased exponentially through a new technique, photogrammetry.
What is photogrammetry? A technique by which, from an image generated with multiple cameras and in different angles, we can produce a 3D model with the geometries, textures and materials of an object, being a garment, a prop or a set decoration item.
With this high-tech photogrammetry process, real world clothing, props, and other items are now able to be scanned and refined using an app like Reality Capture, along with a Pro Pen 3D on a Cintiq Pro. Our friends at Peris Digital Services have a full end-to-end photogrammetry process that makes clothing for digital humans in the Unreal Engine, using the Wacom Cintiq Pro in multiple steps along the way.
“Our process goes beyond photogrammetry, it is artistic and technical expression. 3D applications are becoming more intuitive and the workflow is faster with Wacom tablets and displays. A mouse is not a “natural” tool – the pen leads you to creation, whether in 3D with tools like Blender, Reality Capture, ZBrush or in retouching tools like Photoshop.” – Jesus Luque, Chief Technology Officer, Peris Digital Services
You can watch a full overview of how photogrammetry is used to scan props (and people!) in this highlight reel by our friends at Fathead (a part of the Entertainment Technology Center) and The Scan Truck. This includes the entire end-to-end process including both scanning objects and 3D printing them.
Photographing the physical object
Each digital item starts with a real-world prop. At Peris Digital Services, their team uses a complex photogrammetry rig built with 144 Sony α7C & α7RIV cameras to quickly scan any garment, accessory, prop or set decoration object.
This rig captures the physical asset at all angles and captures even the most minute details, allowing it to be scanned as a 3D object.
Raw geometry process with Reality Capture and Cintiq Pro
Once photographed, an initial raw asset is created using a tool like Reality Capture. Similar to a raw photo file, the digital asset is unrefined at this point and will need to be processed using other tools before being used in the Unreal Engine. Millions and millions of triangles are used to map to points on the model – up to 40 million at a time, which is what allows the digital asset to resemble its real-world counterpart almost identically.
In the next few stages of this process, photogrammetry pros like Peris Digital Services rely on Wacom products for not only exact precision, but high-res views of their work.
“A Cintiq Pro 16 is portable and a 24 is ideal for day-to-day work. But the Cintiq Pro 32 is a gorgeous experience. With its powerful resolution and color quality, it allows us to identify all imperfections, improving the productivity and quality of our scans.” – Jesus Luque, Peris Digital Services
Remesh & UVs with ZBrush and Substance Painter
After assets have been created in Reality Capture, the next phase is clean up and processing to fix any projection errors. This is done using a 3D modeling tool like ZBrush. Then, tools like Substance Painter are used to bake textures onto the 3D asset.
These few steps are what really make a virtual asset feel real. It’s a process of creating and refining, done multiple times, until the perfect result is achieved. This phase of the photogrammetry process is painstakingly thorough, while still providing a much quicker timeline for digital assets than creating them from scratch.
Once assets have achieved the highest level of fidelity and detail, they’re ready to be exported into a platform like Unreal Engine to be used in the digital world.
What’s next in Photogrammetry?
While photogrammetry alone is a technological force to be reckoned with alone, it only opens more doors into the Metaverse going forward. Jesus Luque from Peris Digital explains the future of this industry as:
“Fundamental in all processes, not only in media – but working with 3D visualization opens a new path forward for retailers and overall interactions with customers.”
At Wacom, we’re excited for our products to expand into the photogrammetry and 3D workflows. The Cintiq Pro line of products provides artists in this space with the best-in-class tools. We continue to provide creatives in this industry and beyond with the most precise and powerful hardware as they continue to innovate.
To learn more about Wacom in the world of virtual production, get in touch with a Wacom product specialist at developer@wacom.com.
