Vasili Manfredi

From 03/06/2024 to 03/12/2024

Paired Teams

Photorealistic Imaging and EXact modeLing for enhanced Virtual Reality

Carleton University, CIMS Carleton Immersive Media Studio

Vasili Manfredi is a 3D Artist and Architect. He has a Master Degree in Architecture – Built Environment – Interiors at Politecnico di Milano. His researches are mainly focused on the optimization of complex dataset from the photogrammetry survey process to achieve photorealism in Virtual Reality application, survey data collection with low-end hardware, and data processing with Open-Source software.

He collaborates with LABORA at Politecnico di Milano on different topics regarding VR applications, optimization processes and workshops.

He is currently in Ottawa (Canada) from 03/06/2024 to 03/12/2024  at Carleton University’s CIMS Lab, conducting NGI’s research for six months.   

The PIXEL VR project focuses on optimizing complex datasets derived from photogrammetric surveys to produce a highly efficient 3D mesh that can be used in Virtual Reality environment. The project addresses the need for comprehensive cultural heritage documentation while enhancing the usability of the tridimensional output for different users, including the educational field, historical preservation, tourism sector, scientific and technical experts. Using the photogrammetric output will avoid the mystification often found in 3D reconstruction and will provide the visual counterpart materials without any alteration. Cultural heritage documentation has reached unprecedented levels of depth and accuracy and the dataset derived from the photogrammetric documentation are often unusable, their processing is time consuming for most users due to their complexity and impractical to be used on different application purposes. This research proposes a workflow that enables the creation of a highly optimized 3D representation that preserves the photorealistic data derived from photogrammetry, while remaining accessible to low-end hardware within virtual reality environments. The entire research is conducted using free and/or open-source software to ensure knowledge sharing, process customizability, and accessibility of the final product to end users. The research outcome will be an interactive VR application that can be used on standalone VR headsets, without relying on the computational resources of desktop computers.

The picture shows different decimation processes tested from the High-poly  source to the optimised 3D mesh output.

During the first half term of the expedition different optimization workflows had been tested, from the photogrammetric data processing to the Tri dimensional mesh optimisation. It was possible to identify, from each workflow, the weaknesses and the strengths of each, creating an optimization process that could lead to a highly efficient 3D mesh that kept the photorealistic data. It was possible to decimate the High polygon mesh from 305 million poly to 1.4 k poly without losing any essential visual data. This result led to a general improvement on the performance and a more efficient file (3D mesh, textures and modelling file) size.

Currently, the expedition led to  a general improvement of processing techniques, from the raw data to the post-processed one in order to maximise the quality and the density of the point cloud generation from the photogrammetric survey. We were able to define and test different decimation workflows to optimize and improve current techniques of mesh generation. We are working on the automation of the workflow process. We are working on texture’s supersampling techniques to test possible performance and visual improvements.

The process has been used on ongoing VR project with excellent results. We are currently in the development stage of a VR application that will integrate a complex photorealistic 3D mesh that has been processed and decimated using the workflow to test the possible limitations and bottlenecking. We are collaborating on multiple ongoing projects.

Researches on photogrammetric data capturing with low-cost hardware are in development (in collaboration with Politecnico di Milano).

The fellowship so far resulted profitable and constructive allowing knowledge sharing on different topics, from more technical one to human centric ones.