The ceiling of the ‘Room of the Giants’ in Palazzo Te in Mantua, completed in 1535 by Giulio Romano, has only subtle vaulting. Here, the illusionistic depth contained in the thin layer of paint of the fresco is computed with machine learning and reprojected into a 3D point cloud.
Fresco Profondo explores how neural nets for computer vision – mainly developed for self-driving cars and smartphone cameras – can be used outside of their intended use cases. The LeRes model, developed by Simon Fraser University and Adobe Research, was implemented to compute grayscale 2D depth maps for early and late Baroque illusionistic frescos. These were used to construct coloured, 3D point clouds that digitally manifest the virtual, illusionistic depth. The resulting models can be used both for digital fabrication or virtual reality environments.
