Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo

We present a modern solution to the multi-view photo-metric stereo problem (MVPS). Our work suitably exploitsthe image formation model in a MVPS experimental setupto recover the dense 3D reconstruction of an object fromimages. We procure the surface orientation using a photo-metric stereo (PS) image formation model and blend it witha multi-view neural radiance field representation to recoverthe object’s surface geometry. Contrary to the previousmulti-staged framework to MVPS, where the position, iso-depth contours, or orientation measurements are estimatedindependently and then fused later, our method is simple toimplement and realize. Our method performs neural ren-dering of multi-view images while utilizing surface normalsestimated by a deep photometric stereo network. We ren-der the MVPS images by considering the object’s surfacenormals for each 3D sample point along the viewing di-rection rather than explicitly using the density gradient inthe volume space via 3D occupancy information. We opti-mize the proposed neural radiance field representation forthe MVPS setup efficiently using a fully connected deep net-work to recover the 3D geometry of an object. Extensiveevaluation on the DiLiGenT-MV benchmark dataset showsthat our method performs better than the approaches thatperform only PS or only multi-view stereo (MVS) and pro-vides comparable results against the state-of-the-art multi-stage fusion methods.

• Machine Perception