Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation

1NVIDIA, 2Georgia Tech.
Thanks to Rogelio Olguin for the art asset.

Abstract

We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF target poses. Given a single observed RGB image of the target, we can predict the translation and rotation of the camera by minimizing the residual between pixels rendered from a fastNeRF model and pixels in the observed image. We integrate a momentum-based camera extrinsic optimization procedure intoInstant Neural Graphics Primitives, a recent exceptionally fastNeRF implementation. By introducing parallel Monte Carlo sampling into the pose estimation task, our method over comes local minima and improves efficiency in a more extensive search space. We also show the importance of adopting a more robust pixel-based loss function to reduce error. Experiments demonstrate that our method can achieve improved generalization and robustness on both synthetic and real-world benchmarks.

Video

Results

BibTeX

@inproceedings{lin2023icra:pnerf,
   author = "Yunzhi Lin and Thomas M{\"u}ller and Jonathan Tremblay and Bowen Wen and Stephen Tyree and Alex Evans and Patricio A. Vela and Stan Birchfield",
   title = "Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation",
   booktitle = "ICRA",
   year = 2023
}