TensoRF

TensoRF: Tensorial Radiance Fields

Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su

https://arxiv.org/abs/2203.09517

https://apchenstu.github.io/TensoRF/

ECCV 2022 (poster, final scores: 1, 1, 1)

17 Mar 2022

We present TensoRF, a novel approach to model and reconstruct radiance fields. Unlike NeRF that purely uses MLPs, we model the radiance field of a scene as a 4D tensor, which represents a 3D voxel grid with per-voxel multi-channel features. Our central idea is to factorize the 4D scene tensor into multiple compact low-rank tensor components. We demonstrate that applying traditional CP decomposition -- that factorizes tensors into rank-one components with compact vectors -- in our framework leads to improvements over vanilla NeRF. To further boost performance, we introduce a novel vector-matrix (VM) decomposition that relaxes the low-rank constraints for two modes of a tensor and factorizes tensors into compact vector and matrix factors. Beyond superior rendering quality, our models with CP and VM decompositions lead to a significantly lower memory footprint in comparison to previous and concurrent works that directly optimize per-voxel features. Experimentally, we demonstrate that TensoRF with CP decomposition achieves fast reconstruction (<30 min) with better rendering quality and even a smaller model size (<4 MB) compared to NeRF. Moreover, TensoRF with VM decomposition further boosts rendering quality and outperforms previous state-of-the-art methods, while reducing the reconstruction time (<10 min) and retaining a compact model size (<75 MB).

我们提出了TensoRF，一种新的建模和重建辐射场的方法。与纯粹使用MLP的NeRF不同，我们将场景的辐射场建模为4D张量，表示具有每体素多通道特征的三维体素网格。我们的中心思想是将4D场景张量分解为多个紧凑的低秩张量分量。我们证明了在我们的框架中应用传统的CP分解（将张量分解为具有紧向量的秩一分量）可以改进vanilla NeRF。为了进一步提高性能，我们引入了一种新的向量矩阵分解，该分解放松了张量两种模式的低秩约束，并将张量分解为紧凑的向量和矩阵因子。除了卓越的渲染质量外，与之前和同时进行的直接优化每体素特征的工作相比，我们采用CP和VM分解的模型大大降低了内存占用。实验证明，与NeRF相比，具有CP分解的TensoRF实现了快速重建（<30分钟），具有更好的渲染质量，甚至更小的模型尺寸（<4 MB）。此外，具有VM分解的TensoRF进一步提高了渲染质量，优于以前最先进的方法，同时减少了重建时间（<10分钟），并保持了紧凑的模型大小（<75 MB）。