VA-DepthNet: A Variational Approach to Single Image Depth Prediction
ICLR2023, notable-top-25%
Ce Liu, Suryansh Kumar, Shuhang Gu, Radu Timofte, Luc Van Gool
[CVL ETH Zürich, UESTC China, University of Würzburg 4KU Leuven]
https://arxiv.org/abs/2302.06556
https://github.com/cnexah/VA-DepthNet
https://openreview.net/forum?id=xjxUjHa_Wpa
ICLR2023 AC Paper Decision： 本文通过将经典的一阶变分约束集成到编码器-解码器网络中，提出VA DepthNet关注场景深度的一阶差异，而不是像素度量深度。核心技术发明是V-layer，其预测深度梯度和特征权重。标准基准测试的实验结果显示了显著的改进。总的来说，审稿人对论文持肯定态度。R#3Fa和R#qKm4真的很喜欢这篇论文，尤其是因为他们获得了最好的性能。尽管R#3Fva最初担心与使用自然图像进行深度学习之前的现有工作的比较缺失，但R #3Fva对作者的回应感到满意，其中报告了在NYU深度v2数据集上的新实验，并有显著改进。R#K2Ky在作者解决了过度声称的问题并澄清了V层的稀疏深度捕获和低分辨率后，将分数提高到略为正值。R#PkfN对这篇论文有一些技术问题，持轻微负面态度。作者在反驳中对他们进行了陈述，但R#PkfN没有回应作者的新结果，也没有回应AC的要求。AC认为该论文对CV社区，特别是将变分约束集成到网络的变分层做出了很好的贡献。
摘要：我们介绍了VA DepthNet，这是一种用于单图像深度预测（SIDP）问题的简单、有效和准确的深度神经网络方法。所提出的方法主张对这个问题使用经典的一阶变分约束。尽管用于SIDP的最先进的深度神经网络方法在受监督的环境中从图像中学习场景深度，但它们往往忽略了刚性场景空间中的宝贵不变量和先验，例如场景的规则性。本文的主要贡献是揭示了在SIDP任务的神经网络设计中，经典且有充分依据的变分约束的益处。结果表明，在场景空间中施加一阶变分约束以及流行的基于编码器-解码器的网络架构设计为监督的SIDP任务提供了极好的结果。施加的一阶变分约束使网络知道场景空间中的深度梯度，即规则性。本文通过对几个基准数据集（如KITTI、NYU Depth V2和SUN RGB-D）的广泛评估和消融分析，证明了所提出方法的有效性。测试时的VA DepthNet显示出与现有技术相比深度预测精度的显著提高，并且在场景空间中的高频区域也是准确的。在撰写本文时，我们的方法（标记为VA DepthNet）在KITTI深度预测评估集基准上进行测试时，显示了最先进的结果，是性能最好的公开方法。
We introduce VA-DepthNet, a simple, effective, and accurate deep neural network approach for the single-image depth prediction (SIDP) problem. The proposed approach advocates using classical first-order variational constraints for this problem. While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene. The paper's main contribution is to reveal the benefit of classical and well-founded variational constraints in the neural network design for the SIDP task. It is shown that imposing first-order variational constraints in the scene space together with popular encoder-decoder-based network architecture design provides excellent results for the supervised SIDP task. The imposed first-order variational constraint makes the network aware of the depth gradient in the scene space, i.e., regularity. The paper demonstrates the usefulness of the proposed approach via extensive evaluation and ablation analysis over several benchmark datasets, such as KITTI, NYU Depth V2, and SUN RGB-D. The VA-DepthNet at test time shows considerable improvements in depth prediction accuracy compared to the prior art and is accurate also at high-frequency regions in the scene space. At the time of writing this paper, our method -- labeled as VA-DepthNet, when tested on the KITTI depth-prediction evaluation set benchmarks, shows state-of-the-art results, and is the top-performing published approach.
网友评论