Neus代码的理解

NeRF与Neus相机坐标系的对比:

image.png

Method Pixel to Camera coordinate
NeRF $\vec d = \begin{pmatrix} \frac{i-\frac{W}{2}}{f} \\ -\frac{j-\frac{H}{2}}{f} \\ -1 \\ \end{pmatrix}$ , $intrinsics = K = \begin{bmatrix} f & 0 & \frac{W}{2} \\ 0 & f & \frac{H}{2} \\ 0 & 0 & 1 \\ \end{bmatrix}$
Neus $\vec d = intrinsics^{-1} \times pixel = \begin{bmatrix} \frac{1}{f} & 0 & -\frac{W}{2 \cdot f} \\ 0 & \frac{1}{f} & -\frac{H}{2 \cdot f} \\ 0 & 0 & 1 \\ \end{bmatrix} \begin{pmatrix} i \\ j \\ 1 \\ \end{pmatrix} = \begin{pmatrix} \frac{i-\frac{W}{2}}{f} \\ \frac{j-\frac{H}{2}}{f} \\ 1 \\ \end{pmatrix}$
Read more »

Title Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
Author Thomas Müller Alex Evans Christoph Schied Alexander Keller
Conf/Jour ACM Transactions on Graphics (SIGGRAPH 2022)
Year 2022
Project Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (nvlabs.github.io)
Paper Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (readpaper.com)

哈希函数在cuda(cuda c++)中进行编程,不需深挖具体代码,初学只需理解多分辨率哈希编码思想。i.e.目前只需要学会使用tiny-cuda-nn即可:NVlabs/tiny-cuda-nn: Lightning fast C++/CUDA neural network framework (github.com)

哈希编码思想:
哈希编码后的输出值的数量与L(分辨率数量)、F(特征向量维度)有关,eg: L=16,F=2,则输入一个坐标xyz,根据多分辨率体素网格,插值出来L个特征值,每个特征值维度为2,因此输出值的维度为32

image.png

L为分辨率数量,l为分辨率序号。示例中L=2,$N_{0}= N_{min} =2$ , $N_{1}= N_{max}= 3$ , $b = \frac{3}{2}$

  • L:多分辨率
  • T:每个分辨率下有T个特征向量
  • F:特征向量的维度
  • 最小和最大分辨率:$N_{min} , N_{max}$
  • b:每个level的缩放per_level_scale $b= e^{\frac{ln(\frac{N_{max}}{N_{min}})}{L-1}}$

image.png

Read more »

不要乱用 git reset —hard commit_id 回退 git commit 版本

Read more »

NeRF相关的论文 at CVPR/ICCV/ECCV/NIPS/ICML/ICLR/SIGGRAPH
计算机视觉顶会2022截稿时间及会议时间_ijcai2024截稿日期-CSDN博客
ccf-deadlines (ccfddl.github.io)

My post Brief description status
NeRF + Code NeRF 原理 + 代码理解 Completed
NeuS + Code 表面重建方法 SDFNetwork Completed
InstantNGP + Tiny-cuda-nn 加速 NeRF 的训练和推理 Completed(Tcnn)
Instant-nsr-pl + Code Neus+Tcnn+NSR+pl Completed
Instant-NSR + Code 快速表面重建 Completed
NeRO + Code 考虑镜面和漫反射的体渲染函数 In Processing
NeRF 基于 Instant-nsr-pl 创建的项目 Completed

Related link : 3D Reconstruction | awesome-NeRF-papers

Read more »

Title Nerfstudio: A Modular Framework for Neural Radiance Field Development
Author Tancik, Matthew and Weber, Ethan and Ng, Evonne and Li, Ruilong and Yi, Brentand Kerr, Justin and Wang, Terrance and Kristoffersen, Alexander and Austin,Jake and Salahi, Kamyar and Ahuja, Abhik and McAllister, David and Kanazawa,Angjoo
Conf/Jour ACM SIGGRAPH 2023 Conference Proceedings
Year 2023
Project nerfstudio-project/nerfstudio: A collaboration friendly studio for NeRFs (github.com)
Paper Nerfstudio: A Modular Framework for Neural Radiance Field Development (readpaper.com)

Nerfstudio提供了一个简单的API,可以简化创建、训练和测试NeRF的端到端过程。该库通过将每个组件模块化,支持更易于理解的NeRF实现。通过更模块化的NeRF,我们希望为探索这项技术提供更用户友好的体验。

Read more »

使用Instant-ngp中的编码技术,使Neus可以更快的进行inference,大概只需要5~10min生成一个模型

NVlabs/instant-ngp: Instant neural graphics primitives: lightning fast NeRF and more (github.com)
zhaofuq/Instant-NSR: Pytorch implementation of fast surface resconstructor (github.com)
kwea123/ngp_pl: Instant-ngp in pytorch+cuda trained with pytorch-lightning (high quality with high speed, with only few lines of legible code) (github.com)

neus:对无纹理的区域处理的很差

Read more »

实现了三维重建:从多视角图片中重建出了 mesh 模型

Read more »

Title NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Author Ben Mildenhall*Pratul P. Srinivasan*Matthew Tancik*Jonathan T. BarronRavi RamamoorthiRen Ng
Conf/Jour ECCV 2020 Oral - Best Paper Honorable Mention
Year 2020
Project NeRF: Neural Radiance Fields (matthewtancik.com)
Paper NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (readpaper.com)

NeRF(Neural Radiance Fields)是一种用于生成逼真三维场景的计算机图形学方法。通过神经网络对场景中的每个空间点进行建模,NeRF可以估计每个点的颜色和密度信息。利用渲染方程,NeRF能够合成高质量的逼真图像。相较于传统的渲染方法,NeRF能够处理复杂的光照和反射效果,广泛应用于虚拟现实、增强现实、电影制作和游戏开发等领域。然而,NeRF方法仍面临一些挑战,如计算复杂度和对训练数据的依赖性。研究人员正在不断改进NeRF,以提高其效率和扩展性。

Read more »