QiYun

DL	修仙.炼丹	example
框架	丹炉	PyTorch
网络	丹方.灵阵	CNN
数据集	灵材	MNIST
GPU	真火	NVIDIA
模型	成丹	.ckpt

Method	Pixel to Camera coordinate
NeRF	$\vec d = \begin{pmatrix} \frac{i-\frac{W}{2}}{f} \\ -\frac{j-\frac{H}{2}}{f} \\ -1 \\ \end{pmatrix}$ , $intrinsics = K = \begin{bmatrix} f & 0 & \frac{W}{2} \\ 0 & f & \frac{H}{2} \\ 0 & 0 & 1 \\ \end{bmatrix}$
Neus	$\vec d = intrinsics^{-1} \times pixel = \begin{bmatrix} \frac{1}{f} & 0 & -\frac{W}{2 \cdot f} \\ 0 & \frac{1}{f} & -\frac{H}{2 \cdot f} \\ 0 & 0 & 1 \\ \end{bmatrix} \begin{pmatrix} i \\ j \\ 1 \\ \end{pmatrix} = \begin{pmatrix} \frac{i-\frac{W}{2}}{f} \\ \frac{j-\frac{H}{2}}{f} \\ 1 \\ \end{pmatrix}$

InstantNGP

Posted on 2023-06-27 Edited on 2024-06-29 In 3DReconstruction/Multi-view/Implicit Function
7.7k 28 mins.

Title	Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
Author	Thomas Müller Alex Evans Christoph Schied Alexander Keller
Conf/Jour	ACM Transactions on Graphics (SIGGRAPH 2022)
Year	2022
Project	Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (nvlabs.github.io)
Paper	Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (readpaper.com)

哈希函数在cuda(cuda c++)中进行编程，不需深挖具体代码，初学只需理解多分辨率哈希编码思想。i.e.目前只需要学会使用tiny-cuda-nn即可：NVlabs/tiny-cuda-nn: Lightning fast C++/CUDA neural network framework (github.com)

哈希编码思想：
哈希编码后的输出值的数量与L(分辨率数量)、F(特征向量维度)有关，eg: L=16,F=2，则输入一个坐标xyz，根据多分辨率体素网格，插值出来L个特征值，每个特征值维度为2，因此输出值的维度为32

$\begin{aligned} N_{l} & :=\left\lfloor N_{\mathrm{min}}\cdot b^{l}\right\rfloor, \\ &b:=\exp\left(\frac{\ln N_{\mathrm{max}}-\ln N_{\mathrm{min}}}{L-1}\right). \end{aligned}$

L为分辨率数量，l为分辨率序号。示例中L=2，$N_{0}= N_{min} =2$ , $N_{1}= N_{max}= 3$ , $b = \frac{3}{2}$

L：多分辨率
T：每个分辨率下有T个特征向量
F：特征向量的维度
最小和最大分辨率：$N_{min} , N_{max}$
b：每个level的缩放per_level_scale $b= e^{\frac{ln(\frac{N_{max}}{N_{min}})}{L-1}}$

Git学习

Posted on 2023-06-26 Edited on 2024-11-11 In Learn
654 2 mins.

不要乱用 git reset —hard commit_id 回退 git commit 版本

NeRF目前进展

Posted on 2023-06-25 Edited on 2024-10-15 In 3DReconstruction/Multi-view/Implicit Function
3.2k 11 mins.

NeRF相关的论文 at CVPR/ICCV/ECCV/NIPS/ICML/ICLR/SIGGRAPH
计算机视觉顶会2022截稿时间及会议时间_ijcai2024截稿日期-CSDN博客
 ccf-deadlines (ccfddl.github.io)

My post	Brief description	status
NeRF + Code	NeRF 原理 + 代码理解	Completed
NeuS + Code	表面重建方法 SDFNetwork	Completed
InstantNGP + Tiny-cuda-nn	加速 NeRF 的训练和推理	Completed（Tcnn）
Instant-nsr-pl + Code	Neus+Tcnn+NSR+pl	Completed
Instant-NSR + Code	快速表面重建	Completed
NeRO + Code	考虑镜面和漫反射的体渲染函数	In Processing
NeRF	基于 Instant-nsr-pl 创建的项目	Completed

Related link : 3D Reconstruction | awesome-NeRF-papers

PyTorch学习笔记

Posted on 2023-06-21 Edited on 2024-09-23 In Learn
1.3k 5 mins.

基于Pytorch学习DL时，学习到的一些技巧/code

Python Scripts

Posted on 2023-06-20 Edited on 2024-09-20 In Project
561 2 mins.

Python写的一些小工具

NeRF代码理解

Posted on 2023-06-15 Edited on 2024-06-29 In 3DReconstruction/Multi-view/Implicit Function
2.1k 8 mins.

NeRF代码(基于Pytorch)。流程图：train流程图 and 渲染流程图 (基于Drawio)

Nerfstudio——简化NeRF流程

Posted on 2023-06-15 Edited on 2024-06-29 In 3DReconstruction/Multi-view/Implicit Function
1.2k 4 mins.

Title	Nerfstudio: A Modular Framework for Neural Radiance Field Development
Author	Tancik, Matthew and Weber, Ethan and Ng, Evonne and Li, Ruilong and Yi, Brentand Kerr, Justin and Wang, Terrance and Kristoffersen, Alexander and Austin,Jake and Salahi, Kamyar and Ahuja, Abhik and McAllister, David and Kanazawa,Angjoo
Conf/Jour	ACM SIGGRAPH 2023 Conference Proceedings
Year	2023
Project	nerfstudio-project/nerfstudio: A collaboration friendly studio for NeRFs (github.com)
Paper	Nerfstudio: A Modular Framework for Neural Radiance Field Development (readpaper.com)

Nerfstudio提供了一个简单的API，可以简化创建、训练和测试NeRF的端到端过程。该库通过将每个组件模块化，支持更易于理解的NeRF实现。通过更模块化的NeRF，我们希望为探索这项技术提供更用户友好的体验。

Instant-nsr-pl

Posted on 2023-06-14 Edited on 2024-11-11 In 3DReconstruction/Multi-view/Implicit Function/Efficiency
2.1k 8 mins.

使用Instant-ngp中的编码技术，使Neus可以更快的进行inference，大概只需要5~10min生成一个模型

NVlabs/instant-ngp: Instant neural graphics primitives: lightning fast NeRF and more (github.com)
zhaofuq/Instant-NSR: Pytorch implementation of fast surface resconstructor (github.com)
kwea123/ngp_pl: Instant-ngp in pytorch+cuda trained with pytorch-lightning (high quality with high speed, with only few lines of legible code) (github.com)

neus：对无纹理的区域处理的很差

Title	NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
Author	Peng Wang Lingjie Liu Yuan Liu Christian Theobalt Taku Komura Wenping Wang
Conf/Jour	NeurIPS 2021 (Spotlight)
Year	2021
Project	NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction (lingjie0206.github.io)
Paper	NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction (readpaper.com)

NeRF原理

Posted on 2023-06-14 Edited on 2024-06-29 In 3DReconstruction/Multi-view/Implicit Function/NeRF-based
3.4k 12 mins.

Title	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Author	Ben MildenhallPratul P. SrinivasanMatthew Tancik*Jonathan T. Barron Ravi Ramamoorthi Ren Ng
Conf/Jour	ECCV 2020 Oral - Best Paper Honorable Mention
Year	2020
Project	NeRF: Neural Radiance Fields (matthewtancik.com)
Paper	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (readpaper.com)

NeRF（Neural Radiance Fields）是一种用于生成逼真三维场景的计算机图形学方法。通过神经网络对场景中的每个空间点进行建模，NeRF可以估计每个点的颜色和密度信息。利用渲染方程，NeRF能够合成高质量的逼真图像。相较于传统的渲染方法，NeRF能够处理复杂的光照和反射效果，广泛应用于虚拟现实、增强现实、电影制作和游戏开发等领域。然而，NeRF方法仍面临一些挑战，如计算复杂度和对训练数据的依赖性。研究人员正在不断改进NeRF，以提高其效率和扩展性。