NeRF相关的论文 at CVPR/ICCV/ECCV/NIPS/ICML/ICLR/SIGGRAPH
计算机视觉顶会2022截稿时间及会议时间_ijcai2024截稿日期-CSDN博客
ccf-deadlines (ccfddl.github.io)
| My post | Brief description | status |
|---|---|---|
| NeRF + Code | NeRF 原理 + 代码理解 | Completed |
| NeuS + Code | 表面重建方法 SDFNetwork | Completed |
| InstantNGP + Tiny-cuda-nn | 加速 NeRF 的训练和推理 | Completed（Tcnn） |
| Instant-nsr-pl + Code | Neus+Tcnn+NSR+pl | Completed |
| Instant-NSR + Code | 快速表面重建 | Completed |
| NeRO + Code | 考虑镜面和漫反射的体渲染函数 | In Processing |
| NeRF | 基于 Instant-nsr-pl 创建的项目 | Completed |
Related link : 3D Reconstruction | awesome-NeRF-papers
NeRF
ECCV 2020 Oral - Best Paper Honorable Mention
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2020 | NeRF:Representing Scenes as Neural Radiance Fields for View Synthesis | 初始文 | ECCV |
Review
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | A Critical Analysis of NeRF-Based 3D Reconstruction | 对比了Colmap摄影测量法和NeRF-based方法在3D Reconstruction中的优劣 | MDPI remote sensing |
| 2023 | Maftej/iisnerf (github.com) | 探索了NeRF在工业和机器人领域的应用 | None |
| 2023 | None | 2023年NeRF Review ref others | None |
| 2023 | A Review of Deep Learning-Powered Mesh Reconstruction Methods | 介绍了几种3D模型表示方法+回顾了将DL应用到Mesh重建中的方法 | None |
相机位姿精度问题
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | [2311.17119] Continuous Pose for Monocular Cameras in Neural Implicit Representation (arxiv.org) | 将单目相机姿势优化为时间连续函数 | |
| 2023 | Pose-Free Generalizable Rendering Transformer (zhiwenfan.github.io) | ||
| 2023 | [2310.02687] USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields (arxiv.org) | ||
| 2023 | [2309.11326] How to turn your camera into a perfect pinhole model (arxiv.org) | ||
| 2023 | [2312.08760] CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (arxiv.org) | AAAI | |
| 2023 | [2312.15238] NoPose-NeuS: Jointly Optimizing Camera Poses with Neural Implicit Surfaces for Multi-view Reconstruction (arxiv.org) | ||
| 2024 | KRONC: Keypoint-based Robust Camera Optimization for 3D Car Reconstruction | 利用物体的先验，只针对汽车场景的重建 | |
| 2024 ⭐ |
DUSt3R: Geometric 3D Vision Made Easy - Naver Labs Europe | 直接抛弃相机位姿，使用Transformer来预测点云，并进行点云配准，配准后反向求位姿。可以直接估计点云 | |
| Grounding Image Matching in 3D with MASt3R |
Other paper about camera pose
- How to turn your camera into a perfect pinhole model
图像拍摄质量问题
额外监督/先验
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2024 | [2401.03412] N$^{3}$-Mapping: Normal Guided Neural Non-Projective Signed Distance Fields for Large-scale 3D Mapping (arxiv.org) | 不同的法向监督 | |
| 2024 | [2401.12751] PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction (arxiv.org) | ||
| 2024 | maturk/dn-splatter: DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing | 3DGS + Depth and Normal Priors |
- Normal Map
重建精度/速度
Shadow&Highlight
Sparse images/Generalization
Large Scale Scene
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2020 | nerfplusplus: improves over nerf in 360 capture of unbounded scenes | 将背景的采样点表示为四维向量，与前景分别使用不同的MLP进行训练 | arXiv |
| 2022 | mip-NeRF 360 | 将单位球外的背景参数化，小型提议网络进行精采样，正则化dist消除小区域云雾 | CVPR |
Uncertainty in NeRF
| Year | Paper | 研究对象 | 研究内容 | 研究方法 | Important for me | |
|---|---|---|---|---|---|---|
| 2024 | [2405.02568] ActiveNeuS: Active 3D Reconstruction using Neural Implicit Surface Uncertainty | 3D scene reconstruction | Active learning主动学习 | Neural Implicit Surface Uncertainty | 图像渲染或几何不确定性 利用不同类型的不确定性可以减少在早期训练阶段因输入稀疏而出现的偏差 |
|
| 2024 | [2404.06727] Bayesian NeRF: Quantifying Uncertainty with Volume Density in Neural Radiance Fields | Volume Density in Neural Radiance Fields | quantifying uncertainty based on the geometric structure | Bayesian | 几何体积结构中的不确定性 | |
| 2024 | Bayes’ Rays | NeRF | BaysRays | Uncertainty Quantification | Bayes, 坐标perturbation | |
| 2024 ⭐ |
[Sources of Uncertainty in 3D Scene Reconstruction \ | PDF](https://arxiv.org/pdf/2409.06407) | NeRF and 3DGS | |||
| 2024 | ActNeRF | Robot Manipulators | Uncertainty-aware Active Learning of NeRF-based Object Models | Visual and Re-orientation Actions | 允许机器人在收集视觉观察结果的同时重新定向物体 | |
| 2024 | [2404.01400] NVINS: Robust Visual Inertial Navigation Fused with NeRF-augmented Camera Pose Regressor and Uncertainty Quantification | real-time and robust robotic tasks(机器人实时导航) | NeRF-augmented Camera Pose Regressor and Uncertainty Quantification | Fused | ||
| 2024 | [2403.18476] Modeling uncertainty for Gaussian Splatting | Gaussian Splatting | Modeling uncertainty | Variational Inference-based approach + Area Under Sparsification Error (AUSE) | 在图像渲染质量和不确定性估计精度方面都优于现有方法 | |
| 2024 | Neural Visibility Field for Uncertainty-Driven Active Mapping | NVF 自然会为未观察区域分配更高的不确定性，帮助机器人选择最具信息量的下一个视点 | ||||
| 2024 | Bayesian uncertainty analysis for underwater 3D reconstruction with neural radiance fields |
- Aleatoric Uncertainty
- random effects in the observations include varying lighting and motion blur
- Epistemic Uncertainty
- lack of information in the scene such as occluded (can be reduced by observing more data from new poses)
- Confounding outliers
- non-static scenes (passers by, moving object)
- Pose Uncertainty
- Sensitivity to the camera poses in the scene
原理
Activation Functions
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2020 | Implicit Neural Representations with Periodic Activation Functions | periodic activation functions dubbed sinusoidal representation networks or SIREN | ACM Transactions on Graphics (SIGGRAPH) |
Encoding
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2022 | Instant Neural Graphics Primitives with a Multiresolution Hash Encoding | 多分辨率哈希编码 | ACM Transactions on Graphics (SIGGRAPH) |
| 2023 | bennyguo/instant-nsr-pl | Neus+NeRF+Nerfacc+tcnn | None |
| 2023 | Neuralangelo: High-Fidelity Neural Surface Reconstruction | NGP_but数值梯度+Neus_SDF | IEEE Conference on Computer Vision and Pattern Recognition (CVPR) |
| 2023 | PermutoSDF | NGP_butPermutohedral lattice + Neus_SDF，曲率损失和颜色MLP正则解决镜面+无纹理区域，更光滑 | IEEE/CVF Conference on CVPR |
| 2023 | NeuDA | NGP_butDeformable Anchors+HPE + Neus | CVPR |
| 2023 | Zip-NeRF (jonbarron.info) | NGP + Mip-NeRF360 | ICCV |
| 2023 | Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields (wbhu.github.io) | Tri-MipRF encoding(TensoRF + NGP)+ Cone Sampling(Mip-NeRF) | ICCV |
| 2022 | TensoRF: Tensorial Radiance Fields (apchenstu.github.io) | TensoRF引入VM分解，提高重建的质量和速度，并减小了内存占用 | ECCV |
| 2023 | Zerg-Overmind/Strivec (github.com) | 局部张量CP分解为三向量+多尺度网格+占用网格采样方式 | ICCV |
| 2023 | 3D Gaussian Splatting for Real-Time Radiance Field Rendering (inria.fr) | 优化3D高斯点云，并使用splatting渲染方式，实现实时高分辨率训练+渲染 | SIGGRAPH |
| 2023 | BakedSDF | VolSDF+MarchingCube得到mesh，漫反射+高斯叶来渲染颜色 | SIGGRAPH |
| 2024 | [2402.16366] SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field | 空间预测编码，有效地消除空间冗余，以获得更好的压缩性能 |
Sampling
加速训练和渲染过程、提高新视图质量
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | NerfAcc Documentation — nerfacc 0.5.3 | 一种新的采样方法可以加速NeRF | arXiv |
| 2021 | mip-NeRF | 截头圆锥体采样方法+IPEncoding | ICCV |
| 2023 | Floaters No More: Radiance Field Gradient Scaling for Improved Near-Camera Training | 通过梯度缩放，解决基于NeRF重建场景中的背景塌陷和镜前漂浮物 | The Eurographics Association |
| 2023 | ProNeRF: Learning Efficient Projection-Aware Ray Sampling for Fine-Grained Implicit Neural Radiance Fields (kaist-viclab.github.io) | 投影感知采样（PAS） | |
| 2022 | AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields | 自适应采样方法 | ECCV |
| 2021 | NeuSample | Neural Sample Field光线上的采样点也是用网络进行生成 | |
| 2022 | NeRF-SR | Super-Sampling每个像素发射多条光线 | ACM Multimedia |
- Improving Neural Radiance Fields Using Near-Surface Sampling With Point Cloud Generation (arxiv.org)
- L0-Sampler: An L0 Model Guided Volume Sampling for NeRF 分段常数采样改为分段指数采样
Volume Rendering Function
- mikacuy/PL-NeRF: NeRF Revisited: Fixing Quadrature Instability in Volume Rendering, Neurips 2023 (github.com)
- NeRF 分段常数积分 —> PL-NeRF 分段线性积分
- StEik：基于 SDF 的隐式场优化问题(Neural SDF)，提出一个新的约束项
- Rethinking Directional Integration in Neural Radiance Fields，修改了 NeRF 的渲染方程
PointCloud
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | 3D Gaussian Splatting for Real-Time Radiance Field Rendering (inria.fr) | 优化3D高斯点云，并使用splatting渲染方式，实现实时高分辨率训练+渲染 | SIGGRAPH |
| 2023 | Globally Consistent Normal Orientation for Point Clouds by Regularizing the Winding-Number Field | 使得稀疏点云和薄壁点云法向量一致 | ACM Transactions on Graphics(SIGGRAPH) |
| 2023 | Neural Kernel Surface Reconstruction | 从点云中进行表面重建 | CVPR 2023 Highlight |
| 2022 | Point-NeRF: Point-based Neural Radiance Fields | 生成初始化点云，基于点云进行增删和体渲染 | CVPR 2022 Oral |
| 2024 | HashPoint | 结合rasterization and ray tracing | CVPR 2024 Highlight |
应用
从隐式场中提取表面
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | cong-yi/DualMesh-UDF (github.com) | Surface Extraction from Neural Unsigned Distance Fields | ICCV |
Framework
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | nerfstudio | 集成现有的NeRF方法 | ACM SIGGRAPH |
| 2022 | SDFStudio | 集成基于SDF的NeRF方法 | None |
有趣的应用
| Year | Title&Project Page | Brief Description | Conf/Jour |
|---|---|---|---|
| 2023 | Seeing the World through Your Eyes | 从人眼的倒影中重建物体 | None |
| 2023 | PAniC-3D Stylized Single-view 3D Reconstruction from Portraits of Anime Characters | 从插画风格角色肖像中重建3D | CVPR |
| 2023 | LERF: Language Embedded Radiance Fields | 用语言查询空间中的3D物体，并高亮显示 | ICCV 2023 (Oral) |
| 2024 | IPA-NeRF: Illusory Poisoning Attack Against Neural Radiance Fields | NeRF后门(Illusory Poisoning Attack)，在NeRF渲染的某个固定视图插入后门图像 |