DMTet
Title | Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis |
---|---|
Author | Tianchang Shen and Jun Gao and Kangxue Yin and Ming-Yu Liu and Sanja Fidler |
Conf/Jour | NeurIPS |
Year | 2021 |
Project | Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis (nvidia.com) |
Paper | Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis (readpaper.com) |
输入点云或低分辨率体素,提取特征后利用GAN网络,生成每个顶点的位置和SDF偏移值,得到优化后顶点的位置和SDF
结合显式与隐式表达的表示方法,利用MT,从隐式SDF中重建出显式mesh
RODIN
Title | Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion |
---|---|
Author | Tengfei Wang1† Bo Zhang2 Ting Zhang2 Shuyang Gu2 Jianmin Bao2 |
Conf/Jour | arXiv preprint |
Year | 2022 |
Project | RODIN Diffusion (microsoft.com) |
Paper | Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion (readpaper.com) |
TransHuman
Title | TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering |
---|---|
Author | Xiao Pan1,2,∗, Zongxin Yang1, Jianxin Ma2, Chang Zhou2, Yi Yang1,† |
Conf/Jour | ICCV |
Year | 2023 |
Project | TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering (pansanity666.github.io) |
Paper | TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering (readpaper.com) |
PointCloud Review
PointCloud
- Registration
- Surface Reconstruction
Follow
- Yochengliu/awesome-point-cloud-analysis: A list of papers and datasets about point cloud analysis (processing) (github.com)
- zhulf0804/3D-PointCloud: Papers and Datasets about Point Cloud. (github.com)
- XuyangBai/awesome-point-cloud-registration: A curated list of point cloud registration. (github.com)
Generative Models Review about 3D Reconstruction
Paper | Model | Input | Parameter/Pnum | GPU |
---|---|---|---|---|
DiT-3D | Diffusion Transformers | Voxelized PC | ||
PointFlow | AE flow-based | PointCloud | 1.61M | |
FlowGAN | GAN flow-based | Single Image | N = 2500 | A40 45GB |
BuilDiff | Diffusion models | Single Image | 1024 to 4096 | A40 45GB |
CCD-3DR | CDPM | Single Image | 8192 | 3090Ti 24GB |
SG-GAN | SG-GAN | Single Image | ||
HaP | Diffusion+SMPL+DepthEstimation | Single Image | 10000 | 4x3090Ti |
BuilDiff
Title | BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models |
---|---|
Author | Wei, Yao and Vosselman, George and Yang, Michael Ying |
Conf/Jour | ICCV |
Year | 2023 |
Project | weiyao1996/BuilDiff: BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models (github.com) |
Paper | BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models (readpaper.com) |
- 关注建筑物的重建,为 3D Diffusion Models 中添加了图片的信息嵌入(预训练了一个图片编码器)
- 两阶段的点云降噪模型,第一阶段关注全局,第二阶段关注细节
- 提出了两个自定义的新数据集,并在数据集上验证了本方法的优点
SG-GAN
Title | SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image |
---|---|
Author | Bowen Hu, Baiying Lei, Shuqiang Wang, Senior Member, IEEE |
Conf/Jour | arXiv |
Year | 2023 |
Project | |
Paper | SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image (readpaper.com) |
Stereoscopic-aware graph generative adversarial network (SG-GAN)
FlowGAN
Title | Flow-based GAN for 3D Point Cloud Generation from a Single Image |
---|---|
Author | Yao Wei (University of Twente), George Vosselman (“University of Twente, the Netherlands”), Michael Ying Yang (University of Twente)* |
Conf/Jour | BMVA |
Year | 2022 |
Project | Flow-based GAN for 3D Point Cloud Generation from a Single Image (mpg.de) |
Paper | Flow-based GAN for 3D Point Cloud Generation from a Single Image (readpaper.com) |
- flow-based explicit generative models for sampling point clouds with arbitrary resolutions
- Improving the detailed 3D structures of point clouds by leveraging the implicit generative adversarial networks (GANs).
DiT-3D
Title | DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation |
---|---|
Author | Shentong Mo 1, Enze Xie 2, Ruihang Chu 3, Lewei Yao 2,Lanqing Hong2, Matthias Nießner4, Zhenguo Li2 |
Conf/Jour | arXiv |
Year | 2023 |
Project | DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation |
Paper | DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation (readpaper.com) |
New 3D Diffusion Transformer Model, 在体素化的点云上运行 DDPM(Denoising diffusion probabilistic models) 的去噪过程
Greedy Grid Search
Title | Challenging universal representation of deep models for 3D point cloud registration |
---|---|
Author | Bojani\’{c}, David and Bartol, Kristijan and Forest, Josep and Gumhold, Stefan and Petkovi\’{c}, Tomislav and Pribani\’{c}, Tomislav |
Conf/Jour | BMVC |
Year | 2022 |
Project | DavidBoja/greedy-grid-search: [BMVC 2022 workshop] Greedy Grid Search: A 3D Registration Baseline (github.com) |
Paper | Challenging the Universal Representation of Deep Models for 3D Point Cloud Registration (readpaper.com) |
按步长穷举法,粗配准,需要根据ICP进行精配准
GeoTransformer
Title | Geometric Transformer for Fast and Robust Point Cloud Registration |
---|---|
Author | Zheng Qin1 Hao Yu2 Changjian Wang1 Yulan Guo1,3 Yuxing Peng1 Kai Xu1* |
Conf/Jour | CVPR |
Year | 2022 |
Project | qinzheng93/GeoTransformer: [CVPR2022] Geometric Transformer for Fast and Robust Point Cloud Registration (github.com) |
Paper | Geometric Transformer for Fast and Robust Point Cloud Registration (readpaper.com) |
pairwise registration models, only Ubuntu
SGHR
Title | Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting |
---|---|
Author | Haiping Wang and Yuan Liu and Zhen Dong and Yulan Guo and Yu-Shen Liu and Wenping Wang and Bisheng Yang |
Conf/Jour | CVPR |
Year | 2023 |
Project | WHU-USI3DV/SGHR: [CVPR 2023] Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting (github.com) |
Paper | Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting (readpaper.com) |
Issue:
How should I train my dataset? · Issue #4 · WHU-USI3DV/SGHR (github.com)
I think several point clouds of a single statue is not enough for training deep descriptors. I suggest to directly use pairwise registration models such as Geotrainsformer pre-trained on object-level datasets such as ModelNet40 to solve the pairwise registrations.
And adopt SGHR’s transformation synchronization section to solve the global consistent scan poses.
Basics about 3D Reconstruction
Math basic of 3D Reconstruction
Multi-view Human Body Reconstruction
Terminology/Jargon
- Human Radiance Fields
- 3D Clothed Human Reconstruction | Digitization
Application
- 三维重建设备:手持扫描仪或 360 度相机矩阵(成本高)
- 复刻一个迷你版的自己
Method
- Depth&Normal Estimation(2K2K)
- Implicit Function(PIFu or NeRF)
- Generative approach Generative Models Reconstruction
Awesome Human Body Reconstruction
Method | 泛化 | 数据集监督 | 提取 mesh 方式 | 获得纹理方式 |
---|---|---|---|---|
2k2k | 比较好 | (mesh+texture:)depth、normal、mask、rgb | 高质量深度图 —> 点云 —> mesh | 图片 rgb 贴图 |
PIFu | 比较好 | 点云(obj)、rgb(uv)、mask、camera | 占用场 —> MC —> 点云,mesh | 表面颜色场 |
NeRF | 差 | rgb、camera | 密度场 —> MC —> 点云,mesh | 体积颜色场 |
NeuS | 差 | rgb、camera | SDF —> MC —> 点云,mesh | 体积颜色场 |
ICON | 非常好 | rgb+mask、SMPL、法向量估计器 DR | 占用场 —> MC —> 点云,mesh | 图片 rgb 贴图 |
ECON | 非常好 | rgb+mask、SMPL、法向量估计器 DR | d-BiNI + SC(shape completion) | 图片 rgb 贴图 |
GeoMVSNet
Title | GeoMVSNet: Learning Multi-View Stereo With Geometry Perception |
---|---|
Author | Zhang, Zhe and Peng, Rui and Hu, Yuxi and Wang, Ronggang |
Conf/Jour | CVPR |
Year | 2023 |
Project | doubleZ0108/GeoMVSNet: [CVPR 23’] GeoMVSNet: Learning Multi-View Stereo with Geometry Perception (github.com) |
Paper | GeoMVSNet: Learning Multi-View Stereo with Geometry Perception (readpaper.com) |
MVSNet
Title | MVSNet: Depth Inference for Unstructured Multi-view Stereo |
---|---|
Author | Yao, Yao and Luo, Zixin and Li, Shiwei and Fang, Tian and Quan, Long |
Conf/Jour | ECCV |
Year | 2018 |
Project | YoYo000/MVSNet: MVSNet (ECCV2018) & R-MVSNet (CVPR2019) (github.com) |
Paper | MVSNet: Depth Inference for Unstructured Multi-view Stereo (readpaper.com) |
深度估计方法