QiYun

Note

Posted on Edited on In
1.5k 6 mins.
Title Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting
Author Haiping Wang and Yuan Liu and Zhen Dong and Yulan Guo and Yu-Shen Liu and Wenping Wang and Bisheng Yang
Conf/Jour CVPR
Year 2023
Project WHU-USI3DV/SGHR: [CVPR 2023] Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting (github.com)
Paper Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting (readpaper.com)

image.png|666

Issue:
How should I train my dataset? · Issue #4 · WHU-USI3DV/SGHR (github.com)
I think several point clouds of a single statue is not enough for training deep descriptors. I suggest to directly use pairwise registration models such as Geotrainsformer pre-trained on object-level datasets such as ModelNet40 to solve the pairwise registrations.
And adopt SGHR’s transformation synchronization section to solve the global consistent scan poses.

Read more »

Posted on Edited on In
2.4k 9 mins.

Human.png|666

Terminology/Jargon

  • Human Radiance Fields
  • 3D Clothed Human Reconstruction | Digitization

Application

Method

  1. Depth&Normal Estimation(2K2K)
  2. Implicit Function(PIFu or NeRF)
  3. Generative approach Generative Models Reconstruction

Awesome Human Body Reconstruction

Method 泛化 数据集监督 提取 mesh 方式 获得纹理方式
2k2k 比较好 (mesh+texture:)depth、normal、mask、rgb 高质量深度图 —> 点云 —> mesh 图片 rgb 贴图
PIFu 比较好 点云(obj)、rgb(uv)、mask、camera 占用场 —> MC —> 点云,mesh 表面颜色场
NeRF 差 rgb、camera 密度场 —> MC —> 点云,mesh 体积颜色场
NeuS 差 rgb、camera SDF —> MC —> 点云,mesh 体积颜色场
ICON 非常好 rgb+mask、SMPL、法向量估计器 DR 占用场 —> MC —> 点云,mesh 图片 rgb 贴图
ECON 非常好 rgb+mask、SMPL、法向量估计器 DR d-BiNI + SC(shape completion) 图片 rgb 贴图
Read more »

Posted on Edited on In
1.7k 6 mins.
Title ECON: Explicit Clothed humans Obtained from Normals
Author Yuliang Xiu1 Jinlong Yang1 Xu Cao2 Dimitrios Tzionas3 Michael J. Black1
Conf/Jour CVPR
Year 2023
Project ECON: Explicit Clothed humans Optimized via Normal integration (xiuyuliang.cn)
Paper ECON: Explicit Clothed humans Obtained from Normals (readpaper.com)

image.png

姿态稳定(ICON在难的姿势下较好地重建)+灵活拓扑(ECON还可以较好地重建宽松的衣服)

缺陷：
image.png

Read more »

Posted on Edited on In
1.6k 6 mins.
Title ICON: Implicit Clothed humans Obtained from Normals
Author Yuliang Xiu1, Jinlong Yang1, Dimitrios Tzionas1,2, Michael J. Black1
Conf/Jour CVPR
Year 2022
Project ICON (mpg.de)
Paper ICON: Implicit Clothed humans Obtained from Normals (readpaper.com)

image.png

CVPR 2022 | ICON: 提高三维数字人重建的姿势水平 - 知乎 (zhihu.com)

输入：

  • 经过分割的着衣人类的RGB图像
  • 从图像估计得到的SMPL身体
    • SMPL身体用于指导ICON的两个模块：一个推断着衣人类的详细表面法线（前视图和后视图），另一个推断一个具有可见性感知的隐式表面（占用场的等值表面）
    • 迭代反馈循环使用推断出的详细法线来优化SMPL

缺点：

  • 宽松的衣服无法重建
  • 依赖HPS估计出的SMPL body

image.png

Read more »

Posted on Edited on In
1.9k 7 mins.
Title PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Author Shunsuke Saito1,3 Tomas Simon2 Jason Saragih2 Hanbyul Joo3
Conf/Jour CVPR
Year 2020
Project PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (shunsukesaito.github.io)
Paper PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (readpaper.com)

image.png

Encoder: stacked hourglass network
MLP

  • Coarse L：(257, 1024, 512, 256, 128, 1)
  • Fine H：(272, 512, 256, 128, 1)，将Coarse MLP的第四层输出$\Omega \in \mathbb{R}^{256}$作为输入
    表面法线网络：由9个残差块和4个下采样层组成
  • $\mathcal{L}_{N}=\mathcal{L}_{VGG}+\lambda_{l1}\mathcal{L}_{l1},$ 其中$L_{VGG}$为Johnson等人[17]提出的感知损失，$L_{l1}$为预测与真值法向之间的l1距离
Read more »

Posted on Edited on In
1.4k 5 mins.
Title PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Author Shunsuke Saito1,2 Zeng Huang1,2 Ryota Natsume3 * Shigeo Morishima3 Angjoo Kanazawa4Hao Li1,2,5
Conf/Jour ICCV
Year 2019
Project PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization (shunsukesaito.github.io)
Paper PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization (readpaper.com)

image.png
表面重建网络：stacked hourglass
纹理推断网络：由残差块组成的architecture of CycleGAN
隐函数网络：MLP

Read more »

Posted on Edited on In
867 3 mins.
Title Recovering Fine Details for Neural Implicit Surface Reconstruction
Author Decai Chen1, Peng Zhang1,2, Ingo Feldmann1, Oliver Schreer1, and Peter Eisert1,3
Conf/Jour WACV
Year 2023
Project fraunhoferhhi/D-NeuS: Recovering Fine Details for Neural Implicit Surface Reconstruction (WACV2023) (github.com)
Paper Recovering Fine Details for Neural Implicit Surface Reconstruction (readpaper.com)

image.png

Idea：两个额外的损失函数

  • 几何偏差损失：鼓励隐式SDF场和体渲染的亮度场之间的几何一致性
  • 多视图特征一致性损失：多个观察视图在表面点处的特征一致
Read more »