• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology


Indian Journal of Science and Technology

Year: 2023, Volume: 16, Issue: 44, Pages: 4054-4062

Original Article

3D Human Reconstruction from A Single Image Using Parametric Model-Conditioned Implicit Representation

Received Date:30 June 2023, Accepted Date:13 October 2023, Published Date:26 November 2023


Background: PaMIR is a novel approach for image-based human reconstruction that utilizes a parametric model-conditioned implicit representation. This method enables the generation of a complete 3D mesh of a human body from a single input image. It uses a neural network that is conditioned on a parametric model of the human body to produce an implicit representation of the 3D surface. Objectives: To develop a novel approach for image based human reconstruction by training neural network and to generate high quality images. Method: In our PaMIR-based reconstruction framework, a novel deep neural network is proposed to regularize the free-form deep implicit function using the semantic features of the parametric model, which improves the generalization ability under the scenarios of challenging poses and various clothing topologies. Findings: The quantitative comparison shows that PaMIR method outperforms the state-of-the-art methods in terms of surface reconstruction accuracy. The errors are also provided when ground-truth SMPL annotations are available to present the upper limit of the reconstruction accuracy if the SMPL estimation is perfect. Overall, this method is more general, more robust and more accurate than HMD, Molding Humans, Deep Human and PIFu. Novelty: A novel depth-ambiguity-aware training loss is further integrated to resolve depth ambiguities and enable successful surface detail reconstruction with imperfect body reference. Finally, we propose a body reference optimization method to improve the parametric model estimation accuracy and to enhance the consistency between the parametric model and the implicit function. With the PaMIR representation, our framework can be easily extended to multi-image input scenarios without the need of multi-camera calibration and pose synchronization.

Keywords: Parametric, framework, Non­Parametric, HMD, SMPL, Tex2Shape


  1. Zheng Z, Yu T, Liu Y, Dai Q. PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;44(6):3170–3184. Available from: https://doi.org/10.1109/TPAMI.2021.3050505
  2. Chen L, Peng S, Zhou X. Towards efficient and photorealistic 3D human reconstruction: A brief survey. Visual Informatics. 2021;5(4):11–19. Available from: https://doi.org/10.1016/j.visinf.2021.10.003
  3. Saxena A, Sun M, Ng AY. Make3d: Learning 3d scene structure from a single still image. IEEE transactions on pattern analysis and machine intelligence. 2021;31:824–840. Available from: https://doi.org/10.1109/TPAMI.2008.132
  4. Priya ES, Velvizhy P, Deepa KA. Neural Style Transfer with distortion handling for Audio and Image. International Conference on Data Science. 2022;p. 1–10. Available from: https://doi.org/10.1109/ICDSAAI55433.2022.10028972
  5. Chen Z, Zhang H. Learning Implicit Fields for Generative Shape Modeling. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019;p. 5932–5941. Available from: https://doi.org/10.1109/CVPR.2019.00609
  6. Zheng Z, Yu T, Wei Y, Dai Q, Liu Y. DeepHuman: 3D human reconstruction from a single image. Proceedings of the IEEE International Conference on Computer Vision. 2021;p. 7738–7748. Available from: https://doi.org/10.48550/arXiv.1903.06473
  7. Jiang C, Sud A, Makadia A, Huang J, Niebner M, Funkhouser T. Local Implicit Grid Representations for 3D Scenes. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020;p. 6000–6009. Available from: https://doi.org/10.1109/CVPR42600.2020.00604
  8. Guler RA, Kokkinos I. HoloPose: Holistic 3D Human Reconstruction In-The-Wild. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019;p. 10876–10886. Available from: https://doi.org/10.1109/CVPR.2019.01114
  9. Chibane J, Alldieck T, Pons-Moll G. Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020;p. 6968–6979. Available from: https://doi.org/10.1109/CVPR42600.2020.00700
  10. Gabeur V, Franco JS, Martin X, Schmid C, Rogez G. Moulding humans: Non-parametric 3D human shape estimation from single images. arxiv. 2021;p. 2232–2241. Available from: https://arxiv.org/pdf/1908.00439.pdf
  11. Mescheder L, Oechsle M, Niemeyer M, Nowozin S, Geiger A. Occupancy Networks: Learning 3D Reconstruction in Function Space. Available from: https://doi.org/10.1109/CVPR.2019.00459
  12. Kolotouros N, Pavlakos G, Black M, Daniilidis K. Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019;p. 2252–2261. Available from: https://doi.org/10.1109/ICCV.2019.00234
  13. Velvizhy P, Pravi A, Selvi M, Ganapathy S, Kannan A. Fuzzy-based review rating prediction in e-commerce. 2020. Available from: https://doi.org/10.1504/IJBIDM.2020.108034
  14. Gwak J, Christopher B, Choy D, Xu K, Chen S, Savarese. 3D-R2N2: A unified approach for single and multi-view 3D object reconstruction. 2020. Available from: https://doi.org/10.1007/978-3-319-46484-8_38
  15. Bogo F, Kanazawa A, Lassner C, Gehler P, Romero J, Black MJ. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. Computer Vision and Pattern Recognition. 2020. Available from: https://doi.org/10.48550/arXiv.1607.08128


© 2023 Priya et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)


Subscribe now for latest articles and news.