• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology


Indian Journal of Science and Technology

Year: 2020, Volume: 13, Issue: 23, Pages: 2355-2370

Original Article

Using Azure to construct recent architecture for visualize training in real-time

Received Date:18 May 2020, Accepted Date:24 June 2020, Published Date:07 July 2020


Background/Objectives: In recent years, visualization systems were not only entertainment but also essential in training in our various fields. They do not depend on fixed devices only. They use visual systems, such as the headmounted display that Microsoft developed as the Mixed Reality (MR) HoloLens. Its features are equipped with engines that allow the user to interact with the headset via oral orders to communicate remotely with specialists in the surgical field in real-life situations. The main objective of the study is to use the 3D anatomical information models, the Digital Imaging and Communications in Medicine (DICOM) file and all the patient's data, its registration on Azure cloud computing system, to obtain the necessary training and support in case of encountering any emergency before and during the surgical planning. Method: This study presents the application as divided into two stages of anatomical simulation: training for local and international trainees through MR. The first stage classified (DICOM) files to the 3D model using the machine learning and HoloLens emulator of anatomy operational structure. The second stage involves Microsoft Azure and stores on cloud network by Data Lake, Azure Cosmo DB, and utilization of the Azure Spatial Anchors service to access the trainee to locate it at any time through the ID that is displayed by the IoT Sensor. Finding: This study examines Mixed Reality technology, HoloLens, and Head Mount Display to show the expected potential results to improve the surgeon's actions in surgery. This examination finished by 3D displaying anatomical models because they allow the surgeon to access the best solutions in real-life situations during the process to assess the three-dimensional holograms related to patient imaging or surgical techniques.

Keywords: Mixed reality; 3D anatomical; HoloLens emulator; medical images; Azure spatial anchors; Azure cosmo DB


  1. Flavián C, Ibáñez-Sánchez S, Orús C. The impact of virtual, augmented and mixed reality technologies on the customer experience. Journal of Business Research. 2019;100:547–560. Available from: https://dx.doi.org/10.1016/j.jbusres.2018.10.050
  2. Gregory TM, Gregory J, Sledge J, Allard R, Mir O. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthopaedica. 2018;89(5):480–483. Available from: https://dx.doi.org/10.1080/17453674.2018.1506974
  3. El-Khamisy N, Amer K, Riad A, Mamdouh R. A Survey on Virtual Reality, Augmented Reality and Mixed Reality Techniques for Liver Surgical Operations and Training. Graphics, Vision and Image Processing Journal. 2018;18(2).
  4. Wursthorn S, Coelho AH, Staub G. Applications for mixed Reality. In: InXXth ISPRS Congress. p. 12–23.
  5. Costanza E, Kunz A, Fjeld M. Mixed reality: A survey. In: Human machine interaction. (pp. 47-68) Berlin, Heidelberg. Springer. 2009.
  6. ASD. Mixed Reality. 2016.
  7. Xiaoming D, Zhuo C. Teaching Virtual Experiments: From Course Design to Learning Assessment. International Journal of Online Engineering (iJOE). 2017;13(08):31. Available from: https://dx.doi.org/10.3991/ijoe.v13i08.6880
  8. Smolyaninova O, Bezyzvestnykh E. Implementing Teachers’ Training Technologies at a Federal University: E-portfolio, Digital Laboratory, PROLog Module System. International Journal of Online and Biomedical Engineering (iJOE). 2019;15(04):69. Available from: https://dx.doi.org/10.3991/ijoe.v15i04.9288
  9. Smith M, Maiti A, Maxwell AD, Kist AA. Augmented and Mixed Reality Features and Tools for Remote Laboratory Experiments. International Journal of Online Engineering (iJOE). 2016;12(07):45. Available from: https://dx.doi.org/10.3991/ijoe.v12i07.5851
  10. Documentation M. Microsoft Documentation Overview of Azure Digital Twins. 2020. Available from: https://docs.microsoft.com/en-us/azure/digital-twins/overview
  11. Dias JM, Monteiro L, Santos P, Silvestre R, Bastos R. Developing and authoring mixed Reality with mx toolkit. In: 2003 IEEE International Augmented Reality Toolkit Workshop . IEEE. p. 18–26.
  12. Deemer AD, Bradley CK, Ross NC, Natale DM, Itthipanichpong R, Werblin FS, et al. Low Vision Enhancement with Head-mounted Video Display Systems. Ovid Technologies (Wolters Kluwer Health). 2018. doi: 10.1097/opx.0000000000001278
  13. Höhl W, Behmel A, Kienzl T, Sandtner H, Gründler J. CG Mixed Reality Architectural Workspace. In: Proceedings of Real CORP. 2013.
  14. Moralioglu B. 2018.
  15. Frueh C, Sud A, Kwatra V. Headset removal for virtual and mixed Reality. In: ACM SIGGRAPH 2017 Talks . (pp. 1-2) 2017.
  16. Sauer IM, Queisner M, Tang P, Moosburner S, Hoepfner O, Horner R, et al. Mixed Reality in Visceral Surgery. Annals of Surgery. 2017;266:706–712. Available from: https://dx.doi.org/10.1097/sla.0000000000002448
  17. El-Seoud SA, Mady A, Rashed E. An Interactive Mixed Reality Ray Tracing Rendering Mobile Application of Medical Data in Minimally Invasive Surgeries. International Journal of Interactive Mobile Technologies (iJIM). 2019;13(03):29. Available from: https://dx.doi.org/10.3991/ijim.v13i03.9893
  18. Mateu J, Lasala M, Alamán X. Developing Mixed Reality Educational Applications: The Virtual Touch Toolkit. Sensors. 2015;15(9):21760–21784. Available from: https://dx.doi.org/10.3390/s150921760
  19. Fida B, Cutolo F, Franco Gd, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates in Surgery. 2018;70:389–400. Available from: https://dx.doi.org/10.1007/s13304-018-0567-8
  20. Azer SA, Azer S. 3D Anatomy Models and Impact on Learning: A Review of the Quality of the Literature. Health Professions Education. 2016;2(2):80–98. Available from: https://dx.doi.org/10.1016/j.hpe.2016.05.002
  21. Piccirilli M, Doretto G, Adjeroh D. A framework for analyzing the whole body surface area from a single view. PloS one. 2017;12(1):166749.
  22. Team WM, . Rotation Matrix. Rotation Matrix. (accessed ) Available from: http://mathworld.wolfram.com/RotationMatrix.html
  23. Trend G. 2018. Available from: https://trends.google.com/trends/explore
  24. Mamdouh R, M. H, Riad A, El-Khamisy N. Improve Speed Real-Time Rendering in Mixed Reality HOLOLENS during Training. International Journal of Advanced Computer Science and Applications. 2020;11(3). Available from: https://dx.doi.org/10.14569/ijacsa.2020.0110339


© 2020 Mamdouh, El-Bakry, Riad, El-Khamisy. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. 
Published By Indian Society for Education and Environment (iSee)


Subscribe now for latest articles and news.