• P-ISSN 0974-6846 E-ISSN 0974-5645

Indian Journal of Science and Technology

Article

Using Azure to construct recent architecture for visualize training in real-time
  • VIEWS 1003
  • PDF 1166

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v13i19.649

Year: 2020, Volume: 13, Issue: 23, Pages: 2355-2370

Original Article

Using Azure to construct recent architecture for visualize training in real-time

Rafeek Mamdouh1,2*, Hazem M El-Bakry2, Alaa Riad2, Nashaat El-Khamisy3

1Assistant Lecturer. Multimedia and Graphics Dept., Faculty of Filmmaking & Performing Arts, BADR University, Cairo, Egypt
2Information Systems Department, Faculty of Computer & Information Sciences, Mansoura University, Egypt
3Computers and Information Systems Dept, Sadat Academy, Cairo, Egypt

*Corresponding author
Email: [email protected]

Received Date:18 May 2020, Accepted Date:24 June 2020, Published Date:07 July 2020

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Background/Objectives: In recent years, visualization systems were not only entertainment but also essential in training in our various fields. They do not depend on fixed devices only. They use visual systems, such as the headmounted display that Microsoft developed as the Mixed Reality (MR) HoloLens. Its features are equipped with engines that allow the user to interact with the headset via oral orders to communicate remotely with specialists in the surgical field in real-life situations. The main objective of the study is to use the 3D anatomical information models, the Digital Imaging and Communications in Medicine (DICOM) file and all the patient's data, its registration on Azure cloud computing system, to obtain the necessary training and support in case of encountering any emergency before and during the surgical planning. Method: This study presents the application as divided into two stages of anatomical simulation: training for local and international trainees through MR. The first stage classified (DICOM) files to the 3D model using the machine learning and HoloLens emulator of anatomy operational structure. The second stage involves Microsoft Azure and stores on cloud network by Data Lake, Azure Cosmo DB, and utilization of the Azure Spatial Anchors service to access the trainee to locate it at any time through the ID that is displayed by the IoT Sensor. Finding: This study examines Mixed Reality technology, HoloLens, and Head Mount Display to show the expected potential results to improve the surgeon's actions in surgery. This examination finished by 3D displaying anatomical models because they allow the surgeon to access the best solutions in real-life situations during the process to assess the three-dimensional holograms related to patient imaging or surgical techniques.

Keywords: Mixed reality; 3D anatomical; HoloLens emulator; medical images; Azure spatial anchors; Azure cosmo DB

References

  1. Flavián C, Ibáñez-Sánchez S, Orús C. The impact of virtual, augmented and mixed reality technologies on the customer experience. Journal of Business Research. 2019;100:547–560. Available from: https://dx.doi.org/10.1016/j.jbusres.2018.10.050
  2. Gregory TM, Gregory J, Sledge J, Allard R, Mir O. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthopaedica. 2018;89(5):480–483. Available from: https://dx.doi.org/10.1080/17453674.2018.1506974
  3. El-Khamisy N, Amer K, Riad A, Mamdouh R. A Survey on Virtual Reality, Augmented Reality and Mixed Reality Techniques for Liver Surgical Operations and Training. Graphics, Vision and Image Processing Journal. 2018;18(2).
  4. Wursthorn S, Coelho AH, Staub G. Applications for mixed Reality. In: InXXth ISPRS Congress. p. 12–23.
  5. Costanza E, Kunz A, Fjeld M. Mixed reality: A survey. In: Human machine interaction. (pp. 47-68) Berlin, Heidelberg. Springer. 2009.
  6. ASD. Mixed Reality. 2016.
  7. Xiaoming D, Zhuo C. Teaching Virtual Experiments: From Course Design to Learning Assessment. International Journal of Online Engineering (iJOE). 2017;13(08):31. Available from: https://dx.doi.org/10.3991/ijoe.v13i08.6880
  8. Smolyaninova O, Bezyzvestnykh E. Implementing Teachers’ Training Technologies at a Federal University: E-portfolio, Digital Laboratory, PROLog Module System. International Journal of Online and Biomedical Engineering (iJOE). 2019;15(04):69. Available from: https://dx.doi.org/10.3991/ijoe.v15i04.9288
  9. Smith M, Maiti A, Maxwell AD, Kist AA. Augmented and Mixed Reality Features and Tools for Remote Laboratory Experiments. International Journal of Online Engineering (iJOE). 2016;12(07):45. Available from: https://dx.doi.org/10.3991/ijoe.v12i07.5851
  10. Documentation M. Microsoft Documentation Overview of Azure Digital Twins. 2020. Available from: https://docs.microsoft.com/en-us/azure/digital-twins/overview
  13. Dias JM, Monteiro L, Santos P, Silvestre R, Bastos R. Developing and authoring mixed Reality with mx toolkit. In: 2003 IEEE International Augmented Reality Toolkit Workshop . IEEE. p. 18–26.
  15. Deemer AD, Bradley CK, Ross NC, Natale DM, Itthipanichpong R, Werblin FS, et al. Low Vision Enhancement with Head-mounted Video Display Systems. Ovid Technologies (Wolters Kluwer Health). 2018. doi: 10.1097/opx.0000000000001278
  16. Höhl W, Behmel A, Kienzl T, Sandtner H, Gründler J. CG Mixed Reality Architectural Workspace. In: Proceedings of Real CORP. 2013.
  17. Moralioglu B. 2018.
  18. Frueh C, Sud A, Kwatra V. Headset removal for virtual and mixed Reality. In: ACM SIGGRAPH 2017 Talks . (pp. 1-2) 2017.
  19. Sauer IM, Queisner M, Tang P, Moosburner S, Hoepfner O, Horner R, et al. Mixed Reality in Visceral Surgery. Annals of Surgery. 2017;266:706–712. Available from: https://dx.doi.org/10.1097/sla.0000000000002448
  20. El-Seoud SA, Mady A, Rashed E. An Interactive Mixed Reality Ray Tracing Rendering Mobile Application of Medical Data in Minimally Invasive Surgeries. International Journal of Interactive Mobile Technologies (iJIM). 2019;13(03):29. Available from: https://dx.doi.org/10.3991/ijim.v13i03.9893
  22. Mateu J, Lasala M, Alamán X. Developing Mixed Reality Educational Applications: The Virtual Touch Toolkit. Sensors. 2015;15(9):21760–21784. Available from: https://dx.doi.org/10.3390/s150921760
  23. Fida B, Cutolo F, Franco Gd, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates in Surgery. 2018;70:389–400. Available from: https://dx.doi.org/10.1007/s13304-018-0567-8
  24. Azer SA, Azer S. 3D Anatomy Models and Impact on Learning: A Review of the Quality of the Literature. Health Professions Education. 2016;2(2):80–98. Available from: https://dx.doi.org/10.1016/j.hpe.2016.05.002
  26. Piccirilli M, Doretto G, Adjeroh D. A framework for analyzing the whole body surface area from a single view. PloS one. 2017;12(1):166749.
  27. Team WM, . Rotation Matrix. Rotation Matrix. (accessed ) Available from: http://mathworld.wolfram.com/RotationMatrix.html
  28. Trend G. 2018. Available from: https://trends.google.com/trends/explore
  29. Mamdouh R, M. H, Riad A, El-Khamisy N. Improve Speed Real-Time Rendering in Mixed Reality HOLOLENS during Training. International Journal of Advanced Computer Science and Applications. 2020;11(3). Available from: https://dx.doi.org/10.14569/ijacsa.2020.0110339

Copyright

© 2020 Mamdouh, El-Bakry, Riad, El-Khamisy. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. 
Published By Indian Society for Education and Environment (iSee)

  • 08 July 2020

Year: 2020, Volume: 13, Issue: 23

Article Metrics


Post Graduate Fellowship in Research Communication

More articles

DON'T MISS OUT!

Subscribe now for latest articles and news.