دوره 17، شماره 2 - ( مجله کنترل، جلد 17، شماره 2، تابستان 1402 )
جلد 17 شماره 2,1402 صفحات 46-25 |
برگشت به فهرست نسخه ها
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ahmadi M J, Allahkaram M S, Abdi P, Mohammadi S, D. Taghirad H. Image Processing and Machine Vision in Surgery and Its Training. JoC 2023; 17 (2) :25-46
URL: http://joc.kntu.ac.ir/article-1-999-fa.html
URL: http://joc.kntu.ac.ir/article-1-999-fa.html
احمدی محمد جواد، الله کرم محمد سینا، عبدی پریسا، محمدی سید فرزاد، تقی راد حمید رضا. پردازش تصویر و بینایی ماشین در جراحی و آموزش آن. مجله کنترل. 1402; 17 (2) :25-46
محمد جواد احمدی1 
، محمد سینا الله کرم1 ، پریسا عبدی2 ، سید فرزاد محمدی2 ، حمید رضا تقی راد* 3
، محمد سینا الله کرم1 ، پریسا عبدی2 ، سید فرزاد محمدی2 ، حمید رضا تقی راد* 3
1- گروه کنترل، دانشگاه صنعتی خواجه نصیرالدین طوسی،تهران، ایران
2- مرکز تحقیقات چشم پزشکی ترجمانی، بیمارستان فارابی، دانشگاه علوم پزشکی تهران ، ایران
3- دانشکده مهندسی برق، گروه کنترل، دانشگاه صنعتی خواجه نصیرالدین طوسی ،تهران، ایران
2- مرکز تحقیقات چشم پزشکی ترجمانی، بیمارستان فارابی، دانشگاه علوم پزشکی تهران ، ایران
3- دانشکده مهندسی برق، گروه کنترل، دانشگاه صنعتی خواجه نصیرالدین طوسی ،تهران، ایران
چکیده: (1936 مشاهده)
با پیشرفت‌های هوش مصنوعی در در دهۀ اخیر، استفاده از دادۀ تصویری و ویدیویی و فن‌آوری‌های مبتنی بر پردازش تصویر برای خودکارسازی روش‌های جراحی و آموزش آن، رونق یافته است. امروزه در بیش‌ترِ اتاق‌های عمل از یک یا چند دوربین و یا دستگاه ثبت اطلاعات استفاده می‌شود تا دادۀ مهم پزشکی برای انجام تحلیل‌های بعدی ذخیره شوند. از این اطلاعات تصویری می‌توان برای طراحی و توسعۀ سامانه‌های خودکار هدایت تصویری با هدف کمک به پزشک متخصص حین جراحی و آموزش آن استفاده کرد. هم‌چنین، این سامانه‌ها می‌توانند به‌عنوان مغز ابزارهای رباتیکی کمک‌جراح فعالیت کنند. یک سامانۀ هدایت تصویری جراحی نیاز به قسمت‌های مختلفی دارد. از مهم‌ترین این قسمت‌ها می‌توان به تشخیص، بخش‌بندی و ردیابی ابزارها و نواحی مهم جراحی، تشخیص مراحل، حرکات و ژست‌ها، و تشخیص مهارت‌های جراحی اشاره کرد. خودکارکردن این بخش‌ها با استفاده از پردازش تصویر و بینایی ماشین کمک می‌کند، تا سامانه درک مستقل و عمیقی از صحنۀ جراحی داشته باشد. در این مقاله ابتدا تعدادی از مجموعه‌داده‌های تصویری مهم مربوط به جراحی معرفی شده، و سپس شماری از پژوهش‌های اثرگذار در زمینۀ پردازش تصویر و بینایی ماشین در کاربردهای ذکرشده با هدف ایجاد اجزای یک سامانۀ خودکار هدایت تصویری جراحی، معرفی شده و زمینه های تحقیقاتی پیش رو معرفی می‌شوند.
واژه‌های کلیدی: بینایی ماشین در جراحی، سامانۀ هدایت تصویری جراحی، تشخیص و بخش‌بندی، تشخیص مرحله، اریابی مهارت.
نوع مطالعه: پژوهشي |
موضوع مقاله:
شماره ویژه (رویکرد های نو در مهندسی کنترل)
دریافت: 1402/5/10 | پذیرش: 1402/6/25 | انتشار الکترونیک پیش از انتشار نهایی: 1402/6/28 | انتشار: 1402/6/30
دریافت: 1402/5/10 | پذیرش: 1402/6/25 | انتشار الکترونیک پیش از انتشار نهایی: 1402/6/28 | انتشار: 1402/6/30
فهرست منابع
1. [1] A. Gawande, "Two Hundred Years of Surgery," New England Journal of Medicine, vol. 366, no. 18, 2012, doi: 10.1056/nejmra1202392. [DOI:10.1056/NEJMra1202392]
2. [2] M. Swaminathan, S. Ramasubramanian, R. Pilling, J. Li, and K. Golnik, "ICO-OSCAR for pediatric cataract surgical skill assessment," Journal of AAPOS, vol. 20, no. 4, 2016, doi: 10.1016/j.jaapos.2016.02.015. [DOI:10.1016/j.jaapos.2016.02.015]
3. [3] L. Maier-Hein et al., "Surgical data science - from concepts toward clinical translation," Medical Image Analysis, vol. 76. 2022. doi: 10.1016/j.media.2021.102306. [DOI:10.1016/j.media.2021.102306]
4. [4] T. G. Weiser et al., "An estimation of the global volume of surgery: a modelling strategy based on available data," The Lancet, vol. 372, no. 9633, 2008, doi: 10.1016/S0140-6736(08)60878-8. [DOI:10.1016/S0140-6736(08)60878-8]
5. [5] T. G. Weiser et al., "Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes," The Lancet, vol. 385, 2015, doi: 10.1016/s0140-6736(15)60806-6. [DOI:10.1016/S0140-6736(15)60806-6]
6. [6] N. Alnafisee, S. Zafar, S. S. Vedula, and S. Sikder, "Current methods for assessing technical skill in cataract surgery," Journal of Cataract and Refractive Surgery, vol. 47, no. 2. 2021. doi: 10.1097/j.jcrs.0000000000000322. [DOI:10.1097/j.jcrs.0000000000000322]
7. [7] M. Nathan et al., "Intraoperative adverse events can be compensated by technical performance in neonates and infants after cardiac surgery: A prospective study," Journal of Thoracic and Cardiovascular Surgery, vol. 142, no. 5, 2011, doi: 10.1016/j.jtcvs.2011.07.003. [DOI:10.1016/j.jtcvs.2011.07.003]
8. [8] S. E. Regenbogen, C. C. Greenberg, D. M. Studdert, S. R. Lipsitz, M. J. Zinner, and A. A. Gawande, "Patterns of Technical Error Among Surgical Malpractice Claims," Ann Surg, vol. 246, no. 5, 2007, doi: 10.1097/sla.0b013e31815865f8. [DOI:10.1097/SLA.0b013e31815865f8]
9. [9] R. Anteby et al., "Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis," Surgical Endoscopy, vol. 35, no. 4. 2021. doi: 10.1007/s00464-020-08168-1. [DOI:10.1007/s00464-020-08168-1]
10. [10] K. Nandigam, J. Soh, W. G. Gensheimer, A. Ghazi, and Y. M. Khalifa, "Cost analysis of objective resident cataract surgery assessments," J Cataract Refract Surg, vol. 41, no. 5, 2015, doi: 10.1016/j.jcrs.2014.08.041. [DOI:10.1016/j.jcrs.2014.08.041]
11. [11] Anaconda, "The State of Data Science 2020 Moving from hype toward maturity, 2020."
12. [12] Y. Gao et al., "JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS): A Surgical Activity Dataset for Human Motion Modeling," Modeling and Monitoring of Computer Assisted Interventions (M2CAI) - MICCAI Workshop, 2014.
13. [13] D. Sarikaya, J. J. Corso, and K. A. Guru, "Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection," IEEE Trans Med Imaging, vol. 36, no. 7, 2017, doi: 10.1109/TMI.2017.2665671. [DOI:10.1109/TMI.2017.2665671]
14. [14] E. Colleoni, P. Edwards, and D. Stoyanov, "Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-59716-0_67. [DOI:10.1007/978-3-030-59716-0_67]
15. [15] A. Attanasio et al., "Autonomous Tissue Retraction in Robotic Assisted Minimally Invasive Surgery - A Feasibility Study," IEEE Robot Autom Lett, vol. 5, no. 4, 2020, doi: 10.1109/LRA.2020.3013914. [DOI:10.1109/LRA.2020.3013914]
16. [16] A. P. Twinanda, S. Shehata, D. Mutter, J. Marescaux, M. De Mathelin, and N. Padoy, "EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos," IEEE Trans Med Imaging, vol. 36, no. 1, 2017, doi: 10.1109/TMI.2016.2593957. [DOI:10.1109/TMI.2016.2593957]
17. [17] I. Aksamentov, A. P. Twinanda, D. Mutter, J. Marescaux, and N. Padoy, "Deep neural networks predict remaining surgery duration from cholecystectomy videos," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017. doi: 10.1007/978-3-319-66185-8_66. [DOI:10.1007/978-3-319-66185-8_66]
18. [18] A. Jin et al., "Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks," in Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, 2018. doi: 10.1109/WACV.2018.00081. [DOI:10.1109/WACV.2018.00081]
19. [19] K. Schoeffmann, H. Husslein, S. Kletz, S. Petscharnig, B. Muenzer, and C. Beecks, "Video retrieval in laparoscopic video recordings with dynamic content descriptors," Multimed Tools Appl, vol. 77, no. 13, 2018, doi: 10.1007/s11042-017-5252-2. [DOI:10.1007/s11042-017-5252-2]
20. [20] A. Leibetseder et al., "LapGyn4: A dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology," in Proceedings of the 9th ACM Multimedia Systems Conference, MMSys 2018, 2018. doi: 10.1145/3204949.3208127. [DOI:10.1145/3204949.3208127]
21. [21] A. Leibetseder, S. Kletz, K. Schoeffmann, S. Keckstein, and J. Keckstein, "GLENDA: Gynecologic Laparoscopy Endometriosis Dataset," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-37734-2_36. [DOI:10.1007/978-3-030-37734-2_36]
22. [22] D. Kitaguchi et al., "Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: Experimental research," International Journal of Surgery, vol. 79, 2020, doi: 10.1016/j.ijsu.2020.05.015. [DOI:10.1016/j.ijsu.2020.05.015]
23. [23] M. J. Primus et al., "Frame-Based Classification of Operation Phases in Cataract Surgery Videos," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018. doi: 10.1007/978-3-319-73603-7_20. [DOI:10.1007/978-3-319-73603-7_20]
24. [24] K. Schoeffmann, M. Taschwer, S. Sarny, B. Münzer, M. J. Primus, and D. Putzgruber, "Cataract-101 - Video dataset of 101 cataract surgeries," in Proceedings of the 9th ACM Multimedia Systems Conference, MMSys 2018, 2018. doi: 10.1145/3204949.3208137. [DOI:10.1145/3204949.3208137]
25. [25] M. Grammatikopoulou et al., "CaDIS: Cataract dataset for surgical RGB-image segmentation," Med Image Anal, vol. 71, 2021, doi: 10.1016/j.media.2021.102053. [DOI:10.1016/j.media.2021.102053]
26. [26] M. J. Ahmadi et al., "ARAS-Farabi Experimental Framework for Skill Assessment in Capsulorhexis Surgery," in 9th RSI International Conference on Robotics and Mechatronics, ICRoM 2021, 2021. doi: 10.1109/ICRoM54204.2021.9663494. [DOI:10.1109/ICRoM54204.2021.9663494]
27. [27] M. Lafouti et al., "Surgical Instrument Tracking for Capsulorhexis Eye Surgery Based on Siamese Networks," in 10th RSI International Conference on Robotics and Mechatronics, ICRoM 2022, 2022. doi: 10.1109/ICRoM57054.2022.10025355. [DOI:10.1109/ICRoM57054.2022.10025355]
28. [28] D. Bouget, R. Benenson, M. Omran, L. Riffaud, B. Schiele, and P. Jannin, "Detecting Surgical Tools by Modelling Local Appearance and Global Shape," IEEE Trans Med Imaging, vol. 34, no. 12, 2015, doi: 10.1109/TMI.2015.2450831. [DOI:10.1109/TMI.2015.2450831]
29. [29] S. Wang, A. Raju, and J. Huang, "Deep learning based multi-label classification for surgical tool presence detection in laparoscopic videos," in Proceedings - International Symposium on Biomedical Imaging, 2017. doi: 10.1109/ISBI.2017.7950597. [DOI:10.1109/ISBI.2017.7950597]
30. [30] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2015.
31. [31] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2015. doi: 10.1109/CVPR.2015.7298594. [DOI:10.1109/CVPR.2015.7298594]
32. [32] K. Mishra, R. Sathish, and D. Sheet, "Learning Latent Temporal Connectionism of Deep Residual Visual Abstractions for Identifying Surgical Tools in Laparoscopy Procedures," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2017. doi: 10.1109/CVPRW.2017.277. [DOI:10.1109/CVPRW.2017.277]
33. [33] H. Al Hajj, M. Lamard, P. H. Conze, B. Cochener, and G. Quellec, "Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks," Med Image Anal, vol. 47, 2018, doi: 10.1016/j.media.2018.05.001. [DOI:10.1016/j.media.2018.05.001]
34. [34] X. Du et al., "Articulated multi-instrument 2-d pose estimation using fully convolutional networks," IEEE Trans Med Imaging, vol. 37, no. 5, 2018, doi: 10.1109/TMI.2017.2787672. [DOI:10.1109/TMI.2017.2787672]
35. [35] E. Colleoni, S. Moccia, X. Du, E. De Momi, and D. Stoyanov, "Deep Learning Based Robotic Tool Detection and Articulation Estimation with Spatio-Temporal Layers," IEEE Robot Autom Lett, vol. 4, no. 3, 2019, doi: 10.1109/LRA.2019.2917163. [DOI:10.1109/LRA.2019.2917163]
36. [36] Z. Chen, Z. Zhao, and X. Cheng, "Surgical instruments tracking based on deep learning with lines detection and spatio-temporal context," in Proceedings - 2017 Chinese Automation Congress, CAC 2017, 2017. doi: 10.1109/CAC.2017.8243236. [DOI:10.1109/CAC.2017.8243236]
37. [37] Z. Zhao, T. Cai, F. Chang, and X. Cheng, "Real-time surgical instrument detection in robot-assisted surgery using a convolutional neural network cascade," in Healthcare Technology Letters, 2019. doi: 10.1049/htl.2019.0064. [DOI:10.1049/htl.2019.0064]
38. [38] Y. Liu, Z. Zhao, F. Chang, and S. Hu, "An anchor-free convolutional neural network for real-time surgical tool detection in robot-assisted surgery," IEEE Access, vol. 8, 2020, doi: 10.1109/ACCESS.2020.2989807. [DOI:10.1109/ACCESS.2020.2989807]
39. [39] B. Ran, B. Huang, S. Liang, and Y. Hou, "Surgical Instrument Detection Algorithm Based on Improved YOLOv7x," Sensors, vol. 23, no. 11, 2023, doi: 10.3390/s23115037. [DOI:10.3390/s23115037]
40. [40] C. I. Nwoye, D. Mutter, J. Marescaux, and N. Padoy, "Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos," Int J Comput Assist Radiol Surg, vol. 14, no. 6, 2019, doi: 10.1007/s11548-019-01958-6. [DOI:10.1007/s11548-019-01958-6]
41. [41] M. Islam, Y. Li, and H. Ren, "Learning Where to Look While Tracking Instruments in Robot-Assisted Surgery," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019. doi: 10.1007/978-3-030-32254-0_46. [DOI:10.1007/978-3-030-32254-0_46]
42. [42] X. Du et al., "Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking in surgical video," Med Image Anal, vol. 57, 2019, doi: 10.1016/j.media.2019.07.002. [DOI:10.1016/j.media.2019.07.002]
43. [43] I. Laina et al., "Concurrent segmentation and localization for tracking of surgical instruments," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017. doi: 10.1007/978-3-319-66185-8_75. [DOI:10.1007/978-3-319-66185-8_75]
44. [44] L. C. Garcia-Peraza-Herrera et al., "ToolNet: Holistically-nested real-time segmentation of robotic surgical tools," in IEEE International Conference on Intelligent Robots and Systems, 2017. doi: 10.1109/IROS.2017.8206462. [DOI:10.1109/IROS.2017.8206462]
45. [45] A. A. Shvets, A. Rakhlin, A. A. Kalinin, and V. I. Iglovikov, "Automatic Instrument Segmentation in Robot-Assisted Surgery using Deep Learning," in Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, 2019. doi: 10.1109/ICMLA.2018.00100. [DOI:10.1109/ICMLA.2018.00100]
46. [46] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015. doi: 10.1007/978-3-319-24574-4_28. [DOI:10.1007/978-3-319-24574-4_28]
47. [47] V. I. Iglovikov and A. A. Shvets, "TernausNet," in Computer-Aided Analysis of Gastrointestinal Videos, 2021. doi: 10.1007/978-3-030-64340-9_15. [DOI:10.1007/978-3-030-64340-9_15]
48. [48] A. Chaurasia and E. Culurciello, "LinkNet: Exploiting encoder representations for efficient semantic segmentation," in 2017 IEEE Visual Communications and Image Processing, VCIP 2017, 2018. doi: 10.1109/VCIP.2017.8305148. [DOI:10.1109/VCIP.2017.8305148]
49. [49] D. Pakhomov and N. Navab, "Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-59716-0_62. [DOI:10.1007/978-3-030-59716-0_62]
50. [50] S. M. Kamrul Hasan and C. A. Linte, "U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instruments from Laparoscopic Images," in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2019. doi: 10.1109/EMBC.2019.8856791. [DOI:10.1109/EMBC.2019.8856791]
51. [51] Z. L. Ni, G. Bin Bian, Z. G. Hou, X. H. Zhou, X. L. Xie, and Z. Li, "Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments," in Proceedings - IEEE International Conference on Robotics and Automation, 2020. doi: 10.1109/ICRA40945.2020.9197425. [DOI:10.1109/ICRA40945.2020.9197425]
52. [52] M. Islam, V. VS, C. M. Lim, and H. Ren, "ST-MTL: Spatio-Temporal multitask learning model to predict scanpath while tracking instruments in robotic surgery," Med Image Anal, vol. 67, 2021, doi: 10.1016/j.media.2020.101837. [DOI:10.1016/j.media.2020.101837]
53. [53] A. Nazir et al., "SPST-CNN: Spatial pyramid based searching and tagging of liver's intraoperative live views via CNN for minimal invasive surgery," J Biomed Inform, vol. 106, 2020, doi: 10.1016/j.jbi.2020.103430. [DOI:10.1016/j.jbi.2020.103430]
54. [54] Y. Fu et al., "More unlabelled data or label more data? a study on semi-supervised laparoscopic image segmentation," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019. doi: 10.1007/978-3-030-33391-1_20. [DOI:10.1007/978-3-030-33391-1_20]
55. [55] S. M. Cho, Y. G. Kim, J. Jeong, I. Kim, H. jin Lee, and N. Kim, "Automatic tip detection of surgical instruments in biportal endoscopic spine surgery," Comput Biol Med, vol. 133, 2021, doi: 10.1016/j.compbiomed.2021.104384. [DOI:10.1016/j.compbiomed.2021.104384]
56. [56] T. Cheng et al., "Deep learning assisted robotic magnetic anchored and guided endoscope for real-time instrument tracking," IEEE Robot Autom Lett, vol. 6, no. 2, 2021, doi: 10.1109/LRA.2021.3066834. [DOI:10.1109/LRA.2021.3066834]
57. [57] J. Choi, S. Cho, J. W. Chung, and N. Kim, "Video recognition of simple mastoidectomy using convolutional neural networks: Detection and segmentation of surgical tools and anatomical regions," Comput Methods Programs Biomed, vol. 208, 2021, doi: 10.1016/j.cmpb.2021.106251. [DOI:10.1016/j.cmpb.2021.106251]
58. [58] M. Kalia, T. A. Aleef, N. Navab, P. Black, and S. E. Salcudean, "Co-generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2021. doi: 10.1007/978-3-030-87202-1_39. [DOI:10.1007/978-3-030-87202-1_39]
59. [59] J. Wang, Y. Jin, L. Wang, S. Cai, P. A. Heng, and J. Qin, "Efficient Global-Local Memory for Real-Time Instrument Segmentation of Robotic Surgical Video," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2021. doi: 10.1007/978-3-030-87202-1_33. [DOI:10.1007/978-3-030-87202-1_33]
60. [60] K. Zinchenko and K. T. Song, "Autonomous Endoscope Robot Positioning Using Instrument Segmentation with Virtual Reality Visualization," IEEE Access, vol. 9, 2021, doi: 10.1109/ACCESS.2021.3079427. [DOI:10.1109/ACCESS.2021.3079427]
61. [61] A. Boonkong, D. Hormdee, S. Sonsilphong, and K. Khampitak, "Surgical Instrument Detection for Laparoscopic Surgery using Deep Learning," in 19th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, ECTI-CON 2022, 2022. doi: 10.1109/ECTI-CON54298.2022.9795561. [DOI:10.1109/ECTI-CON54298.2022.9795561]
62. [62] Z. L. Ni et al., "SurgiNet: Pyramid Attention Aggregation and Class-wise Self-Distillation for Surgical Instrument Segmentation," Med Image Anal, vol. 76, 2022, doi: 10.1016/j.media.2021.102310. [DOI:10.1016/j.media.2021.102310]
63. [63] X. Sun, Y. Zou, S. Wang, H. Su, and B. Guan, "A parallel network utilizing local features and global representations for segmentation of surgical instruments," Int J Comput Assist Radiol Surg, vol. 17, no. 10, 2022, doi: 10.1007/s11548-022-02687-z. [DOI:10.1007/s11548-022-02687-z]
64. [64] P. F. Baldi, S. Abdelkarim, J. Liu, J. K. To, M. D. Ibarra, and A. W. Browne, "Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning," Transl Vis Sci Technol, vol. 12, no. 1, 2023, doi: 10.1167/tvst.12.1.20. [DOI:10.1167/tvst.12.1.20]
65. [65] J. N. Paranjape, N. G. Nair, S. Sikder, S. S. Vedula, and V. M. Patel, "AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation," Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.03726
66. [66] A. Kirillov et al., "Segment Anything," Apr. 2023, [Online]. Available: http://arxiv.org/abs/2304.02643
67. [67] A. Wang, M. Islam, M. Xu, Y. Zhang, and H. Ren, "SAM Meets Robotic Surgery: An Empirical Study on Generalization, Robustness and Adaptation," Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.07156
68. [68] S. Petscharnig and K. Schöffmann, "Deep learning for shot classification in gynecologic surgery videos," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017. doi: 10.1007/978-3-319-51811-4_57. [DOI:10.1007/978-3-319-51811-4_57]
69. [69] S. Petscharnig and K. Schöffmann, "Learning laparoscopic video shot classification for gynecological surgery," Multimed Tools Appl, vol. 77, no. 7, 2018, doi: 10.1007/s11042-017-4699-5. [DOI:10.1007/s11042-017-4699-5]
70. [70] S. Petscharnig, K. Schoffmann, J. Benois-Pineau, S. Chaabouni, and J. Keckstein, "Early and Late Fusion of Temporal Information for Classification of Surgical Actions in Laparoscopic Gynecology," in Proceedings - IEEE Symposium on Computer-Based Medical Systems, 2018. doi: 10.1109/CBMS.2018.00071. [DOI:10.1109/CBMS.2018.00071]
71. [71] Y. Jin et al., "Multi-task recurrent convolutional network with correlation loss for surgical video analysis," Med Image Anal, vol. 59, 2020, doi: 10.1016/j.media.2019.101572. [DOI:10.1016/j.media.2019.101572]
72. [72] Y. Jin et al., "SV-RCNet: Workflow recognition from surgical videos using recurrent convolutional network," IEEE Trans Med Imaging, vol. 37, no. 5, 2018, doi: 10.1109/TMI.2017.2787657. [DOI:10.1109/TMI.2017.2787657]
73. [73] Y. Liu et al., "LoViT: Long Video Transformer for Surgical Phase Recognition," May 2023, [Online]. Available: http://arxiv.org/abs/2305.08989
74. [74] I. Funke, A. Jenke, S. T. Mees, J. Weitz, S. Speidel, and S. Bodenstedt, "Temporal coherence-based self-supervised learning for laparoscopic workflow analysis," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018. doi: 10.1007/978-3-030-01201-4_11. [DOI:10.1007/978-3-030-01201-4_11]
75. [75] D. R. Chittajallu et al., "XAI-CBIR: Explainable ai system for content based retrieval of video frames from minimally invasive surgery videos," in Proceedings - International Symposium on Biomedical Imaging, 2019. doi: 10.1109/ISBI.2019.8759428. [DOI:10.1109/ISBI.2019.8759428]
76. [76] B. Namazi, G. Sankaranarayanan, and V. Devarajan, "Attention-based surgical phase boundaries detection in laparoscopic videos," in Proceedings - 6th Annual Conference on Computational Science and Computational Intelligence, CSCI 2019, 2019. doi: 10.1109/CSCI49370.2019.00109. [DOI:10.1109/CSCI49370.2019.00109]
77. [77] S. Kannan, G. Yengera, D. Mutter, J. Marescaux, and N. Padoy, "Future-State Predicting LSTM for Early Surgery Type Recognition," IEEE Trans Med Imaging, vol. 39, no. 3, 2020, doi: 10.1109/TMI.2019.2931158. [DOI:10.1109/TMI.2019.2931158]
78. [78] Y. Chen, Q. L. Sun, and K. Zhong, "Semi-supervised spatio-temporal CNN for recognition of surgical workflow," EURASIP J Image Video Process, vol. 2018, no. 1, 2018, doi: 10.1186/s13640-018-0316-4. [DOI:10.1186/s13640-018-0316-4]
79. [79] I. Funke, D. Rivoir, and S. Speidel, "Metrics Matter in Surgical Phase Recognition," May 2023, [Online]. Available: http://arxiv.org/abs/2305.13961
80. [80] I. Funke, S. Bodenstedt, F. Oehme, F. von Bechtolsheim, J. Weitz, and S. Speidel, "Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Video," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019. doi: 10.1007/978-3-030-32254-0_52. [DOI:10.1007/978-3-030-32254-0_52]
81. [81] T. Khatibi and P. Dezyani, "Proposing novel methods for gynecologic surgical action recognition on laparoscopic videos," Multimed Tools Appl, vol. 79, no. 41-42, 2020, doi: 10.1007/s11042-020-09540-y. [DOI:10.1007/s11042-020-09540-y]
82. [82] F. Luongo, R. Hakim, J. H. Nguyen, A. Anandkumar, and A. J. Hung, "Deep learning-based computer vision to recognize and classify suturing gestures in robot-assisted surgery," Surgery (United States), vol. 169, no. 5, 2021, doi: 10.1016/j.surg.2020.08.016. [DOI:10.1016/j.surg.2020.08.016]
83. [83] C. I. Nwoye et al., "Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-59716-0_35. [DOI:10.1007/978-3-030-59716-0_35]
84. [84] A. Murali et al., "TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning," in Proceedings - IEEE International Conference on Robotics and Automation, 2016. doi: 10.1109/ICRA.2016.7487607. [DOI:10.1109/ICRA.2016.7487607]
85. [85] J. Xie, H. Zhao, Z. Shao, Z. Shi, and Y. Guan, "A Fast Approach for Multi-Modality Surgical Trajectory Segmentation with Unsupervised Deep Learning," Jiqiren/Robot, vol. 41, no. 3, 2019, doi: 10.13973/j.cnki.robot.180387.
86. [86] H. Zhao, J. Xie, Z. Shao, Y. Qu, Y. Guan, and J. Tan, "A fast unsupervised approach for multi-modality surgical trajectory segmentation," IEEE Access, vol. 6, 2018, doi: 10.1109/ACCESS.2018.2872635. [DOI:10.1109/ACCESS.2018.2872635]
87. [87] R. Tao, X. Zou, and G. Zheng, "LAST: LAtent Space-constrained Transformers for Automatic Surgical Phase Recognition and Tool Presence Detection," IEEE Trans Med Imaging, 2023, doi: 10.1109/TMI.2023.3279838. [DOI:10.1109/TMI.2023.3279838]
88. [88] D. Kiyasseh et al., "A vision transformer for decoding surgeon activity from surgical videos," Nat Biomed Eng, 2023, doi: 10.1038/s41551-023-01010-8. [DOI:10.1038/s41551-023-01010-8]
89. [89] B. Zhang et al., "Surgical workflow recognition with temporal convolution and transformer for action segmentation," Int J Comput Assist Radiol Surg, vol. 18, no. 4, 2023, doi: 10.1007/s11548-022-02811-z. [DOI:10.1007/s11548-022-02811-z]
90. [90] K. Lam et al., "Machine learning for technical skill assessment in surgery: a systematic review," npj Digital Medicine, vol. 5, no. 1. 2022. doi: 10.1038/s41746-022-00566-0. [DOI:10.1038/s41746-022-00566-0]
91. [91] L. Maier-Hein et al., "Surgical data science for next-generation interventions," Nature Biomedical Engineering, vol. 1, no. 9. 2017. doi: 10.1038/s41551-017-0132-7. [DOI:10.1038/s41551-017-0132-7]
92. [92] L. Wang et al., "Temporal Segment Networks for Action Recognition in Videos," IEEE Trans Pattern Anal Mach Intell, vol. 41, no. 11, 2019, doi: 10.1109/TPAMI.2018.2868668. [DOI:10.1109/TPAMI.2018.2868668]
93. [93] L. Wang et al., "Temporal segment networks: Towards good practices for deep action recognition," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016. doi: 10.1007/978-3-319-46484-8_2. [DOI:10.1007/978-3-319-46484-8_2]
94. [94] I. Funke, S. T. Mees, J. Weitz, and S. Speidel, "Video-based surgical skill assessment using 3D convolutional neural networks," Int J Comput Assist Radiol Surg, vol. 14, no. 7, 2019, doi: 10.1007/s11548-019-01995-1. [DOI:10.1007/s11548-019-01995-1]
95. [95] Y. Jia et al., "C3D: Generic Features for Video Analysis (Learning Spatiotemporal Features with 3D Convolutional Networks)," 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014.
96. [96] S. Ji, W. Xu, M. Yang, and K. Yu, "3D Convolutional neural networks for human action recognition," IEEE Trans Pattern Anal Mach Intell, vol. 35, no. 1, 2013, doi: 10.1109/TPAMI.2012.59. [DOI:10.1109/TPAMI.2012.59]
97. [97] J. Carreira and A. Zisserman, "Quo Vadis, action recognition? A new model and the kinetics dataset," in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017. doi: 10.1109/CVPR.2017.502. [DOI:10.1109/CVPR.2017.502]
98. [98] J. E. Kim, P. Weber, and A. Szabo, "Medical malpractice claims related to cataract surgery complicated by retained lens fragments (an American ophthalmological society thesis)," Trans Am Ophthalmol Soc, vol. 110, 2012.
99. [99] D. Kitaguchi, N. Takeshita, H. Matsuzaki, T. Igaki, H. Hasegawa, and M. Ito, "Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis," JAMA Netw Open, vol. 4, no. 8, 2021, doi: 10.1001/jamanetworkopen.2021.20786. [DOI:10.1001/jamanetworkopen.2021.20786]
100. [100] H. Doughty, D. Damen, and W. Mayol-Cuevas, "Who's Better? Who's Best? Pairwise Deep Ranking for Skill Determination," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018. doi: 10.1109/CVPR.2018.00634. [DOI:10.1109/CVPR.2018.00634]
101. [101] D. Liu, T. Jiang, Y. Wang, R. Miao, F. Shan, and Z. Li, "Clearness of operating field: a surrogate for surgical skills on in vivo clinical data," Int J Comput Assist Radiol Surg, vol. 15, no. 11, 2020, doi: 10.1007/s11548-020-02267-z. [DOI:10.1007/s11548-020-02267-z]
102. [102] T. Wang, Y. Wang, and M. Li, "Towards Accurate and Interpretable Surgical Skill Assessment: A Video-Based Method Incorporating Recognized Surgical Gestures and Skill Levels," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-59716-0_64. [DOI:10.1007/978-3-030-59716-0_64]
103. [103] Z. Li, L. Gu, W. Wang, R. Nakamura, and Y. Sato, "Surgical Skill Assessment via Video Semantic Aggregation," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2022. doi: 10.1007/978-3-031-16449-1_39. [DOI:10.1007/978-3-031-16449-1_39]
104. [104] A. Soleymani, A. A. Sadat Asl, M. Yeganejou, S. Dick, M. Tavakoli, and X. Li, "Surgical Skill Evaluation from Robot-Assisted Surgery Recordings," in 2021 International Symposium on Medical Robotics, ISMR 2021, 2021. doi: 10.1109/ISMR48346.2021.9661527. [DOI:10.1109/ISMR48346.2021.9661527]
105. [105] M. Shafiq and Z. Gu, "Deep Residual Learning for Image Recognition: A Survey," Applied Sciences (Switzerland), vol. 12, no. 18. 2022. doi: 10.3390/app12188972. [DOI:10.3390/app12188972]
106. [106] F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017. doi: 10.1109/CVPR.2017.195. [DOI:10.1109/CVPR.2017.195]
107. [107] Y. Gu et al., "Construction of Quantitative Indexes for Cataract Surgery Evaluation Based on Deep Learning," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-63419-3_20. [DOI:10.1007/978-3-030-63419-3_20]
108. [108] T. S. Kim, M. O'Brien, S. Zafar, G. D. Hager, S. Sikder, and S. S. Vedula, "Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery," Int J Comput Assist Radiol Surg, vol. 14, no. 6, 2019, doi: 10.1007/s11548-019-01956-8. [DOI:10.1007/s11548-019-01956-8]
109. [109] F. Setti et al., "A Multirobots Teleoperated Platform for Artificial Intelligence Training Data Collection in Minimally Invasive Surgery," in 2019 International Symposium on Medical Robotics, ISMR 2019, 2019. doi: 10.1109/ISMR.2019.8710209. [DOI:10.1109/ISMR.2019.8710209]
110. [110] S. S. Vedula, M. Ishii, and G. D. Hager, "Objective Assessment of Surgical Technical Skill and Competency in the Operating Room," Annu Rev Biomed Eng, vol. 19, 2017, doi: 10.1146/annurev-bioeng-071516-044435. [DOI:10.1146/annurev-bioeng-071516-044435]
ارسال پیام به نویسنده مسئول
| بازنشر اطلاعات | |
 |
این مقاله تحت شرایط Creative Commons Attribution-NonCommercial 4.0 International License قابل بازنشر است. |
