Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

dc.contributor.authorLajkó, Gábor
dc.contributor.authorNagyné Elek, Renáta
dc.contributor.authorHaidegger, Tamás
dc.date.accessioned2021-09-14T12:10:26Z
dc.date.available2021-09-14T12:10:26Z
dc.date.issued2021-08-10
dc.date.updated2021-09-13T13:37:18Z
dc.description.abstractObjective skill assessment-based personal performance feedback is a vital part of surgical training. Either kinematic—acquired through surgical robotic systems, mounted sensors on tooltips or wearable sensors—or visual input data can be employed to perform objective algorithm-driven skill assessment. Kinematic data have been successfully linked with the expertise of surgeons performing Robot-Assisted Minimally Invasive Surgery (RAMIS) procedures, but for traditional, manual Minimally Invasive Surgery (MIS), they are not readily available as a method. 3D visual features-based evaluation methods tend to outperform 2D methods, but their utility is limited and not suited to MIS training, therefore our proposed solution relies on 2D features. The application of additional sensors potentially enhances the performance of either approach. This paper introduces a general 2D image-based solution that enables the creation and application of surgical skill assessment in any training environment. The 2D features were processed using the feature extraction techniques of a previously published benchmark to assess the attainable accuracy. We relied on the JHU–ISI Gesture and Skill Assessment Working Set dataset—co-developed by the Johns Hopkins University and Intuitive Surgical Inc. Using this well-established set gives us the opportunity to comparatively evaluate different feature extraction techniques. The algorithm reached up to 95.74% accuracy in individual trials. The highest mean accuracy—averaged over five cross-validation trials—for the surgical subtask of Knot-Tying was 83.54%, for Needle-Passing 84.23% and for Suturing 81.58%. The proposed method measured well against the state of the art in 2D visual-based skill assessment, with more than 80% accuracy for all three surgical subtasks available in JIGSAWS (Knot-Tying, Suturing and Needle-Passing). By introducing new visual features—such as image-based orientation and image-based collision detection—or, from the evaluation side, utilising other Support Vector Machine kernel methods, tuning the hyperparameters or using other classification methods (e.g., the boosted trees algorithm) instead, classification accuracy can be further improved. We showed the potential use of optical flow as an input for RAMIS skill assessment, highlighting the maximum accuracy achievable with these data by evaluating it with an established skill assessment benchmark, by evaluating its methods independently. The highest performing method, the Residual Neural Network, reached means of 81.89%, 84.23% and 83.54% accuracy for the skills of Suturing, Needle-Passing and Knot-Tying, respectively.en
dc.identifier.eissn1424-8220
dc.identifier.urihttps://depositonce.tu-berlin.de/handle/11303/13582
dc.identifier.urihttp://dx.doi.org/10.14279/depositonce-12369
dc.language.isoenen
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/en
dc.subject.ddc000 Informatik, Informationswissenschaft, allgemeine Werkede
dc.subject.otherrobot-assisted minimally invasive surgeryen
dc.subject.othersurgical skill assessmenten
dc.subject.otherJIGSAWSen
dc.subject.otheroptical flowen
dc.titleEndoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgeryen
dc.typeArticleen
dc.type.versionpublishedVersionen
dcterms.bibliographicCitation.articlenumber5412en
dcterms.bibliographicCitation.doi10.3390/s21165412en
dcterms.bibliographicCitation.issue16en
dcterms.bibliographicCitation.journaltitleSensorsen
dcterms.bibliographicCitation.originalpublishernameMDPIen
dcterms.bibliographicCitation.originalpublisherplaceBaselen
dcterms.bibliographicCitation.volume21en
tub.accessrights.dnbfreeen
tub.affiliationFak. 4 Elektrotechnik und Informatik::Inst. Softwaretechnik und Theoretische Informatik::FG Datenbanksysteme und Informationsmanagement (DIMA)de
tub.affiliation.facultyFak. 4 Elektrotechnik und Informatikde
tub.affiliation.groupFG Datenbanksysteme und Informationsmanagement (DIMA)de
tub.affiliation.instituteInst. Softwaretechnik und Theoretische Informatikde
tub.publisher.universityorinstitutionTechnische Universität Berlinen

Files

Original bundle
Now showing 1 - 1 of 1
Loading…
Thumbnail Image
Name:
sensors-21-05412-v2.pdf
Size:
2.95 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
4.9 KB
Format:
Item-specific license agreed upon to submission
Description:

Collections