Deeper Insights: Optical Flow Post Processing for Medical Devices used for Knee Surgery
Case study from Deeper Insights.
Background & Description
In the context of Innovate UK funding, Deeper Insights has developed an innovative AI-driven markerless navigation system designed for application in Robotic-Assisted Total Knee Arthroplasty. The primary objective of this system was to identify and delineate key anatomical structures, specifically the femur and tibia, utilising data exclusively captured from a stereoscopic camera. Notably, the system was engineered to maintain its functionality across a spectrum of leg orientations, occlusions, and diverse lighting conditions.
The workflow encompassed the acquisition of data and real-time segmentation of the target anatomical structures. These segmentation results were then seamlessly incorporated into a navigation system, thereby enabling the assisted navigation component during robot-assisted surgical procedures.
However, it was observed that while the AI model demonstrated exceptional accuracy under typical leg positioning (i.e. bent leg around 60 degrees), its performance fell below established standards when confronted with extreme leg positions (i.e. extended leg).
To address these issues, Deeper Insights utilised Optical Flow. Optical flow is a technical approach to analyse movement of structures between frames. This approach was applied based on the assumption that the structure of the tibia and the femur should have aligned movement along consecutive frames. The output of the segmentation deep learning model was applied to a rules-based system, powered by optical flow, to validate the leg position and cross-check body part identification. Essentially, the output of the deep learning model was the input to the rules-based optical flow tool which created a clinical validity check.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles.
Safety, Security and Robustness
The biomedical field offers abundant possibilities for AI applications, yet it also introduces significant challenges, including concerns about trust, risk and physical safety of patients.
One of the significant challenges encountered during extreme leg positioning was the unintended interchange of the two anatomical structures. Although such positions rarely arose in standard procedures, we were committed to ensuring the system’s resilience and its ability to perform effectively under demanding acquisition conditions.
To address these extreme conditions in the processing pipeline, we implemented a clinical validity post-processing module, powered through an optical flow technique. In this, we have implemented confidence-based levels as well as heuristic measures designed to assess the correctness of the segmentation of the femur and tibia.
The optical flow technique was used to verify domain knowledge - particularly that femurs and tibias were being identified correctly by the model. This was a rules-based approach, optical flow was used to follow the direction of each bone detected, to see if the relative positions of the classified body parts made anatomical sense. This validated the body part detection (models’ output) through post processing.
The heuristic measurements were grounded in the fundamental concept that the “direction of tibia movement should align with the direction of femur movement.” To achieve such type of measurements, we incorporated optical flow into the analysis and computed both cosine similarity measurements as well as vector norms.
Regarding the confidence-based analysis, we examined the confidence values generated by the segmentation model and, after a series of morphological operations, we designated segmentation as valid under the condition that at least 75% of the segmented regions displayed a confidence level exceeding 90%.
Here safety of patients was supported by improvements in accuracy and cross verification of the models output (body part detection).
Appropriate Transparency & Explainability
For representation of the clinical validity post-processing, we have developed a graphical user interface that allows surgical staff to evaluate measurement validity during the decision-making process. The introduction of such a tool has made the entire workflow more transparent, providing a clear rationale, additional to the AI’s recommendations.
Why we took this approach
The approach to explore the clinical validity steps through the incorporation of optical flow analysis, cosine similarity measurements, and confidence value assessments stems from the central aim of the project. Our objective was to support our client in optimising the performance of medical devices with the power of AI, whilst maintaining the pivotal role of human decision-makers in medical procedures.
By introducing these additional measures, we actively enhanced the transparency and support mechanisms of the AI technology, ultimately fostering user trust in its safety, security, and robustness.
Benefits to the organisation using the technique
A comprehensive and assured tool addresses the potential for AI technologies to face challenges whilst acknowledging their limitations. This not only instils trust but also strengthens the overall reliability and safety of the tool in critical medical applications, underscoring our commitment to safety, security, and robustness.
By openly acknowledging the system’s strengths and limitations, we make strides towards a more comprehensive and accountable tool, aligning with the principles of appropriate transparency and explainability.
Technical measures and processes such as these must be utilised to reduce the inherent risks associated with AI technologies, ultimately allowing the system to be deployed in real-world settings and it’s benefits realised.
Limitations of the approach
-
The approach may add complexity to the surgical process.
-
The clinical validity step may not completely eliminate the risk of incorrect AI outputs.
Further AI Assurance Information
-
For more information about other techniques visit the CDEI Portfolio of AI Assurance Tools: https://www.gov.uk/ai-assurance-techniques
-
For more information on relevant standards visit the AI Standards Hub: https://aistandardshub.org/