This article was first published on Computer Vision News of November 2021.
Pietro Mascagni is a medical doctor, concluding a PhD in surgical data science between the Catholic University of Rome; ICube, University of Strasbourg; and IHU Strasbourg. His supervisors are Professor Guido Costamagna in Rome from the clinical side and Professor Nicolas Padoy in Strasbourg from the computer science side. He speaks to us about his work at the intersection of the two communities.
Pietro has a bold mission: “I want to bridge the gap between surgical need and computer science to innovate for patients, surgeons, and healthcare systems.”
But how can innovations the computer science community has fostered over the years be integrated into the clinical workflow to bring value to healthcare? Pietro tells us both sides need to communicate, collaborate, and crosspollinate. This calls for people who have a foot in both camps and can translate clinical need into scientific questions to be answered by means of computer science. Also, multidisciplinary efforts are needed to develop datasets and metrics that work for both communities.
Nassir Navab told us that he insists engineers in his lab meet regularly with clinicians to facilitate the exchange of information and culture.
“Professor Navab is a pioneer in this field and fully grasps the need for this reciprocal contamination,” Pietro attests. “Computer scientists must understand how their work fits in the broader picture of clinical care if they want to have an impact. This works both ways. Physicians often over or underestimate what can be done or have completely the wrong idea about technology. Visionaries like Professor Navab and computer science communities like MICCAI, who have just launched their first clinical day, are recognizing that we need to team up.”
The surgical community is taking a step towards computer science too. Half of the members of the European Association of Endoscopic Surgery (EAES), of which Pietro is on the technology committee, are engineers. The same is true for the American Society for Gastrointestinal Endoscopy (ASGE).
“Modern surgery has become a complex socio-technical process, with multiple highly specialized professionals coordinating and taking decisions in seconds, interpreting signals from highly technological devices,” Pietro explains. “The process of surgical care, when it goes smoothly, is highly effective, but it’s also highly error prone, and the complexity can be described as a problem of information flow. We have analytics that are great at modeling this kind of unstructured information and could be used to deliver the right information to the right person at the right time.”
This is the vision that motivated computer scientists and surgeons to team up in recent years to establish the field of surgical data science, which has huge potential for improving preoperative, intraoperative, and postoperative care. Pietro is focused on translating that vision into something concrete by developing software that supports surgeons intraoperatively in operating rooms. 65% of medical error happens there, and an estimated 50% of complications are preventable.
“The fact is we know very little about what happens in the operating room because it is kind of a black box,” he points out. “The only direct information on operating room activity is operator written reports, so I’m using videos to shine a light on what happens.”
The practical test case for this is cholecystectomy – the surgical removal of the gallbladder. It is a very common abdominal surgical procedure performed with a minimally invasive laparoscopic approach in 98% of cases. An endoscopic camera is inserted into the abdomen and a video guides the surgeon’s movement, which is digital information that could be analyzed by deep learning and computer vision methods.
“We want to improve safety in cholecystectomy, in particular the risk of bile duct injury,” Pietro tells us. “The gallbladder sits underneath the liver, connected by the biliary tree to the gut. You need to detach the gallbladder from the biliary tree to remove it. Every clinical outcome improved with the introduction of the minimally invasive technique, except the rate of bile duct injury. When there is a bile duct injury it’s not usually a technical mistake.
What happens is the visual perception illusion. The procedure is guided by 2D or 3D images and the human brain is wired in a way that it tends to see what it expects to see, so if the common bile duct is retracted towards the gallbladder, then the surgeon can think they’re cutting the cystic duct while actually dividing the common bile duct.”
This complication happens three times more in minimally invasive surgery than it did in open surgery, where there would be haptic as well as visual feedback. Even though the incidence of this adverse event is low, at between 0.3% and 1.5%, given the high number of procedures performed each year, this translates to a huge number of patients. Patients experiencing a bile duct injury have a three-fold mortality increase at one year. This is devastating for surgeons too and is estimated to cost $1 billion every year in the US alone.
“A solution already exists,” Pietro reveals. “In 1995, surgeon Steven Strasberg proposed a secure target identification technique often used in hunting – you don’t shoot an animal before you know for sure it’s an animal. The same type of concept applies in surgery. You don’t clip the cystic duct until you’re sure it’s the cystic duct.”
Strasberg called it the critical view of safety (CVS) and it is defined by three anatomical landmarks. However, despite the fact there is a large surgical consensus around it, a recent review found the rate of bile duct injury was not decreasing.
The reason for this lies in the fact it is a low-incidence problem, so it tends not to be a priority in the surgeon’s mind when operating. Video-based assessment shows CVS is achieved in only 9-20% of the cases. In 99% of cases, the procedure is still successful, but the error trap lies in the other 1%. Surgeons must perform CVS systematically so that it is there when they need it.
“Moving continuously between the clinics and computer science labs has been the most exciting part of the work we’ve been doing,” Pietro tells us. “I must stress the ‘we’ because this is the work of a surgical data science team. In most of my publications I have co-first authors who are engineers and computer scientists. We do different jobs, but we work together, and it’s highly rewarding. It’s been a big learning opportunity for me, but I hope also for Deepak Alapatt and Armine Vardazaryan, who are the two main research engineers I work with under Nicolas’s supervision.”
Pietro says both sides have had to converge in order to speak the same language.
“I had to learn how to code because it forces you to better understand what you’re trying to do,” he points out. “It allows you to be more autonomous in running these models, analyzing them on your own, and following, understanding, and improving the computer science research a little bit as well. The point isn’t to devise these tools myself – I don’t want to steal the work of computer scientists and the computer scientists I work with are not interested in doing what I’m doing!”
Four years ago, Pietro ended up in Strasbourg by chance with a mobility scholarship from the European Union.
He says he got “super passionate” about the research being performed at IHU Strasbourg and was lucky enough to win a research grant from EAES. At that point, Nicolas offered him the opportunity to do a PhD with his computer science team.
“My clinical supervisors were quite puzzled at first!” he reveals. “They were afraid it would take me away from my profession because at the end of the day, I’m a medical doctor.”
But Pietro took a chance, and four years later, he is very glad he did.
“I’m super, super happy!” he grins. “My supervisors, who were initially skeptical, are happy now too. I’m even more motivated than before to go back to clinical practice. In November, I will be starting a residency and will eventually be translating what we’ve been developing in the lab to the operating room.”
Pietro will be defending his PhD thesis in the spring, and he promises to return to our pages to continue this discussion then.
He teases: “Next time, I will tell you a story of how a clinical need drives innovation in computer science, and how observation done with computer science methods are fed back to the surgical department to improve the quality of the care they deliver!”
Keep reading the Medical Imaging News section of our magazine.
Read about RSIP Vision’s R&D work on the medical applications of AI.