Exo – AI in the service of heart exams
Thanks to Machine Learning and appropriate data collection Exo Imaging created a coaching program for young cardiologists. We have transformed the knowledge and experience of senior specialists into digital training.
About Exo Imaging
Exo is an ultrasound platform for medical imaging. The company developed a complete solution that supports the work of medics even in the fast-paced hospitals’ environment. The software ensures a sharp 3D image, intuitive interface and the best tools when it comes to USG.
The Exo team is derived from the most famous companies: Apple, Google, GE, Johnson & Johnson, Maxim, Medtronic and Siemens. They started the cooperation with Skyrise’s experts to develop image processing algorithms.
The challenge: demanding technologies
An echocardiogram can help detect damage caused by a heart attack, heart failure, congenital heart disease, problems with the heart valves, cardiomyopathies and more. But to discover abnormalities a doctor needs to know what to look for.
As we said earlier, young medics learn the art of a heart examination with Exo’s software. And the software’s algorithms learn from data that we collected with experienced operators.
The Exo project was challenging because of the machine learning and artificial intelligence processes and technologies. We were to:
// find medics, hospitals and universities eager to deliver examination data,
// get permissions to use the data,
// collect a huge amount of data,
// process and label them properly,
// and create an intelligent system with an intuitive UI that would teach the algorithms about heart examinations.
But this was not the end of our tasks. We also needed to test in real-life if Exo’s product would work.
Our approach: a total immersion
A hospital is a different world. It is a constant battle between the time, illnesses and… medical devices. Every day, there is a queue of patients waiting in lines for services. One of the most important services is, of course, an ultrasound examination of a heart.
A person who holds the probe needs to immediately spot and recognise both, bad and good marks on the heart. In other case, they could misdiagnose. A lot depends on the machinery and software.
The Skyrise’s team considered every little detail of medics’ work. We asked many questions:
// Under what conditions do they work?
// How many hours per day? When are they exhausted?
// How long does it take to exam the heart?
// How do they move the probe? At what angle is the probe positioned?
// What information should they see on the screen?
// What pieces of data make a useful medical information?
To get those answers we interviewed medical personnel, observed them while they worked, we talked with scientists, and even put into our office an exam table and screens the same as you see in hospitals. We immersed completely!
Gathering data from more than a hundred examinations
In order to provide modern end-user applications, we refactored the existing system. Moreover, we extended it with components and developed new features. Therefore, to handle the complexity of the existing back–end system, we built an API facade over it along with an OpenID Connect–based authentication.
R&D and data labeling
We started our cooperation with Exo Imaging by examining the possibilities of sensors of the cardiographic probe. Our R&D experts learned how the devices collect information during the test, at what angle they are set, how quickly the doctor works with them, etc. With such a knowledge, we were prepared to start the next step: data labeling.
How to manage the process of labeling images to be used as data to build machine learning models? The Skyrise’s team faced that challenge. Fortunately, due to previous machine learning projects, we knew what to do. We prepared an advanced data labeling system that immediately recognised an image category and content, labeled it and created a base for further learning of AI.
But this was not the end. To work, the system needed integrations with medical devices for data downloading. The Skyrise’s team prepared and tested those integrations.
Contextual inquiry for a better UI
Everything we created was to be used to help medics. However, we did not want to rely only on our own assumptions and had to see how a doctor was really using our data collection application.
Our User Experience designer went to one of the hospitals to find out how the application was used by the examiner. It turned out that the initial interface was a bit problematic: the icons were too small and visual communication not clear enough. We also spotted that the doctors were fatigued after some time and in a constant hurry. Basing on that contextual research we designed a new, comfortable user interface (UI).