ResearchOct 6, 20269 min read

AI for Endometriosis Detection on Ultrasound: Building a Real-Time POC

Why endometriosis is hard to see on transvaginal ultrasound, and how to build an AI proof of concept that works anyway: video segmentation, temporal tracking, learning from scarce labels, honest validation and deployment as a containerized API.

Endometriosis is common, painful and slow to diagnose. The World Health Organization estimates that it affects roughly 10% of women and girls of reproductive age, around 190 million people, and patients frequently wait several years between first symptoms and a diagnosis. For a long time, the reference diagnosis was surgical. The 2022 ESHRE guideline changed that emphasis: imaging, mainly transvaginal ultrasound (TVUS) and MRI, is now recommended as part of the first-line work-up, with laparoscopy reserved for cases where imaging is negative or empirical treatment has failed.

That shift puts more weight on ultrasound, and ultrasound for endometriosis is hard. We recently built a real-time proof of concept for Femnov that detects, segments and tracks endometriosis lesions on endovaginal ultrasound video. This article covers the engineering problems behind that kind of system and how we think about each one. The project write-up describes the engagement itself.

Why endometriosis is hard to see on transvaginal ultrasound

Low contrast and variable appearance

Endometriosis takes several forms: ovarian endometriomas, deep endometriosis infiltrating structures such as the uterosacral ligaments, bowel or bladder, and superficial peritoneal disease. Their ultrasound appearance varies. Some lesions are reasonably distinct; many are small, irregular and close in echogenicity to the tissue around them. Superficial disease in particular is often not visible on ultrasound at all, which is a known limitation of the modality and a reason to be careful about what any model is claimed to detect.

For a segmentation model, low contrast means the boundary between lesion and background is weak, and the signal that separates them is partly contextual: location relative to anatomical landmarks, shape, and how tissue behaves when the probe applies pressure.

Speckle, shadowing and motion

Ultrasound images are formed from reflected sound, and they carry artifacts that natural-image models never see in pretraining. Speckle, the granular interference pattern across the image, looks like texture but is not anatomy. Acoustic shadowing and enhancement change brightness behind dense or fluid-filled structures. Gain, depth and focus settings differ between exams. The probe moves continuously, so a structure may be visible for a few dozen frames, partially out of plane, and then gone.

Operator dependence

TVUS is strongly operator-dependent. The International Deep Endometriosis Analysis (IDEA) group published a consensus in 2016 that defines a systematic four-step examination precisely because results depend so much on how the exam is performed. Studies using that protocol report good accuracy for deep endometriosis and endometriomas in expert hands. Outside expert centers, both the views captured and their interpretation vary more.

This matters twice for AI. It is part of the motivation for decision support, and it is a source of distribution shift. Video from one sonographer, machine and protocol can look different from video from another.

Segmentation on ultrasound video

Detection, segmentation or both

A detector draws boxes around candidate lesions. A segmentation model outputs a pixel mask. For an assistive overlay, segmentation is usually more useful because it shows extent and shape, but small, low-contrast lesions are where segmentation models struggle most. A common pattern is to combine the two: a detection signal decides whether something is there, and a segmentation head describes it. That separation also helps evaluation, since "did we find it" and "did we outline it correctly" are different questions with different clinical meaning.

Working on video, not curated frames

Many medical imaging datasets consist of still frames a clinician chose to save. Those are the clearest views of the exam. A system that runs live sees everything else too: transitions, partial views, frames dominated by artifacts. Training and evaluating on full video sequences, or at least on frames sampled from them without cherry-picking, gives a much more honest picture of how the model behaves in use.

Architecture choices under a latency budget

Encoder-decoder architectures in the U-Net family remain strong baselines for medical segmentation and are efficient enough for real time on a modern GPU. Transformer-based encoders can capture more global context, which helps when the lesion is only recognizable relative to surrounding anatomy, but they cost more compute per frame. The right choice depends on input resolution, the hardware available at the deployment site, and the framerate the overlay has to sustain. We treat the latency budget as a design constraint from the first experiment, not an optimization pass at the end.

Temporal consistency across frames

A per-frame model applied to ultrasound video flickers. A lesion is segmented in one frame, missed in the next, and reappears with a different shape in the third. Each prediction might be individually defensible, but the sequence is hard to watch and harder to trust.

There are several ways to address this, and they can be combined:

Temporal context in the model. Feeding a short window of frames, or carrying a hidden state forward, lets the network use motion and persistence as evidence. A structure that stays coherent as the probe moves is more likely to be real than speckle that changes every frame.
Tracking on top of detection. Associating detections across frames, with a motion model and an appearance model, turns isolated predictions into tracks. A track can be required to persist for a minimum number of frames before it is shown, and can survive a frame or two of missed detection.
Mask propagation. Once a lesion is segmented, its mask can be propagated to the next frame and refined, rather than recomputed from scratch.
Smoothing with care. Simple temporal averaging reduces flicker but adds lag. In a live overlay, lag is visible as a mask trailing behind the anatomy, so any smoothing has to be tuned against the framerate.

Temporal tracking was a core part of the Femnov system, and it is what made frame-to-frame output consistent at near-live framerate.

Learning with scarce annotated clinical data

Labeled endometriosis ultrasound is scarce. Annotating it requires gynecologists with specific imaging expertise, it takes time per sequence, and the underlying data is sensitive patient information that cannot be freely pooled. Most projects start with far less labeled data than a generic segmentation benchmark would assume.

Transfer learning and domain-specific pretraining

Initializing from weights pretrained on natural images helps with low-level features, but ultrasound is visually far from photographs. Pretraining or fine-tuning on other ultrasound data, even from different organs or tasks, usually transfers better because the encoder has already learned what speckle and shadowing look like.

Self-supervised pretraining on unlabeled video

Clinical sites often have much more unlabeled video than labeled frames. Self-supervised methods, such as masked image modeling or contrastive learning across frames of the same sequence, can learn useful representations from that video without annotations. The labeled set is then used to fine-tune a model that already understands the domain. Unlabeled data still has to be anonymized and governed like any clinical data.

Annotation strategy with experts

Expert time is the scarcest resource in the project, so the annotation protocol deserves as much design as the model:

Define the labels with the clinicians first. Which lesion types are in scope, how boundaries are drawn when they are ambiguous, and what counts as "not visible" should be written down before annotation starts.
Annotate sparsely, propagate carefully. Labeling every frame of a sequence is rarely necessary. Annotating key frames and propagating labels, with expert correction, stretches expert time much further.
Measure agreement. Having a subset annotated by more than one expert shows how much the experts themselves disagree. That inter-rater variability sets a realistic ceiling for the model and informs how to interpret its errors.
Prioritize hard cases. Once a first model exists, routing its uncertain predictions to experts for labeling, a form of active learning, tends to improve the model faster than labeling more easy cases.

Augmentation that respects the physics

Augmentations for ultrasound should mimic real variation: gain and contrast changes, speckle noise, small deformations, shifts in depth and field of view. Some augmentations common in natural images, such as arbitrary color changes, are meaningless here, and flips need to respect anatomical orientation conventions.

How to validate a POC honestly

A proof of concept exists to answer one question: is this approach worth further investment? The answer is only useful if the validation is set up so that it could fail.

Anonymize before anything else

Data should be anonymized before it reaches the modeling team. With ultrasound, that means more than stripping DICOM headers. Patient details can be burned into the image itself, in on-screen text overlays, and those regions need to be removed or masked. Anonymization is also a prerequisite for working with clinical partners under GDPR and similar regimes.

Split by patient, not by frame

Consecutive video frames are nearly identical. If frames from the same patient appear in both training and test sets, the model is effectively tested on data it has already seen, and the results will be inflated. Splits must be made at the patient level, and ideally the test set should also include exams from operators or machines the model was not trained on.

Compare against clinical ground truth, with expert review

Model output on a screen is persuasive and easy to over-read. The useful comparison is against clinical ground truth established independently of the model, with predictions reviewed by expert gynecologists. Expert review also captures something a metric does not: whether the errors the model makes are the kind a clinician would find acceptable, confusing or dangerous. In the Femnov POC, predictions were reviewed by expert gynecologists against clinical ground truth on an anonymized clinical dataset.

Report what the POC does not show

A POC on a single dataset says nothing definitive about performance on other populations, machines or operators. It is not evidence of clinical benefit and not a substitute for a prospective study. Writing these limits down alongside the results keeps the next decision grounded and makes the path to a clinical product clearer: more sites, prospective evaluation, a quality management system and regulatory planning for software that assists diagnosis.

Packaging for clinical environments

A model that runs in a notebook is not something a clinical partner can test. For the Femnov POC we delivered the system as a containerized API built with Docker and FastAPI. The reasons generalize:

Reproducibility. The container pins the model weights, preprocessing and dependencies, so the system that was validated is the system that runs. Versioned images make it possible to say exactly which model produced which output.
On-premise deployment. Hospitals and clinics often cannot send patient video to an external cloud. A self-contained service can run on hardware inside the clinical network, which simplifies data protection reviews.
Integration through a standard interface. An HTTP API lets the partner's software send frames or streams and receive masks and tracks without depending on the model's internals. The model can be updated without changing the integration.
Latency as part of the contract. For live use, the end-to-end loop matters: decoding, preprocessing, inference, tracking and returning results. Measuring that full path on the target hardware, under realistic load, is the only reliable way to know whether the overlay will keep up with the exam.
Predictable failure. When input quality is too poor or the service is unavailable, the system should degrade visibly and safely, showing no overlay rather than a stale or misleading one.

Summary

Endometriosis on transvaginal ultrasound combines most of the hard problems in medical computer vision: low-contrast targets, noisy and moving video, operator-dependent acquisition and very little labeled data. A credible proof of concept addresses each of them directly: segmentation trained and evaluated on real video, temporal tracking for frame-to-frame consistency, pretraining and expert-driven annotation to make the most of scarce labels, patient-level validation against clinical ground truth with expert review, and packaging that a clinical partner can actually deploy.

If you are building AI for ultrasound or other medical imaging and need a team that treats validation and deployment as part of the model, see our healthcare AI services or get in touch.

Related projects

Healthcare AIReal-Time Endometriosis Detection on UltrasoundBuilt a real-time AI proof of concept for Femnov that detects and segments endometriosis lesions on endovaginal ultrasound video, with temporal tracking at near-live framerate, delivered as a containerized API.