Overview and structure
Introduction
AUDIT is a framework designed to standardize the evaluation and analysis of medical image segmentation models. The project focuses on providing a reproducible pipeline that connects raw imaging data, model predictions, quantitative evaluation metrics, and interactive analysis tools within a single ecosystem.
The goal is to facilitate transparent benchmarking, and exploration of segmentation models, particularly in medical imaging segmentation workflows such as MRI-based analysis.
This documentation provides and overview of the architecture, modules, and workflow of the AUDIT framework. The different components shown in the diagram below are introduced here and described in more detail in the following sections of the documentation.
1. Data management
The pipeline starts from raw medical imaging data, typically composed of MRI volumes, Segmentation masks, Associated metadata.
The Utils module handles preprocessing tasks required to standardize the dataset before analysis. Typical operations include:
- Renaming and organizing files
- Adjusting orientation or anatomical planes
- Label replacement or harmonization
- Dataset normalization and structure validation
The output of this stage is a postprocessed dataset that can be consistently used by downstream modules.
2. Feature extraction
In addition to segmentation metrics, AUDIT can compute image-derived features from MRI sequences.
The Features module extracts quantitative descriptors such as:
- First-order statistics
- Second-order texture features
- Tumor or region-specific features
- Additional radiomics descriptors
The input to this module consists of MRI sequences (e.g. T1, T1Gd, T2, FLAIR) and (if available) corresponding segmentation masks. The output is a collection of structured feature tables (CSV files) that can later be explored in the AUDIT interface.
3. Metrics extraction
One of the core components of AUDIT is the Metrics module, responsible for computing performance metrics for segmentation models. The module compares model predictions and ground truth annotations (reference image), and produces a set of quantitative evaluation metrics.
The system is designed to support multiple metric backends, allowing flexible evaluation strategies and compatibility with existing metric libraries. For instance, metrics can be computed either based on custom metrics predefined within AUDIT library or taking advantage of standard Python libraries such as Pymia or Metrics Reloaded.
Metric results are exported as structured CSV files, which can be further analyzed in the AUDIT interface.
4. AUDIT dashboard (APP)
The results generated by the pipeline are aggregated in the AUDIT application, an interactive environment for exploring segmentation performance and extracted features.
The application relies on modern visualization libraries such as Plotly and Streamlit, enabling interactive plots and exploratory analysis. This interface serves as the central hub for model evaluation and comparison within the AUDIT ecosystem.
5. Configuration system
The framework relies on a configuration module to control different parts of the pipeline.
Configurations are defined using YAML files, which specify parameters for: Feature extraction, Metric extraction, and APP settings.
This approach allows experiments to be reproducible and easily configurable, without modifying the underlying codebase.
6. External tools integration
AUDIT also supports integration with external tools that complement the evaluation workflow.
Examples include:
- Visualization tools for inspecting segmentations (e.g. medical image viewers)
- Command-line utilities for launching analysis pipelines
These integrations allow users to combine AUDIT with existing tools used in medical imaging workflows.
Readers are encouraged to use this page as a conceptual overview of the system architecture before exploring the individual modules described in the following sections.