Habitat Segmentation Configuration

Habitat Analysis Configuration Parameters

This section covers habitat analysis configuration. CLI: habit get-habitat -c <yaml>. Demo training: config/habitat/config_habitat_two_step.yaml; prediction: config/habitat/config_habitat_two_step_predict.yaml; SLIC supervoxel example: config/habitat/config_habitat_two_step_supervoxel_slic.yaml (parameter details in “habitat_segmentation.supervoxel — SLIC Superpixel Configuration” below).

Example configuration file:

run_mode: train
pipeline_path: ./results/habitat_pipeline.pkl
data_dir: ./file_habitat.yaml
out_dir: ./results/habitat/train

feature_construction:
  voxel_level:
    method: concat(raw(delay2), raw(delay3), raw(delay5))

  supervoxel_level:
    supervoxel_file_keyword: '*_supervoxel.nrrd'
    method: mean_voxel_features()

  preprocessing_for_subject_level:
    methods:
      - method: winsorize
        winsor_limits: [0.05, 0.05]
        global_normalize: true
      - method: minmax
        global_normalize: true

  preprocessing_for_group_level:
    methods:
      - method: binning
        n_bins: 10
        bin_strategy: uniform
        global_normalize: false

habitat_segmentation:
  clustering_mode: two_step

  supervoxel:
    algorithm: kmeans
    n_clusters: 50
    random_state: 42
    max_iter: 300
    n_init: 10

  habitat:
    algorithm: kmeans
    max_clusters: 10
    habitat_cluster_selection_method:
      - inertia
      - silhouette
    fixed_n_clusters: null
    random_state: 42
    max_iter: 300
    n_init: 10

processes: 2
cap_processes_to_gpu_pool: false
individual_subject_timeout_sec: 900
individual_subject_spawn_timeout_sec: 120
resume: true
strict_checkpoint_hash: false
checkpoint_dir: null
force_rerun_subjects: []
retry_failed_subjects: false
individual_subject_auto_retry_rounds: 2
individual_subject_parallel_mode: persistent
persistent_worker_max_consecutive_failures: 1
persistent_worker_recycle_after_tasks: 0
clear_checkpoint_on_success: false
plot_curves: true
save_images: true
save_results_csv: true
random_state: 42
verbose: true
debug: false

run_mode: Run mode

• Type: string

• Required: no

• Default: train

• Allowed values: train, predict

• Description: train trains a new model; predict runs inference with a pretrained pipeline.

• Example: train

pipeline_path: Pipeline file path

• Type: string

• Required: no (required in predict mode)

• Default: null

• Description: Path to the trained pipeline file.

• Example: ./results/habitat_pipeline.pkl

data_dir/out_dir (habitat analysis top level)

• Type: string

• Required: yes

• Default: none (required)

• Description: data_dir may be a directory or a manifest such as file_habitat.yaml; out_dir is the default parent directory for results and checkpoints.

feature_construction: Feature extraction settings

• Type: object

• Required: required in train; may be omitted in predict

• Default: null

• Description: If unset, validation rejects the config or errors according to run mode. Sub-blocks voxel_level / supervoxel_level / preprocessing_* are documented below.

YAML field naming in this workflow (avoid confusing method / methods / algorithm):

• feature_construction.*.method (singular): one functional expression string (concat(voxel_radiomics(T2))).

• preprocessing_for_* .methods (plural): an ordered list of preprocessing steps; each item has its own method: winsorize step type.

• habitat_segmentation.*.algorithm: clustering backend (kmeans, gmm, slic), not a feature extractor.

• habitat_cluster_selection_method / selection_method: cluster-count validation metric (elbow, silhouette, …).

voxel_level: Voxel-level feature extraction

• method: Feature extraction method expression

  • Type: string

  • Required: yes

  • Default: none (required)

  • Description: Supports functional syntax to combine multiple extractors.

  • Expression conventions (shared by voxel_level / supervoxel_level):

    • When extracting per modality (raw, voxel_radiomics, supervoxel_radiomics, etc.), even for a single modality wrap with an outer combiner such as concat(...); inner expressions are per-modality sub-expressions; the outer layer merges results across modalities.

    • Comma-separated tokens inside parentheses are parameter name placeholders or image modality names (matching images/<subject>/<modality>/ subdirectories), not Python keyword arguments; actual paths and numeric values go in a sibling params dict.

    • Binding rule: a parameter binds to function F only when its name appears inside F(...) parentheses (modality names excepted). Keys in params that are not listed in any function’s parentheses trigger a deprecation warning.

    • Defaults: optional parameters omitted from both parentheses and params receive built-in defaults from each method’s method_param_spec (see extractor classes under habit/core/habitat_analysis/clustering_features/).

    • Bundled ``params_file``: for voxel_radiomics / supervoxel_radiomics, params_file is optional. When omitted, HABIT uses bundled presets shipped in habit/resources/radiomics/ (CT R3B12 voxel preset; full-set supervoxel preset). The resolved absolute path is logged at runtime.

    • Recommended minimal form: concat(voxel_radiomics(T2)) with params: {} — uses bundled CT R3B12 params_file, kernel_radius: 3, voxel_batch: 1000, use_torch_radiomics: auto. List a name in parentheses only when overriding (e.g. concat(voxel_radiomics(T2, kernel_radius)) with kernel_radius: 1 for MRI).

• params: Voxel-level extractor parameter dictionary

  • Type: dict

  • Required: no

  • Default: {}

  • Description: Key-value pairs passed to extractors referenced in method (e.g. params_file for voxel_radiomics may live in params depending on resolver behavior). Omit the entire params block when there are no extra parameters (defaults to empty dict).

  • Available methods and parameters:

raw(image_name):

• Description: Extract raw image voxel values (most basic feature)

• Parameters: none

• Example: raw(delay2)

concat(…):

• Description: Concatenate multiple feature vectors

• Parameters: accepts multiple feature extraction expressions

• Example: concat(raw(delay2), raw(delay3), raw(delay5))

kinetic(…):

• Description: Extract kinetic features (wash-in/wash-out slopes, etc.)

• Parameters:

  • timestamps (str, required): path to timestamps file

  • accepts multiple raw(image_name) expressions

• Example: kinetic(raw(LAP), raw(PVP), raw(delay_3min), timestamps=...)

• Extracted features:

  • wash_in_slope: wash-in slope

  • wash_out_slope_lap_pvp: wash-out slope from LAP to PVP

  • wash_out_slope_pvp_dp: wash-out slope from PVP to delay phase

local_entropy(…):

• Description: Compute local entropy (local texture complexity)

• Parameters:

  • kernel_size (int, default: 3): local neighborhood size

  • bins (int, default: 32): histogram bin count

• Example: local_entropy(raw(delay2), kernel_size=5, bins=32)

voxel_radiomics(…):

• Description: Extract voxel-level radiomics features

• Parameters:

  • params_file (str, optional): PyRadiomics parameter file path; omit for bundled CT R3B12 preset

  • kernel_radius (int, default: 3 for CT R3B12 bundled preset; 1 = 3×3×3, 3 = 7×7×7): local neighborhood radius

  • voxel_batch (int, default: 1000): voxel batch size; -1 processes all ROI voxels at once (native PyRadiomics, no batching). Positive values limit memory (GPU or large ROI: 512–1000 recommended)

  • use_torch_radiomics (str, default: auto): auto uses TorchRadiomics when torch is installed and CUDA is available, otherwise CPU PyRadiomics; true forces torch; false always CPU

  • torch_device (str, default: auto): single GPU device when torch_gpus is unset

  • torch_gpus (list/int/str): allowed GPU indices, e.g. [0, 1, 2] or "0,1,2"; overrides torch_device when set

  • torch_gpu_count (int, optional): use first N GPUs from torch_gpus

  • torch_dtype (str, default: float32): Torch compute dtype (float32 or float64; float64 closer to CPU PyRadiomics)

• Voxel GLCM note: Use config/radiomics/params_voxel_radiomics.yaml (explicit list of 21 stable GLCM features). If params_file lists only glcm: without feature names, PyRadiomics computes all 24 GLCM features; in small neighborhoods with many uniform voxels, GLCM degenerates to 1×1 matrices and MCC / Imc1 / Imc2 feature values or mutual information can crash or produce NaN on CUDA/MKL. HABIT auto-replaces with the 21 stable features and logs a warning when GLCM is unrestricted; if features are explicitly listed in params_file, user configuration is respected.

• CT voxel texture (R3B12): Literature recommends kernel_radius: 3 and binWidth: 12 HU (R3B12 configuration; better repeatability and robustness to kernel/binning than R1B25). Omit params_file to use bundled params_voxel_radiomics.yaml; kernel_radius defaults to 3. Reference: Petersen A, et al. Radiol Artif Intell. 2024;6(2):e230118. https://doi.org/10.1148/ryai.230118

• Example: concat(voxel_radiomics(T2)) with params: {} (CT R3B12 defaults); override example: concat(voxel_radiomics(T2, kernel_radius)) with kernel_radius: 1 in params

• Full examples:

  # Simple concatenation of raw images
  voxel_level:
    method: concat(raw(delay2), raw(delay3), raw(delay5))
  
  # Kinetic features
  voxel_level:
    method: kinetic(raw(LAP), raw(PVP), raw(delay_3min))
    params:
      timestamps: ./timestamps.txt
  
  # Combine local entropy and raw values
  voxel_level:
    method: concat(raw(delay2), local_entropy(raw(delay2)))
    params:
      kernel_size: 5
      bins: 32
  
  # Voxel-level radiomics (texture features, slower; single modality still needs concat)
  voxel_level:
    method: concat(voxel_radiomics(T2))
    params: {}
  

• params: Global parameters

  • Type: dict

  • Required: no

  • Default: {}

  • Description: Shared parameters for all extractors. voxel_radiomics-specific keys (voxel_batch, use_torch_radiomics, etc.) belong in params and need not appear in the method expression string; keys not listed in the expression are auto-merged and forwarded.

  • Common parameters:

    • timestamps (str): timestamps file path (for kinetic)

    • kernel_size (int): local neighborhood size (for local_entropy)

    • bins (int): histogram bin count (for local_entropy)

    • params_file (str): PyRadiomics parameter file (for voxel_radiomics)

    • kernel_radius (int): voxel radiomics neighborhood radius (for voxel_radiomics)

    • voxel_batch (int): voxel radiomics batch size (for voxel_radiomics; default 1000; -1 = no batching)

    • use_torch_radiomics (str): TorchRadiomics acceleration (auto / true / false)

    • torch_device (str): single GPU device (when torch_gpus unset)

    • torch_gpus (list/int/str): allowed GPU list

    • torch_gpu_count (int): cap on GPUs actually used

    • torch_dtype (str): Torch dtype (voxel_radiomics torch backend)

supervoxel_level: Superpixel-level feature extraction (optional)

• Block default: null (omit = no supervoxel block; two_step training usually requires it)

• supervoxel_file_keyword: Superpixel file glob pattern

  • Type: string

  • Required: no (when supervoxel_level is configured)

  • Default: *_supervoxel.nrrd

  • Description: Matches existing supervoxel segmentation files (from two_step mode).

  • Example: "*_supervoxel.nrrd"

• method: Feature aggregation/extraction method

  • Type: string

  • Required: no (recommended when supervoxel_level is configured)

  • Default: mean_voxel_features()

  • Description: Defines how voxel features aggregate to supervoxels, or direct supervoxel extraction. For multi-modality extractors such as supervoxel_radiomics, expression conventions match voxel_level (single modality still needs outer concat(...) etc.; see “Expression conventions” above).

  • Available methods and parameters:

mean_voxel_features():

• Description: Mean of voxel features within each supervoxel (most common)

• Parameters: none

• Use case: Aggregate voxel-level features (from voxel_level) to supervoxel level

• Example: mean_voxel_features()

concat(supervoxel_radiomics(<modality>, params_file), …):

• Description: Per supervoxel label, extract whole-ROI radiomics texture (not voxel kernel neighborhoods). Must sit inside concat(...) (or another outer combiner); single-modality example: concat(supervoxel_radiomics(T2, params_file)).

• Discretization: One PyRadiomics _applyBinning on the union mask of all supervoxels (sv_map > 0), then per-label cMatrices

• Matrix backend: use_supervoxel_cext default auto: use C extension batch matrix build when supervoxel_cext is compiled (pip install -e .); otherwise fallback Torch/PyRadiomics stacked matrix path. false forces Torch/PyRadiomics stacked matrices (matrix_backend=torch_cmatrices) even if C extension exists

• Feature backend: When use_torch_radiomics resolves to torch, TorchRadiomics (GPU/CPU torch); else CPU PyRadiomics (same semantics)

• Parameters (in feature_construction.supervoxel_level.params; may inherit torch keys from voxel_level.params):

  • params_file (str, optional): PyRadiomics parameter YAML; omit for bundled full-set preset

  • supervoxel_batch (int): batch group size, default 64 (not kernel radius)

  • supervoxel_union_bbox_crop (bool): crop to union bbox, default true

  • use_supervoxel_cext (str | bool): auto / true / false, default auto; must be in supervoxel_level.params (not in params_file)

  • use_torch_radiomics (str): auto / true / false

  • torch_gpus / torch_gpu_count / torch_device / torch_dtype: same as voxel level

• Note: kernel_radius is for voxel_radiomics only; supervoxel_radiomics does not use it

• Use case: Texture radiomics directly from supervoxel regions without voxel_level features

• Example: concat(supervoxel_radiomics(T2)) with params: {} (bundled preset)

• Method comparison:

  • mean_voxel_features(): depends on voxel_level features; fast; suitable for most cases

  • concat(supervoxel_radiomics(...), ...): standalone ROI radiomics; union-mask bin once + per-label extract; feature values differ from legacy per-label execute (per-label bin)

• Full examples:

  # Scenario 1: aggregate voxel features (recommended)
  supervoxel_level:
    supervoxel_file_keyword: '*_supervoxel.nrrd'
    method: mean_voxel_features()
  
  # Scenario 2: direct radiomics (single modality still needs concat; replace T2 with modality under data_dir)
  supervoxel_level:
    supervoxel_file_keyword: '*_supervoxel.nrrd'
    method: concat(supervoxel_radiomics(T2))
    params: {}
  
  # Scenario 2b: multi-modality supervoxel radiomics
  supervoxel_level:
    supervoxel_file_keyword: '*_supervoxel.nrrd'
    method: concat(
      supervoxel_radiomics(T1),
      supervoxel_radiomics(T2))
    params: {}
  

• params: Parameters

  • Type: dict

  • Required: no

  • Default: {}

  • Description: Parameters for extractors. Omit params for parameterless methods like mean_voxel_features(). Common supervoxel_radiomics keys: params_file, supervoxel_batch, supervoxel_union_bbox_crop, use_supervoxel_cext, use_torch_radiomics, torch_gpus, torch_gpu_count, torch_dtype (torch keys may inherit from voxel_level.params).

preprocessing_for_subject_level: Subject-level preprocessing (optional)

• methods: Preprocessing method list

  • Type: list

  • Required: no

  • Default: []

  • Description: Preprocess features per subject to reduce within-subject outliers and scale differences. Dispatched by PreprocessingMethodFactory with DataFrame in/out (see “Habitat feature preprocessing implementation and extension” below).

  • Note: In two_step and direct_pooling, subject level must not use column-dropping methods (variance_filter, correlation_filter), or concatenated subjects will have inconsistent columns; two_step rejects such configs at validation. one_step may use column-dropping methods at subject level (per-subject clustering).

  • Supported methods and parameters:

winsorize (winsorization):

• winsor_limits (list, default: [0.05, 0.05]): lower and upper tail truncation fractions

• global_normalize (bool, default: false): global normalization across all features

minmax (min-max normalization):

• global_normalize (bool, default: false): global normalization

zscore (Z-score standardization):

• global_normalize (bool, default: false): global standardization

robust (robust standardization):

• global_normalize (bool, default: false): global normalization

• Scales using interquartile range (IQR); robust to outliers

log (log transform):

• global_normalize (bool, default: false): global transform

• Handles negative values (shift then log)

variance_filter (low-variance filter):

• variance_threshold (float, default: 0.0): keep features with variance above threshold

• Note: drops feature columns

correlation_filter (high-correlation filter):

• corr_threshold (float, default: 0.95): drop redundant features when |correlation| exceeds threshold

• corr_method (str, default: spearman): pearson / spearman / kendall

• Note: drops feature columns

• Examples:

  # Winsorize then normalize
  - method: winsorize
    winsor_limits: [0.05, 0.05]
    global_normalize: true
  - method: minmax
    global_normalize: true
  
  # Z-score standardization
  - method: zscore
    global_normalize: false
  

preprocessing_for_group_level: Group-level preprocessing (optional)

• methods: Preprocessing method list

  • Type: list

  • Required: no

  • Default: []

  • Description: Preprocess features at cohort level; often used for discretization to stabilize clustering.

  • Applicable modes: two_step and direct_pooling only; one_step pipeline has no group step—configured group preprocessing has no effect.

  • Supported methods and parameters:

binning (feature discretization / binning):

• n_bins (int, default: 10): number of bins

• bin_strategy (str, default: uniform): binning strategy:

  • uniform: equal-width bins

  • quantile: quantile (equal-frequency) bins

  • kmeans: K-means cluster binning

• global_normalize (bool, default: false): global binning across features

winsorize (winsorization):

• winsor_limits (list, default: [0.05, 0.05]): lower and upper tail truncation fractions

• global_normalize (bool, default: false): global normalization

minmax / zscore / robust / log:

• Same as preprocessing_for_subject_level, applied after group aggregation

variance_filter / correlation_filter (recommended at group level):

• Column dropping for unsupervised feature selection; reduces noise and redundancy

• variance_filter parameter: variance_threshold

• correlation_filter parameters: corr_threshold, corr_method

• Recommendation: fit column set during training; reuse same columns at prediction

• Examples:

  # Uniform binning (recommended for habitat analysis)
  - method: binning
    n_bins: 10
    bin_strategy: uniform
    global_normalize: false
  
  # Quantile binning (equal frequency)
  - method: binning
    n_bins: 20
    bin_strategy: quantile
    global_normalize: false
  

YAML structure essentials

preprocessing_for_*_level must contain a single methods: key whose value is a list; do not duplicate methods: or place list items outside the methods: block:

# Correct
preprocessing_for_group_level:
  methods:
    - method: winsorize
      winsor_limits: [0.05, 0.05]
    - method: variance_filter
      variance_threshold: 0.0

# Wrong: duplicate methods key; variance_filter not in list
preprocessing_for_group_level:
  methods:
    - method: winsorize
  methods:
    - method: variance_filter

Preprocessing activation matrix by clustering_mode

Preprocessing block	one_step	two_step	direct_pooling
preprocessing_for_subject_level	active	active (variance_filter / correlation_filter forbidden)	active
preprocessing_for_group_level	inactive	active (Stage 2 group level)	active (after pooling, group level)

Recommended pipeline (train → predict reuse)

• two_step / direct_pooling (train): subject-level winsorize + minmax (or zscore) → group-level binning (discretization aids clustering) → optional variance_filter / correlation_filter (column-dropping; fit caches column set during training).

• two_step / direct_pooling (predict): same methods order as training; PreprocessingState loaded from habitat_pipeline.pkl—do not change thresholds on column-dropping methods to avoid column mismatch.

• one_step: configure preprocessing_for_subject_level only; group block is ignored.

Habitat feature preprocessing implementation and extension

• Unified interface: Built-in and custom methods implement BaseFeaturePreprocessing, registered via @PreprocessingMethodFactory.register.

• Execution path: preprocessing_for_subject_level → stateless subject-level apply_stateless_preprocessing; preprocessing_for_group_level → PreprocessingState.fit/transform (training caches baseline and per-step step_states; prediction reuses).

• Column-dropping methods: variance_filter, correlation_filter set changes_columns=True; two_step forbids them at subject level (see note above).

• Adding a method:

  1. See habit/core/habitat_analysis/feature_preprocessing/custom_preprocessing_template.py

  2. Register the method name in habit/core/schemas/workflows/habitat.py (PreprocessingMethod.method Literal) or rely on schema extension hooks documented in Extension and Plugin Guide

  3. If column-dropping, update DROPPING_PREPROCESSING_METHODS

  4. Ensure module import so registration decorator runs

• Compatibility: YAML format unchanged; legacy habitat_pipeline.pkl with pre-refactor PreprocessingState requires re-train.

habitat_segmentation: Habitat segmentation settings

• Type: object

• Required: required in train; recommended in predict (at least clustering_mode)

• Default: null (if fully omitted, Pydantic uses HabitatSegmentationConfig default sub-blocks; see table below)

• clustering_mode: Clustering strategy

  • Type: string

  • Required: no

  • Default: two_step

  • Allowed values:

    • one_step: cluster voxels directly.

    • two_step: build supervoxels first, then cluster supervoxels into habitats.

    • direct_pooling: pool voxels across all subjects then cluster (compute-heavy).

  • Example: two_step

supervoxel: Supervoxel clustering settings

• Applicable modes:

  • two_step: this block configures per-subject voxel → supervoxel clustering (algorithm, n_clusters, etc.).

  • one_step: algorithm and one_step_settings for per-subject voxel → habitat; n_clusters overridden when auto-selecting k.

  • direct_pooling: supervoxel block not used for clustering (schema defaults may remain).

• algorithm: Clustering or segmentation algorithm

  • Type: string

  • Default: kmeans

  • Allowed values (aligned with SupervoxelClusteringConfig):

    • kmeans: K-means; clusters by feature vectors only—supervoxels may be spatially scattered.

    • gmm: Gaussian mixture; underlying default covariance_type='full'; SupervoxelClusteringConfig does not expose this field—YAML may ignore it.

    • slic: SLIC superpixels; joint multi-channel features and 3D coordinates for spatially compact supervoxels. Dedicated parameters (compactness / sigma / enforce_connectivity and tuning) see habitat_segmentation.supervoxel — SLIC Superpixel Configuration below.

• n_clusters: Supervoxel count (or SLIC n_segments)

  • Type: integer

  • Default: 50

  • Description:

    • two_step: fixed supervoxel count per subject; must be less than ROI voxel count. Typical 30–100; small ROI 20–30.

    • one_step with empty fixed_n_clusters: upper bound for auto k search (see one_step_settings.max_clusters), not the final fixed value.

• random_state: Random seed

  • Type: integer or null

  • Default: null (inherits HabitatAnalysisConfig.random_state)

• max_iter: Maximum iterations

  • Type: integer

  • Default: 300

  • Description: kmeans / gmm: sklearn max iterations; slic differs—see habitat_segmentation.supervoxel — SLIC Superpixel Configuration.

• n_init: Number of initializations

  • Type: integer

  • Default: 10

  • Description: kmeans / gmm: sklearn n_init; slic: only in one_step auto k—see habitat_segmentation.supervoxel — SLIC Superpixel Configuration.

• compactness / sigma / enforce_connectivity: active only when algorithm: slic—see habitat_segmentation.supervoxel — SLIC Superpixel Configuration.

• one_step_settings: Nested one-step auto cluster count (see one_step_settings below). Not used for per-subject supervoxel clustering in ``two_step`` (two-step always uses fixed n_clusters).

• Full example (K-Means supervoxels):

  supervoxel:
    algorithm: kmeans
    n_clusters: 50
    random_state: 42
    max_iter: 300
    n_init: 10
  

habitat_segmentation.supervoxel — SLIC Superpixel Configuration

When habitat_segmentation.supervoxel.algorithm: slic, habit calls skimage.segmentation.slic inside the ROI, jointly segmenting multi-channel voxel features (last dimension = feature channel) with 3D spatial coordinates into n_clusters supervoxels. Anisotropic voxel spacing spacing is read automatically from the mask NIfTI header—no YAML entry required.

Use cases

• Recommended for two_step stage 1: supervoxels are more spatially compact than pure K-Means supervoxels.

• one_step may also use slic with one_step_settings for per-subject auto k.

• High-dimensional voxel_radiomics features incur much higher compute and memory than raw / mean_voxel_features (builds [Z, Y, X, n_features] feature volume in ROI bbox).

YAML path: keys under habitat_segmentation.supervoxel; repo template config/habitat/config_habitat_two_step_supervoxel_slic.yaml.

Parameter reference

• n_clusters: Target supervoxel count (n_segments)

  • Default: 50 (shared with supervoxel block)

  • Tuning: small ROI 20–30; finer texture 60–100; must be less than total ROI voxels.

• compactness: Balance between feature similarity and spatial compactness

  • Type: float

  • Default: 0.1

  • Description: Passed to skimage.segmentation.slic. Features are usually winsorized / minmax-normalized before SLIC, so 0.1 is a reasonable start (different scale from large compactness values common on RGB images).

  • Tuning:

    • Increase (0.15–0.3): more spatial grouping, smoother boundaries, fewer fragments.

    • Decrease (0.05): follows feature differences more closely; boundaries track intensity/texture change but may fragment.

• sigma: Gaussian smoothing width before SLIC

  • Type: float

  • Default: 0.0 (no smoothing)

  • Description: Gaussian smooth on multi-channel feature volume inside ROI before segmentation; units are voxels (with auto spacing for anisotropy). Suppresses high-frequency noise in voxel/texture features; does not replace winsorize etc. in preprocessing_for_subject_level.

  • Tuning:

    • 0.0: sharpest boundaries; use when preprocessing is already strong.

    • 0.3–0.5: mild denoising; try first for voxel radiomics (e.g. 0.5).

    • > 1.0: may blur real small structures—generally not recommended.

• enforce_connectivity: Enforce spatial connectivity

  • Type: bool

  • Default: true

  • Description: When true, merges isolated small regions to avoid “floating” supervoxels; keep true for medical ROIs.

• max_iter: SLIC iteration count

  • Type: integer

  • Schema default: 300 (shared field name with kmeans; do not use 300 for SLIC)

  • Recommended: 10 (maps to max_num_iter); 15–20 possible with diminishing returns.

• n_init: Number of initializations

  • Type: integer

  • Default: 10

  • Description: Only for embedded KMeans in find_optimal_clusters under one_step; no effect on SLIC fit in two_step.

• random_state: Random seed

  • Default: null (inherits top-level random_state); explicit 42 recommended for reproducibility.

Comparison with K-Means supervoxels

Algorithm	Advantages	Caveats
kmeans	Fast; simple	No spatial adjacency constraint; supervoxels may scatter
slic	Spatially compact supervoxels; good two-step local units	Slower, more memory with high-dim radiomics; needs mask / coordinates

Recommended example (two_step supervoxel stage):

habitat_segmentation:
  clustering_mode: two_step
  supervoxel:
    algorithm: slic
    n_clusters: 50
    compactness: 0.1
    sigma: 0.5
    enforce_connectivity: true
    max_iter: 10
    random_state: 42

one_step_settings note: Nested block is one_step only; under two_step, SLIC supervoxel count is fixed by n_clusters—do not rely on one_step_settings for k selection.

one_step_settings: One-step mode settings (one_step only)

• min_clusters: Minimum cluster count

  • Type: integer

  • Default: 2

  • Description: Lower bound for automatic selection

• max_clusters: Maximum cluster count

  • Type: integer

  • Default: 10

  • Description: Upper bound for automatic selection

• fixed_n_clusters: Fixed cluster count

  • Type: integer or null

  • Default: null

  • Description: If set, skips auto selection and uses this value.

• selection_method: Auto-selection metric

  • Type: string

  • Default: elbow

  • Allowed values and meaning:

    • elbow: elbow on inertia curve (default)

    • silhouette: silhouette coefficient (-1 to 1; closer to 1 = tighter clusters)

    • calinski_harabasz: Calinski-Harabasz index (higher = better)

    • davies_bouldin: Davies-Bouldin index (lower = better)

    • inertia: within-cluster sum of squares (lower = tighter; Kneedle elbow internally)

    • kneedle: Kneedle on normalized inertia curve (max deviation point)

    • gap: gap statistic

    • aic / bic: information criteria (gmm-oriented)

  • Recommendation: elbow (schema default) or silhouette for interpretability

• plot_validation_curves: Plot validation curves

  • Type: bool

  • Default: true

  • Description: Plots metrics vs cluster count to interpret auto selection

habitat: Habitat clustering settings

• algorithm: Clustering algorithm

  • Type: string

  • Default: kmeans

  • Allowed values:

    • kmeans: K-means clustering

    • gmm: Gaussian mixture model

• max_clusters: Maximum habitat count

  • Type: integer

  • Required: no

  • Default: 10

  • Description: Upper bound when auto-selecting habitat count. Recommended: 5–10.

  • Example: 10

• min_clusters: Minimum habitat count

  • Type: integer

  • Default: 2

  • Description: Lower bound when auto-selecting habitat count.

• habitat_cluster_selection_method: Auto-selection metrics

  • Type: list or string

  • Default: elbow (YAML may use string or single-element list)

  • Allowed values and meaning:

    • inertia: within-cluster SS (lower better for kmeans; Kneedle internally)

    • kneedle: Kneedle on normalized inertia curve

    • silhouette: silhouette (-1 to 1; closer to 1 better)

    • calinski_harabasz: Calinski-Harabasz (higher better)

    • davies_bouldin: Davies-Bouldin (lower better)

    • aic: Akaike information criterion (lower better; gmm only)

    • bic: Bayesian information criterion (lower better; gmm only)

  • Description: Multiple metrics may be specified; system combines them to pick best habitat count.

  • Example: [inertia, silhouette]

• fixed_n_clusters: Fixed habitat count

  • Type: integer or null

  • Default: null

  • Description: If set to a number, skips auto selection and uses that habitat count.

• random_state: Random seed

  • Type: integer or null

  • Default: null (inherits HabitatAnalysisConfig.random_state)

  • Description: direct_pooling / two_step group habitat clustering; one_step per-subject voxel→habitat (overrides supervoxel.random_state when set explicitly).

• max_iter: Maximum iterations

  • Type: integer

  • Default: 300 (both kmeans and gmm in HabitatClusteringConfig)

• n_init: Number of initializations

  • Type: integer

  • Default: 10 (both kmeans and gmm)

• Full examples:

  # Auto-select habitat count (recommended)
  habitat:
    algorithm: kmeans
    max_clusters: 10
    min_clusters: 2
    habitat_cluster_selection_method:
      - inertia
      - silhouette
    fixed_n_clusters: null
    random_state: 42
  
  # Fixed habitat count
  habitat:
    algorithm: kmeans
    fixed_n_clusters: 5
    random_state: 42
  

postprocess_supervoxel / postprocess_habitat: Connected-component postprocessing

• Type: dict

• Required: no

• Default: enabled: false

• Description:

  • postprocess_supervoxel applies to supervoxel label maps (mainly two_step stage).

  • postprocess_habitat applies to final habitat label maps (one_step/two_step/direct_pooling).

  • Current implementation uses SimpleITK fast path: remove small components per label, then reassign by nearest seed label.

  • Reduces fragmentation while preserving ROI voxel coverage.

• Sub-parameters (aligned with ConnectedComponentPostprocessConfig):

  • enabled (bool, default: false)

  • min_component_size (int, default: 30, ≥1)

  • connectivity (1 / 2 / 3): 6/18/26-neighbor

  • reassign_method: currently only neighbor_vote (placeholder)

  • max_iterations (int, default 3, ≥1): cleanup iteration cap

• Example:

  habitat_segmentation:
    postprocess_supervoxel:
      enabled: false
      min_component_size: 30
      connectivity: 1
      reassign_method: neighbor_vote
      max_iterations: 3
  
    postprocess_habitat:
      enabled: true
      min_component_size: 30
      connectivity: 1
      reassign_method: neighbor_vote
      max_iterations: 3
  

plot_curves: Generate and save plots

• Type: bool

• Default: true

• Description: During group clustering auto k search, if a logger is passed, logs Trying N cluster(s) [i/total] and Cluster search finished: selected k=... for progress (forced off in predict mode).

Habitat Stage-1 Parallelism and Checkpoint Resume (Top-Level Field Reference)

These fields sit at the top level of the habitat YAML (same level as data_dir). Both train and predict use processes, checkpoint paths, and resume where applicable (predict checkpoints default to <out_dir>/.habitat_predict_checkpoint). CLI: habit get-habitat --resume.

Field	Type	Default	config_hash	Summary
processes	int	2	no	Stage 1 max parallel workers; peak memory ≈ processes × per-subject memory
cap_processes_to_gpu_pool	bool	false	no	Torch CUDA radiomics: true caps workers to len(torch_gpus); false keeps processes and shares GPUs across workers
individual_subject_timeout_sec	float / null	900	no	Per-subject wall-clock cap (seconds); null disables
individual_subject_graceful_shutdown_sec	float	15	no	Seconds to wait after terminate() before kill() on timeout
individual_subject_spawn_timeout_sec	float / null	120	no	Spawn-phase cap; null disables (avoids parent stuck on import)
on_subject_failure	str	continue	no	continue log failure and proceed; fail_fast abort on first failure
oom_backoff	bool	true	no	After MemoryError, reduce workers by oom_reduce_workers_by (not native crashes)
oom_reduce_workers_by	int	1	no	Workers removed per OOM event
resume	bool	true	—	Skip completed subjects from checkpoint; train and predict
checkpoint_dir	str / null	null	no	Default <out_dir>/.habitat_checkpoint
force_rerun_subjects	list[str]	[]	no	Subject IDs to force rerun on resume
retry_failed_subjects	bool	false	no	On next resume, rerun all failed_subjects in manifest
individual_subject_auto_retry_rounds	int	2	no	In-run Stage 1 auto-retry rounds; 0 disables
individual_subject_parallel_mode	str	persistent	no	persistent long-lived workers; isolated spawn per subject
persistent_worker_max_consecutive_failures	int	1	no	Under persistent, restart slot after N consecutive failures
persistent_worker_recycle_after_tasks	int	0	no	Under persistent, recycle worker after N successes; 0 off
clear_checkpoint_on_success	bool	false	no	Delete checkpoint dir after full train success

processes (habitat analysis top level): Parallel workers for per-subject steps

• Type: integer

• Default: 2 (must be > 0)

• Description: See table above; interacts with cap_processes_to_gpu_pool and torch_gpus in feature_construction.*.params.

cap_processes_to_gpu_pool (habitat analysis top level): Cap Stage 1 workers to GPU pool size

• Type: bool

• Default: true

• Description: When use_torch_radiomics uses CUDA (true or auto with CUDA detected):

  • true (default): effective workers min(processes, len(torch_gpus)), one GPU per slot (gpu_slot_index), less VRAM contention;

  • false: full processes; workers share GPUs via gpu_slot_index % len(torch_gpus)—good for “single GPU, many CPU” parallel non-GPU steps, but GPU radiomics may OOM on same card.

• Not in config_hash; may change on resume.

• No effect on CPU-only (use_torch_radiomics: false or no CUDA).

individual_subject_timeout_sec (habitat analysis top level): Per-subject wall-clock cap in parallel Stage 1

• Type: float / int (seconds) or null

• Default: 900 (15 minutes); omit in YAML to use default.

• Description: On timeout, skip subject (record failure) and continue; null disables per-subject timeout. Child processes may still run in background until they exit.

individual_subject_graceful_shutdown_sec (habitat analysis top level): Grace period after timeout before kill

• Type: float (seconds)

• Default: 15

• Description: After individual_subject_timeout_sec, parent calls terminate(), waits this many seconds, then kill() on isolated child.

individual_subject_spawn_timeout_sec (habitat analysis top level): Spawn-phase wall-clock cap

• Type: float / int (seconds) or null

• Default: 120

• Description: Cap from dispatch to subject processing start; on timeout, mark subject failed and continue—avoids parent blocked forever on spawn/import. null = no spawn time limit.

on_subject_failure (habitat analysis top level): Failure policy for parallel per-subject Stage 1

• Type: string

• Default: continue

• Allowed values:

  • continue: record failed subjects; continue to Stage 2 if any succeeded

  • fail_fast: abort entire run on first failure or timeout

oom_backoff (habitat analysis top level): Reduce parallelism after memory errors

• Type: bool

• Default: true (built-in default); repo config/habitat/*.yaml examples often use false—tune for your RAM

• Description: When true, isolated child MemoryError reduces worker count for pending subjects by oom_reduce_workers_by (minimum 1). Does not handle native crashes (e.g. Windows exit 3221225477 / 0xC0000005).

oom_reduce_workers_by (habitat analysis top level): Workers removed per OOM

• Type: integer

• Default: 1

• Description: Only when oom_backoff: true.

resume (habitat analysis top level): Stage 1 checkpoint resume

• Type: bool

• Default: true

• Description: When true, reads manifest.json from checkpoint_dir (default <out_dir>/.habitat_checkpoint for train, <out_dir>/.habitat_predict_checkpoint for predict), skips completed_subjects, loads per-subject pickles; subjects in failed_subjects are not auto-retried on next resume unless retry_failed_subjects: true or listed in force_rerun_subjects. Within the same run, individual_subject_auto_retry_rounds retries Stage 1 failures by default. Applies to both train and predict.

• CLI: habit get-habitat --resume equivalent to resume: true.

• See also: checkpoint / resume fields on this page.

• Parallel reliability plan: docs/HABITAT_PARALLEL_RELIABILITY_PLAN.md at repo root (GPU worker slots, processes cap, Phase 2/3 roadmap).

strict_checkpoint_hash (habitat analysis top level): Error on incompatible checkpoint hash

• Type: bool

• Default: false

• Description: With resume: true. When true, manifest.json config_hash or run_mode mismatch raises CheckpointConfigHashError and preserves checkpoint; when false (default), logs warning and deletes checkpoint for fresh run. Legacy hash migration for Stage-2-only changes still allows resume.

• Not in config_hash; may change on resume.

checkpoint_dir (habitat analysis top level): Checkpoint root directory

• Type: string or null

• Default: null (train → <out_dir>/.habitat_checkpoint; predict → <out_dir>/.habitat_predict_checkpoint)

• Description: Must match previous run for resume; may differ from out_dir when set explicitly.

force_rerun_subjects (habitat analysis top level): Subject IDs to force rerun

• Type: list of strings

• Default: []

• Description: With resume: true, still reprocess listed subjects (removed from completed/failed and rerun).

retry_failed_subjects (habitat analysis top level): Rerun all failed subjects from checkpoint

• Type: bool

• Default: false

• Description: With resume: true, enqueue all failed_subjects from manifest.json for Stage 1 rerun. Successful subjects still skipped unless also in force_rerun_subjects.

individual_subject_auto_retry_rounds (habitat analysis top level): In-run auto-retry for failed subjects

• Type: integer

• Default: 2

• Description: After first Stage 1 parallel pass, if failed_subjects remain, rerun up to this many rounds in the same process (failed only). 0 disables (legacy behavior). Unlike retry_failed_subjects (next resume only), this applies in the current get-habitat / fit() call. With on_subject_failure: fail_fast, error only after all retry rounds exhausted.

individual_subject_parallel_mode (habitat analysis top level): Stage 1 parallel execution strategy

• Type: string

• Default: persistent

• Allowed values: isolated, persistent

• Description: persistent (default): one long-lived child per worker slot, reused within the same run (including auto-retry rounds), amortizing import/spawn. isolated: spawn per subject (stronger isolation; unpickleable pipeline or spawn debugging). Single GPU persistent is still serial—main benefit is startup cost. When processes=1 and individual_subject_timeout_sec: null, both modes run sequentially in main process without spawn. Used in both train and predict individual-level stages.

persistent_worker_max_consecutive_failures (habitat analysis top level): Persistent worker restart threshold

• Type: integer

• Default: 1

• Description: Only when individual_subject_parallel_mode: persistent. After this many consecutive failures on a slot, parent terminates and restarts that worker.

persistent_worker_recycle_after_tasks (habitat analysis top level): Periodic persistent worker recycle

• Type: integer

• Default: 0

• Description: persistent only. Worker exits after this many successful tasks and parent respawns—mitigates slow GPU memory leaks. 0 disables.

clear_checkpoint_on_success (habitat analysis top level): Delete checkpoint after successful train

• Type: bool

• Default: false

• Description: When true, deletes entire checkpoint directory after Stage 1 + Stage 2 complete successfully.

config_hash and resume compatibility

• In hash (Stage 1 per-subject; change clears checkpoint): data_dir, feature_construction.voxel_level / preprocessing_for_subject_level / supervoxel_level, habitat_segmentation.clustering_mode, per-subject clustering block (two_step → supervoxel; one_step → supervoxel + habitat).

• Not in hash (may resume: true): preprocessing_for_group_level, group habitat.* for two_step/direct_pooling, processes, cap_processes_to_gpu_pool, strict_checkpoint_hash, individual_subject_timeout_sec, individual_subject_graceful_shutdown_sec, individual_subject_spawn_timeout_sec, plot_curves, save_results_csv, habitats_results_format, save_images, verbose, debug, on_subject_failure, oom_backoff, oom_reduce_workers_by, retry_failed_subjects, individual_subject_auto_retry_rounds, individual_subject_parallel_mode, persistent_worker_max_consecutive_failures, persistent_worker_recycle_after_tasks, force_rerun_subjects, out_dir, etc.

• manifest.json also stores individual_config_hash (same as config_hash); legacy full-hash-only manifests migrate hash on Stage 2-only changes and keep pkls.

• On resume: true startup, program compares hash. Individual hash mismatch without Stage 2 drift: default (strict_checkpoint_hash: false) warns and deletes checkpoint; true raises CheckpointConfigHashError.

Checkpoint directory layout

<checkpoint_dir>/
├── manifest.json      # completed_subjects, failed_subjects, config_hash, individual_config_hash, stage
└── subjects/
    └── {subject_id}.pkl

Checkpoint boundaries by clustering_mode

• two_step / one_step: saved after merge_supervoxel_features (supervoxel_df)

• direct_pooling: saved after individual_preprocessing (voxel-level features; larger pkls)

• Stage 2 (combine / concat / group clustering) has no checkpoint

save_results_csv: Save habitat results table

• Type: bool

• Default: true

• Description: When true, writes habitats.parquet (default) or habitats.csv per habitats_results_format.

habitats_results_format: Habitat results table file format

• Type: string

• Default: parquet

• Allowed values: parquet, csv

• Description: parquet smaller and faster—good for large direct_pooling voxel tables; csv opens in Excel. Output filename is habitats.parquet or habitats.csv.

• Example:

  save_results_csv: true
  habitats_results_format: parquet
  

random_state (habitat analysis top level)

• Type: integer

• Default: 42

debug (habitat analysis top level)

• Type: bool

• Default: false

habitat_pipeline.pkl (training artifact at <out_dir>/habitat_pipeline.pkl)

• Contents: joblib-serialized fitted HabitatPipeline (group clustering model, PreprocessingState, config, etc.).

• Auto slimming on save (HabitatPipeline.save() calls prepare_pipeline_for_save):

  • Removes training labels_ (prediction needs centers/model params only)

  • Does not write mask_info_cache (NRRD write loads mask from data_dir on demand via FeatureService.load_mask_info)

  • Does not write _train_checkpoint (checkpoints remain in checkpoint_dir)

• Size:

  • No longer inflated by save_images: true: mask volume decoupled from pkl; direct_pooling large queues typically tens to hundreds of MB (mainly feature dimension and model params vs subject count)

  • Legacy pkls embedding mask_array need re-train and save to shrink

save_images: Save image outputs from the run (*_habitats.nrrd, etc.)

• Type: bool

• Default: true

• Description: Maps to HabitatAnalysisConfig.save_images. When true, train/predict write habitat label maps; masks load from config.data_dir at write time—not stored in habitat_pipeline.pkl. When false, downstream analysis via habitats.parquet / habitats.csv without NRRD output.

verbose: Verbose run logging

• Type: bool

• Default: true