Machine Learning Configuration

Machine learning configuration parameters

This section documents machine learning configuration. CLI: habit model -c <yaml> (K-fold: habit cv). Demo: config/machine_learning/config_machine_learning_radiomics.yaml; prediction: config/machine_learning/config_machine_learning_predict.yaml.

Example configuration file:

run_mode: train
input:
  - path: ./results/features/combined_features.csv
    name: training_data
    subject_id_col: Subject
    label_col: label
output: ./demo_data/results/ml/train
random_state: 42

split_method: stratified
test_size: 0.3

resampling:
  enabled: false
  method: random_over
  position: before_model
  ratio: 1.0

normalization:
  method: z_score

feature_selection_methods:
  - method: variance
    params:
      threshold: 0.0
  - method: correlation
    params:
      threshold: 0.9

models:
  RandomForest:
    params:
      n_estimators: 100
      random_state: 42
  LogisticRegression:
    params:
      max_iter: 1000

is_visualize: true
is_save_model: true

visualization:
  enabled: true
  plot_types: [roc, dca, calibration, pr, confusion, shap]
  dpi: 600
  format: pdf

Prediction mode YAML example (run_mode: predict):

run_mode: predict
pipeline_path: ./demo_data/results/ml/clinical/models/LogisticRegression_final_pipeline.pkl

input:
  - path: ./demo_data/ml_data/clinical_feature.csv
    subject_id_col: subject_id
    label_col: label   # required in predict input

output: ./demo_data/results/ml/clinical/predictions

evaluate: true
output_label_col: predicted_label
output_prob_col: predicted_probability

run_mode (YAML)

Type: string
Default: train
Allowed values: train, predict
Description: train and predict share the same ML config structure. predict requires pipeline_path; models is ignored in predict mode.

mode / run_mode (CLI)

Command: habit model --mode <train|predict>
Description: --mode overrides YAML run_mode (see cmd_ml.run_ml). CLI takes precedence.

pipeline_path

Type: string
Required: required when run_mode is predict
Default: null
Description: path to saved *_final_pipeline.pkl.

random_state (MLConfig top level)

Type: integer
Required: no
Default: 42
Description: split, K-fold, resampling fallback, models without seed in models.*.params, etc.

input: input data configuration

Type: list
Required: yes
Description: each element is InputFileConfig. Predict mode uses only input[0].path as the data table.
Sub-parameters:
- path: feature CSV/Excel path (required, no default).
- name: dataset name; default "".
- subject_id_col: subject ID column (required, no default).
- label_col: label column (required, no default).
- features: use only these columns as features; default null (auto-infer numeric feature columns).
- features_from_log: parse feature column names from log/auxiliary file; default null.
- split_col: custom split grouping column; default null.
- pred_col: existing prediction column name; default null.

output: output directory

Type: string
Required: yes
Default: none (required)
Description: results, models, and plots directory. CLI writes logs here: processing.log for training, prediction.log for prediction (see habit.commands.cmd_ml).

split_method: data split method

Type: string
Default: stratified
Allowed values: random, stratified, custom

test_size: test set proportion

Type: float
Default: 0.3
Range: (0, 1)

train_ids_file / test_ids_file

Type: string path (optional)
Default: null
Description: used when split_method: custom; text files with one subject ID per line for fixed train/test split.

n_splits / stratified

Type: integer / boolean
Default: n_splits=5, stratified=true
Description: for habit cv (K-fold); field definitions in MLConfig.

normalization: feature normalization settings

method: normalization method
- Type: string
- Default: z_score
- Allowed values:
  - z_score: Z-Score standardization (StandardScaler)
  - min_max: min-max scaling (MinMaxScaler)
  - robust: robust scaling (RobustScaler)
  - max_abs: max absolute value scaling (MaxAbsScaler)
  - normalizer: L1/L2 normalization (Normalizer)
  - quantile: quantile transform (QuantileTransformer)
  - power: power transform (PowerTransformer)
params: method-specific parameters
- Type: dict
- Default: {}
- Description: pass different parameters per method. Omit params for default behavior (e.g. z_score, max_abs).
- Parameters supported per method:

z_score (StandardScaler):

with_mean (bool, default: true): center data before scaling

with_std (bool, default: true): scale to unit variance

min_max (MinMaxScaler):

feature_range (list, default: [0, 1]): target range, e.g. [0, 1] or [-1, 1]

robust (RobustScaler):

with_centering (bool, default: true): center data before scaling

with_scaling (bool, default: true): scale by IQR

quantile_range (list, default: [25.0, 75.0]): quantile range for scaling (IQR)

max_abs (MaxAbsScaler):

no special parameters (defaults suffice)

quantile (QuantileTransformer):

n_quantiles (int, default: 1000): number of quantiles

output_distribution (str, default: uniform): uniform or normal

subsample (int, default: 10000): max samples for quantile estimation

power (PowerTransformer):

method (str, default: yeo-johnson): yeo-johnson or box-cox

Examples:

# Z-Score (no extra params needed)
normalization:
  method: z_score

# Min-max to [-1, 1]
normalization:
  method: min_max
  params:
    feature_range: [-1, 1]

# Robust scaling (outlier-resistant)
normalization:
  method: robust
  params:
    quantile_range: [25.0, 75.0]

resampling: training set resampling (ResamplingConfig)

YAML key: resampling (recommended). Legacy key sampling is auto-mapped to resampling on load (see MLConfig._migrate_legacy_sampling_key).
Required: no
Default: enabled: false (no class resampling on training set)
Description: resampling applies to training data only; validation/test sets are not resampled.
enabled: whether enabled
- Type: boolean
- Default: false
method: algorithm
- Type: string
- Default: random_over
- Allowed values: random_over | random_under | smote (SMOTE requires imbalanced-learn)
position: position in the pipeline
- Type: string
- Default: before_model
- Allowed values: before_feature_selection, before_normalization, after_normalization, before_model
- Description: order relative to feature selection / normalization / modeling (see habit.core.machine_learning workflow).
ratio: resampling ratio; must be > 0
- Default: 1.0
random_state: random seed (default null, inherits MLConfig.random_state; explicit value overrides top level)
When it runs: training calls internal _resample_training_data before model fit; holdout and K-fold share this logic.
Log keywords (confirm execution): Sampling enabled, Sampling completed, etc.

Example:

resampling:
  enabled: true
  method: random_over
  position: before_model
  ratio: 1.0
  random_state: 42

feature_selection_methods: feature selection method list

Type: list
Default: [] (empty list = no feature selection step)
Description: sequential feature selection steps; each method has specific parameters.
Methods and parameters:

variance (variance threshold):

threshold (float, default: 0.0): remove features with variance below this

top_k (int, optional): keep top k highest-variance features (overrides threshold if set)

top_percent (float, optional): keep top x% highest-variance features (0–100)

plot_variances (bool, default: true): plot variance distribution

correlation (correlation filter):

threshold (float, default: 0.8): remove one of pairs with correlation above this

method (str, default: spearman): pearson, spearman, kendall

visualize (bool, default: false): generate correlation heatmap

anova (ANOVA):

p_threshold (float, default: 0.05): p-value threshold

n_features_to_select (int or float, optional): select top features (overrides p_threshold if set); integer >= 1 = absolute count (e.g. 20), value in (0, 1) = ratio of candidate features (e.g. 0.2 keeps top 20%)

plot_importance (bool, default: true): plot feature importance

chi2 (chi-square test):

p_threshold (float, default: 0.05): p-value threshold

n_features_to_select (int or float, optional): select top features; integer >= 1 = absolute count, value in (0, 1) = ratio of candidate features

plot_importance (bool, default: true): plot feature importance

Note: non-negative features only

lasso (Lasso regularization):

cv (int, default: 10): cross-validation folds

n_alphas (int, default: 100): number of alpha values

alphas (list, optional): custom alpha list

random_state (int, default: 42): random seed

visualize (bool, default: false): coefficient path plot

rfecv (recursive feature elimination + CV):

estimator (str, default: RandomForestClassifier): estimator, options:

classifiers: LogisticRegression, RandomForestClassifier, SVC, GradientBoostingClassifier, XGBClassifier

regressors: LinearRegression, RandomForestRegressor, SVR, GradientBoostingRegressor, XGBRegressor

step (int, default: 1): features removed per iteration

cv (int, default: 5): CV folds

scoring (str, default: roc_auc): scoring metric

min_features_to_select (int, default: 1): minimum features to keep

n_jobs (int, default: -1): parallel jobs (-1 = all CPUs)

random_state (int, optional): random seed

statistical_test (t-test / Mann-Whitney U, automatic or forced):

p_threshold (float, default: 0.05)

n_features_to_select (int or float, optional): overrides p threshold if set; integer >= 1 = absolute count, value in (0, 1) = ratio of candidate features (e.g. 0.2 keeps top 20%)

normality_test_threshold (float, default: 0.05): Shapiro-Wilk normality threshold

force_test (str, optional): ttest or mannwhitney; auto-select by normality if unset

plot_importance (bool, default: true)

icc (stability filter from ICC JSON):

icc_results / icc_results_path (str): JSON path from habit icc

keys / groups (list): group names to check in ICC results

threshold (float, default: 0.75)

metric (str, optional): e.g. ICC3, ICC2

mrmr (minimum redundancy maximum relevance):

n_features (int, default: 10)

task_type (str, default: classification): classification or regression

vif (variance inflation factor, collinearity removal):

max_vif (float, default: 10.0)

visualize (bool, default: false)

stepwise (Python stepwise logistic regression):

direction (str, default: backward): forward, backward, both

threshold_in / threshold_out (float, default: 0.05): when criterion='pvalue'

criterion (str, default: aic): aic, bic, or pvalue

verbose (bool, default: false)

stepwise_r (R stepwise regression; requires R):

same parameters as stepwise; method key name is stepwise_r

univariate_logistic (univariate logistic regression):

alpha (float, default: 0.05): significance level

Examples:

# Variance threshold
feature_selection_methods:
  - method: variance
    params:
      threshold: 0.0
      plot_variances: true

# Correlation filter + ANOVA
feature_selection_methods:
  - method: correlation
    params:
      threshold: 0.9
      method: spearman
  - method: anova
    params:
      p_threshold: 0.05

models: model training settings

Type: dict (model name → ModelConfig)
Default: null (required and non-empty when run_mode: train; ignored when predict)
Description: one or more models to train.
Supported model types and common parameters:

LogisticRegression:

max_iter (int, default: 1000): max iterations

C (float, default: 1.0): inverse regularization strength

penalty (str, default: l2): l1, l2, elasticnet, none

solver (str, default: liblinear): newton-cg, lbfgs, liblinear, sag, saga

random_state (int, default: 42)

class_weight (str/dict, optional)

RandomForest:

n_estimators (int, default: 100): number of trees

max_depth (int, optional): max tree depth

min_samples_split (int, default: 2): min samples to split

min_samples_leaf (int, default: 1): min samples per leaf

max_features (str/int, default: sqrt): max features per split

random_state (int): random seed

XGBoost:

n_estimators (int, default: 100): boosting rounds

max_depth (int, default: 3): max tree depth

learning_rate (float, default: 0.1): learning rate

subsample (float, default: 0.8): row subsample ratio

colsample_bytree (float, default: 0.8): column subsample ratio

random_state (int): random seed

SVM (implemented as LinearSVC + CalibratedClassifierCV):

C (float, default: 1.0): regularization

max_iter (int, default: 1000): max iterations for LinearSVC

class_weight (str/dict, optional)

random_state (int, default: 42)

KNN:

n_neighbors (int, default: 5): number of neighbors

weights (str, default: uniform): uniform, distance

metric (str, default: minkowski): distance metric

DecisionTree:

max_depth (int, optional)

min_samples_split (int, default: 2)

min_samples_leaf (int, default: 1)

random_state (int)

MLP:

hidden_layer_sizes (tuple/list, default: (100,))

activation (str, default: relu)

max_iter (int, default: 200)

random_state (int)

AdaBoost / GradientBoosting:

n_estimators (int, default: 50 for AdaBoost, 100 for GradientBoosting)

learning_rate (float, default: 1.0 for AdaBoost, 0.1 for GradientBoosting)

random_state (int)

GaussianNB / MultinomialNB / BernoulliNB:

mostly sklearn defaults; MultinomialNB requires non-negative features

AutoGluonTabular (YAML model key AutoGluonTabular):

time_limit (int): training time limit (seconds)

presets (str, default: medium_quality): best_quality, high_quality, medium_quality

Example:

# Train multiple models
models:
  LogisticRegression:
    params:
      max_iter: 1000
      C: 1.0
      random_state: 42

  RandomForest:
    params:
      n_estimators: 200
      max_depth: 10
      random_state: 42

  XGBoost:
    params:
      n_estimators: 100
      max_depth: 5
      learning_rate: 0.1
      random_state: 42

is_visualize: enable visualization

Type: boolean
Default: true

visualization: visualization settings (VisualizationConfig)

enabled: default true
plot_types: default [roc, dca, calibration, pr, confusion, shap]; values match listed type names
dpi: default 600
format: default pdf

is_save_model: save trained pipeline to output (default true).

Predict-mode-only fields (when run_mode: predict)

evaluate (bool, default false): compute metrics after prediction if labels exist.
output_label_col (default predicted_label): predicted class column in output table.
output_prob_col (default predicted_probability): probability column in output table.
probability_class_index: class index for probability column in multiclass (None = all or per implementation).
binary_positive_class_index (default 1): positive class index in probability vector for binary classification.

Model comparison (`habit compare`)

Multi-model ROC / DCA / DeLong comparison uses a separate schema (ModelComparisonConfig). Full field reference: Model Comparison Configuration.

Machine Learning Configuration

Machine learning configuration parameters

Model comparison (habit compare)

Model comparison (`habit compare`)