search_hyperparameters

Note

This page is a reference documentation. It only explains the function signature, and not how to use it. Please refer to the Habitat Guide and Python API guide (v2.0) for usage.

search_hyperparameters(table: FeatureTable, spec: MLSpec, param_grid: Mapping[str, Sequence[Any]] | Sequence[Mapping[str, Sequence[Any]]], *, n_splits: int = 5, seed: int | None = None, strategy: str = 'grid', n_iter: int = 10, objective: str | None = None, refit: bool = True) → SearchResult[source]

Tune hyperparameters by K-fold search and write the winners into the spec.

The search itself is scikit-learn’s (GridSearchCV / RandomizedSearchCV) driving a TablePipeline, which IS an sklearn.pipeline.Pipeline; nothing about the search is reimplemented here. What this recipe adds is the two things a study needs and sklearn does not provide:

  • The folds are HABIT’s own. They come from habit.evaluation.split.kfold_indices() with the spec’s seed, so a search over K folds partitions the rows exactly as cross_validate() would with the same n_splits and seed. Each candidate is therefore fitted on training rows only – preprocessing statistics and feature selection included, since they are steps of the pipeline being cloned per fold, never precomputed on all the rows.

  • The winners land in the ``MLSpec``. A tuned model is a DEFINITION, not just a fitted object: written back into the spec it keeps its fingerprint, serialises to YAML, and can be re-run by someone else. A search that returned only best_estimator_ would end the provenance chain at the point the parameters were chosen.

Grid key syntax. Keys address one HABIT component parameter through the pipeline’s step names: "<step>__component__<parameter>". The terminal model’s step is called "model" and every transformation step is named after its registered spec name, so {"model__component__C": [0.1, 1, 10], "variance__component__threshold": [0.0, 0.01]} tunes the classifier’s regularisation and the variance filter’s threshold. Keys of any other shape are rejected up front rather than searched: this recipe can only write a value back into the spec if it knows which component and which parameter it belongs to, and a search whose result cannot be recorded is worse than no search.

The objective. objective names a registered HABIT metric (not an sklearn scorer string), and the metric’s own greater_is_better decides the direction, so a “lower is better” metric is minimised without the caller negating anything. Omitted, the objective is the FIRST metric of spec.metrics, and for a spec with no metric panel it is auc. Scoring goes through TablePipeline.evaluate(), so the search optimises exactly the quantity the final report prints, in the same vocabulary.

Parameters:

  • table – Feature table with a declared outcome; the rows the search partitions into folds.

  • spec – The modelling definition to tune. Everything not named in the grid is left exactly as declared.

  • param_grid – One mapping of key to candidate values, or a sequence of such mappings (sklearn’s disjoint-grids form). For strategy="random" a value may also be a scipy distribution.

  • n_splits – Number of search folds; must be at least 2.

  • seed – Optional seed override, folded into the spec before anything runs, and therefore driving the fold shuffling, the component seeding and the random search’s own sampling.

  • strategy – "grid" for an exhaustive search over the product of the candidate lists, "random" for n_iter sampled candidates.

  • n_iter – Candidate budget for strategy="random"; ignored by "grid", where the budget is the grid.

  • objective – Registered metric name to optimise, or None for the fallback chain described above.

  • refit – Fit the tuned spec on the whole table and return the result in SearchResult.model. False returns the tuned spec only, which is what nested cross-validation needs (it refits on its own outer training rows).

Returns:

The tuned spec, the winning parameters and score, the per-candidate trial table, and – unless refit=False – the final model.

Raises:

HABITAPIError – If the table declares no outcome, n_splits is below 2, the strategy is unknown, the grid is empty, or a grid key does not address a tunable component parameter of this pipeline.

Examples

>>> from habit.datasets import make_synthetic_feature_table
>>> from habit.spec import MLSpec, Spec
>>> import habit.recipes as recipes
>>> table = make_synthetic_feature_table(n_rows=60, n_features=8, rng=42)
>>> spec = MLSpec(
...     name="demo",
...     steps=(Spec("zscore"),),
...     classifier=Spec("LogisticRegression", {"max_iter": 500}),
...     metrics=(Spec("auc"),),
... )
>>> tuned = recipes.search_hyperparameters(
...     table, spec, {"model__component__C": [0.01, 1.0]},
...     n_splits=3, seed=42,
... )
>>> tuned.spec.classifier.params["C"] in (0.01, 1.0)
True
>>> tuned.objective
'auc'