gunz_ml.management package

Submodules

gunz_ml.management.artifacts module

Juno Artifact SDK for S3/MinIO.

Handles discovery, retrieval, and smart extraction of artifacts from Juno’s artifact storage.

class gunz_ml.management.artifacts.ArtifactStore(mlflow_uri: str = 'http://juno.tnt.uni-hannover.de:5200', s3_endpoint: str = 'http://juno.tnt.uni-hannover.de:9000', access_key: str = 'minioadmin', secret_key: str = 'minioadmin')

Bases: object

Manager for Juno S3 artifacts (MinIO backend).

download_file(run_id: str, artifact_path: str, local_dir: str | Path) → Path

Downloads a specific artifact file.

fetch_points(run_id: str) → ndarray | None

Smart-fetches point coordinates from a run. Searches for ‘points.npz’ or ‘trajectory.npz’.

list_recursive(run_id: str, path: str = '') → List[str]

Recursively lists all artifact paths for a run.

gunz_ml.management.core module

Gunz ML - Experiment Management Core Logic. Standardized discovery, monitoring, and cleanup for HPO experiments.

class gunz_ml.management.core.ExperimentDiscovery(tracking_uri: str, prefix: str)

Bases: object

Finds and filters MLflow experiments.

get_experiments(pattern: str | None = None) → List[Dict[str, Any]]

class gunz_ml.management.core.FastGC(db_uri: str, s3_client: Any, bucket: str)

Bases: object

High-performance MLflow GC using direct DB and S3 access.

purge_experiment(experiment_id: str, dry_run: bool = True)

class gunz_ml.management.core.ProjectMetadata(start_path: Path)

Bases: object

Extracts project-level metadata from Hydra configurations.

get_optuna_storage_url() → str | None

Constructs the Optuna storage URL.

class gunz_ml.management.core.PruneService(optuna_storage: str, mlflow_uri: str, s3_client: Any = None, bucket: str = 'mlflow')

Bases: object

High-performance service to obliterate broken trials from SQL and S3.

find_broken_trials(study_name: str, orphans: bool = True, failed: bool = True) → List[int]

Identifies trial IDs for deletion based on brokenness.

prune(study_name: str, trial_ids: List[int], dry_run: bool = True) → Dict[str, int]

Obliterates specific trials from Optuna, MLflow, and S3.

class gunz_ml.management.core.SlurmTracker(user: str | None = None)

Bases: object

Interface to SLURM to track active jobs.

is_job_active(job_id: str) → bool

class gunz_ml.management.core.StudyCleaner(storage_url: str, slurm: SlurmTracker)

Bases: object

Detects and fails orphaned trials based on Slurm status.

cleanup_study(study_name: str, dry_run: bool = True) → int

class gunz_ml.management.core.StudyDiscovery(storage_url: str, prefix: str)

Bases: object

Finds and filters Optuna studies.

filter_studies(pattern: str) → List[str]

get_all_studies() → List[str]

class gunz_ml.management.core.StudyInitializer(script_name: str, project_root: Path)

Bases: object

Orchestrates mass-initialization of Hydra-based experiments.

initialize(config_name: str, overrides: List[str] = None) → bool

class gunz_ml.management.core.StudyMonitor(storage_url: str)

Bases: object

Aggregates metrics and status across multiple studies.

get_study_stats(study_names: List[str]) → DataFrame

class gunz_ml.management.core.StudyTruncator(optuna_storage: str, mlflow_uri: str)

Bases: object

Truncates Optuna studies and MLflow experiments to a target count.

truncate(study_name: str, target_count: int, dry_run: bool = True) → bool

gunz_ml.management.monitor module

HPO Monitoring and Auditing SDK.

Provides high-level progress tracking and health checks for large-scale Hyperparameter Optimization sweeps.

class gunz_ml.management.monitor.AuditService(optuna_db: str = 'mysql+pymysql://optuna:optuna@juno:3311/optuna')

Bases: object

Service for monitoring HPO progress and health.

bulk_audit(pattern: str) → DataFrame

Audits multiple studies matching a pattern.

get_study_progress(study_name: str) → Dict[str, Any]

Returns progress statistics for a single study.

gunz_ml.management.scaffold module

Project Scaffolding and Boilerplate Generation.

gunz_ml.management.scaffold.scaffold_project(project_name: str, interactive: bool = True, target_dir: str | Path = '.')

gunz_ml.management.tracking module

Juno Tracking SDK for MLflow and Optuna.

Provides a high-level interface for searching, ranking, and retrieving experiment and trial data from the tracking backends.

class gunz_ml.management.tracking.RunInfo(run_id: str, experiment_id: str, name: str, metrics: Dict[str, float], params: Dict[str, Any], tags: Dict[str, str])

Bases: object

experiment_id: str

metrics: Dict[str, float]

name: str

params: Dict[str, Any]

run_id: str

tags: Dict[str, str]

class gunz_ml.management.tracking.TrackingManager(mlflow_uri: str = 'http://juno.tnt.uni-hannover.de:5200', optuna_db: str = 'mysql+pymysql://optuna:optuna@juno:3311/optuna')

Bases: object

Unified interface for MLflow and Optuna discovery.

find_experiment(name_pattern: str) → List[mlflow.entities.Experiment]

Finds MLflow experiments matching a regex pattern.

get_best_run(experiment_name: str, metric_name: str = 'metric-spearmann_r', maximize: bool = True) → RunInfo | None

Returns the single best run for an experiment based on a metric.

get_best_run_id(experiment_name: str, metric_name: str = 'metric-spearmann_r', maximize: bool = True) → str | None

High-level query to get the ID of the best run.

list_studies(pattern: str | None = None) → List[optuna.study.StudySummary]

Lists Optuna studies matching an optional pattern.

search_by_metric(experiment_name: str, metric_name: str, threshold: float, mode: Literal['>', '<', '>=', '<='] = '>=', max_results: int = 100) → DataFrame

Searches runs based on a metric threshold.

search_runs(experiment_name: str, filter_string: str | None = None, max_results: int = 100) → DataFrame

Searches runs and returns a pandas DataFrame.