SagemakerExperimentsLogger Class

class experiments_addon.logger.SagemakerExperimentsLogger(run_name: str | None = None, experiment_name: str | None = None, sagemaker_session: Session | None = None)

Bases: Logger

Log to AWS Sagemaker Experiments .

Implemented using experiments API. Install api with pip:

pip install sagemaker

It can be used in several ways:

  1. Use SagemakerExperimentsLogger by explicitly passing in run_name and experiment_name.

If run_name and experiment_name are passed in, they are honored over the default experiment_name config of the loffer. A new experiment_name with run job will be created.

Note

Both run_name and experiment_name should be supplied to make this usage work. Otherwise, you may get a ValueError.

from pytorch_lightning import Trainer
from experiments_addon.logger import SagemakerExperimentsLogger

logger = SagemakerExperimentsLogger(experiment_name="test_experiment", run_name="test_run")
trainer = Trainer(logger=logger)

  1. Use the SagemakerExperimentsLogger in a job script without supplying run_name and experiment_name.

In this case, the default experiment_name config (specified when creating the job) is fetched from the job environment to load the run. For example inside a training job.

from pytorch_lightning import Trainer
from sagemaker.experiments.run import Run:
from experiments_addon.logger import SagemakerExperimentsLogger

logger = SagemakerExperimentsLogger()
trainer = Trainer(logger=logger)

3. Use the SagemakerExperimentsLogger in a notebook within a run context (i.e. the with block) but without supplying run_name and experiment_name.

Every time we call with Run(...) as _:, the initialized run is tracked in the run context and an experiment_name and job is created. Then when using SagemakerExperimentsLogger under this in the context is loaded by default.

from pytorch_lightning import Trainer
from sagemaker.experiments.run import Run
from experiments_addon.logger import SagemakerExperimentsLogger

with Run(experiment_name="test_experiment", run_name="test_run") as _:
    logger = SagemakerExperimentsLogger()
    trainer = Trainer(logger=logger)
Parameters:

  • run_name (str) – The name of the run to be created (default: None). If it is None, run name of the job will be fetched to load the run.

  • experiment_name (str) – The name of the experiment_name to be created (default: None). Note: the experiment_name must be supplied along with a valid run_name. Otherwise, it will be ignored. If it is None, the experiment_name name will be fetched to load the experiment_name.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

__init__(run_name: str | None = None, experiment_name: str | None = None, sagemaker_session: Session | None = None) → None
log_hyperparams(params: Dict[str, Any] | Namespace) → None

Log hyperparameters.

Function map to sagemaker.experiments.run.Run.log_parameters() of the SageMaker Experiments API . Implements the abstract function of the pytorch_lightning.loggers.logger.Logger base class and will be called automatically from pytorch_lightning.Trainer. To being compatible to log_parameters(), the hyperparameters will be converted into dictionary and boolean will be converted to “True” and “False”.

Parameters:

params (dict or namespace) – a dictionary-like container with the hyperparameters

log_metrics(metrics: Dict[str, Tensor | float], step: int | None = None) → None

Log evaluation metrics.

Function map to sagemaker.experiments.run.Run.log_metric() of the SageMaker Experiments API . Implementes the abstract function of the pytorch_lightning.loggers.logger.Logger base class and will be called automatically from pytorch_lightning.Trainer.

Parameters:

  • metrics (dict of str and Tensor or float) – a dictionary containing metric values.

  • step (int) – Determines the iteration step of the metric (default: None)

log_precision_recall(y_true: Iterable, predicted_probabilities: Iterable, positive_label: str | int | float | None = None, title: str | None = None, is_output: bool = True, no_skill: int | None = None) → None

Create and log a precision recall graph artifact for Sagemaker Studio UI to render.

Function map to sagemaker.experiments.run.Run.log_precision_recall() of the SageMaker Experiments API

Parameters:

  • y_true (list or array) – True labels. If labels are not binary then positive_label should be given.

  • predicted_probabilities (list or array) – Estimated/predicted probabilities.

  • positive_label (str or int) – Label of the positive class (default: None).

  • title (str) – Title of the graph (default: None).

  • is_output (bool) – Determines direction of association to the run. Defaults to True (output artifact). If set to False then represented as input association.

  • no_skill (int) – The precision threshold under which the classifier cannot discriminate between the classes and would predict a random class or a constant class in all cases (default: None).

Example:

self.logger.log_precision_recall(...)
log_roc_curve(y_true: Iterable, y_score: Iterable, title: str | None = None, is_output: bool = True) → None

Create and log a receiver operating characteristic (ROC curve) artifact.

Function map to sagemaker.experiments.run.Run.log_roc_curve() of the SageMaker Experiments API

Parameters:

  • y_true (list or array) – True labels. If labels are not binary then positive_label should be given.

  • y_score (list or array) – Estimated/predicted probabilities.

  • title (str) – Title of the graph (default: None).

  • is_output (bool) – Determines direction of association to the run. Defaults to True (output artifact). If set to False then represented as input association.

Example:

self.logger.log_roc_curve(...)
log_confusion_matrix(y_true: Iterable, y_pred: Iterable, title: str | None = None, is_output: bool = True) → None

Create and log a confusion matrix artifact.

Function map to sagemaker.experiments.run.Run.log_confusion_matrix() of the SageMaker Experiments API

Parameters:

  • y_true (list or array) – True labels. If labels are not binary then positive_label should be given.

  • y_pred (list or array) – Predicted labels.

  • title (str) – Title of the graph (default: None).

  • is_output (bool) – Determines direction of association to the run. Defaults to True (output artifact). If set to False then represented as input association.

Example:

self.logger.log_confusion_matrix(...)
log_artifact(name: str, value: str, media_type: str | None = None, is_output: bool = True) → None

Record a single artifact.

Function map to sagemaker.experiments.run.Run.log_artifact() of the SageMaker Experiments API

Parameters:

  • name (str) – The name of the artifact.

  • value (str) – The value.

  • media_type (str) – The MediaType (MIME type) of the value (default: None).

  • is_output (bool) – Determines direction of association to the run. Defaults to True (output artifact). If set to False then represented as input association.

Example:

self.logger.log_artifact(...)
log_file(file_path: str, name: str | None = None, media_type: str | None = None, is_output: bool = True) → None

Upload a file to s3 and store it as an input/output artifact.

Function map to sagemaker.experiments.run.Run.log_file() of the SageMaker Experiments API

Parameters:

  • file_path (str) – The path of the local file to upload.

  • name (str) – The name of the artifact (default: None).

  • media_type (str) – The MediaType (MIME type) of the file. If not specified, this library will attempt to infer the media type from the file extension of file_path.

  • is_output (bool) – Determines direction of association to the run. Defaults to True (output artifact). If set to False then represented as input association.

property name: str

Get the name of the experiment.

Returns:

experiment_name of the current run.

property version: int | str

Get the version which is similar to the run name.

Returns:

run_name of the current run.