Package org.mlflow.tracking

Class MlflowClient

java.lang.Object
org.mlflow.tracking.MlflowClient
All Implemented Interfaces:
java.io.Serializable
public class MlflowClient
extends java.lang.Object
implements java.io.Serializable
Client to an MLflow Tracking Sever.
See Also:
Serialized Form

  • Field Summary

    Fields 
    Modifier and Type Field Description
    protected static java.lang.String DEFAULT_EXPERIMENT_ID  

  • Constructor Summary

    Constructors 
    Constructor Description
    MlflowClient()
    Return a default client based on the MLFLOW_TRACKING_URI environment variable.
    MlflowClient​(java.lang.String trackingUri)
    Instantiate a new client using the provided tracking uri.
    MlflowClient​(MlflowHostCredsProvider hostCredsProvider)
    Create a new MlflowClient; users should prefer constructing ApiClients via MlflowClient() or MlflowClient(String) if possible.

  • Method Summary

    Modifier and Type Method Description
    java.lang.String createExperiment​(java.lang.String experimentName)
    Create a new experiment using the default artifact location provided by the server.
    Service.RunInfo createRun()
    Create a new run under the default experiment with no application name.
    Service.RunInfo createRun​(java.lang.String experimentId)
    Create a new run under the given experiment.
    Service.RunInfo createRun​(Service.CreateRun request)
    Create a new run.
    void deleteExperiment​(java.lang.String experimentId)
    Mark an experiment and associated runs, params, metrics, etc.
    void deleteRun​(java.lang.String runId)
    Delete a run with the given ID.
    void deleteTag​(java.lang.String runId, java.lang.String key)
    Delete a tag on the run ID with a specific key.
    java.io.File downloadArtifacts​(java.lang.String runId)
    Return a local directory containing *all* artifacts within the run's artifact directory.
    java.io.File downloadArtifacts​(java.lang.String runId, java.lang.String artifactPath)
    Return a local file or directory containing all artifacts within the given artifactPath within the run's root artifactDirectory.
    java.io.File downloadLatestModelVersion​(java.lang.String modelName, java.lang.String stage)
    :: experimental :: this api may change or be removed in a future release without warning.
    java.io.File downloadModelVersion​(java.lang.String modelName, java.lang.String version)
    :: Experimental :: This API may change or be removed in a future release without warning.
    Service.GetExperiment.Response getExperiment​(java.lang.String experimentId)  
    java.util.Optional<Service.Experiment> getExperimentByName​(java.lang.String experimentName)  
    java.util.List<ModelRegistry.ModelVersion> getLatestVersions​(java.lang.String modelName)
    :: Experimental :: This API may change or be removed in a future release without warning.
    java.util.List<ModelRegistry.ModelVersion> getLatestVersions​(java.lang.String modelName, java.lang.Iterable<java.lang.String> stages)
    :: Experimental :: This API may change or be removed in a future release without warning.
    java.util.List<Service.Metric> getMetricHistory​(java.lang.String runId, java.lang.String key)  
    java.lang.String getModelVersionDownloadUri​(java.lang.String modelName, java.lang.String version)
    :: Experimental :: This API may change or be removed in a future release without warning.
    Service.Run getRun​(java.lang.String runId)
    Get metadata, params, tags, and metrics for a run.
    java.util.List<Service.FileInfo> listArtifacts​(java.lang.String runId)
    List the artifacts immediately under the run's root artifact directory.
    java.util.List<Service.FileInfo> listArtifacts​(java.lang.String runId, java.lang.String artifactPath)
    List the artifacts immediately under the given artifactPath within the run's root artifact directory.
    java.util.List<Service.Experiment> listExperiments()  
    java.util.List<Service.RunInfo> listRunInfos​(java.lang.String experimentId)  
    void logArtifact​(java.lang.String runId, java.io.File localFile)
    Upload the given local file or directory to the run's root artifact directory.
    void logArtifact​(java.lang.String runId, java.io.File localFile, java.lang.String artifactPath)
    Upload the given local file or directory to an artifactPath within the run's root directory.
    void logArtifacts​(java.lang.String runId, java.io.File localDir)
    Upload all files within the given local directory the run's root artifact directory.
    void logArtifacts​(java.lang.String runId, java.io.File localDir, java.lang.String artifactPath)
    Upload all files within the given local director an artifactPath within the run's root artifact directory.
    void logBatch​(java.lang.String runId, java.lang.Iterable<Service.Metric> metrics, java.lang.Iterable<Service.Param> params, java.lang.Iterable<Service.RunTag> tags)
    Log multiple metrics, params, and/or tags against a given run (argument runId).
    void logMetric​(java.lang.String runId, java.lang.String key, double value)
    Log a new metric against the given run, as a key-value pair.
    void logMetric​(java.lang.String runId, java.lang.String key, double value, long timestamp, long step)
    Log a new metric against the given run, as a key-value pair.
    void logParam​(java.lang.String runId, java.lang.String key, java.lang.String value)
    Log a parameter against the given run, as a key-value pair.
    void renameExperiment​(java.lang.String experimentId, java.lang.String newName)
    Update an experiment's name.
    void restoreExperiment​(java.lang.String experimentId)
    Restore an experiment marked for deletion.
    void restoreRun​(java.lang.String runId)
    Restore a deleted run with the given ID.
    java.util.List<Service.RunInfo> searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter)
    Deprecated.
    As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
    java.util.List<Service.RunInfo> searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType)
    Deprecated.
    As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
    RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults)
    Return runs from provided list of experiments that satisfy the search query.
    RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults, java.util.List<java.lang.String> orderBy)
    Return runs from provided list of experiments that satisfy the search query.
    RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults, java.util.List<java.lang.String> orderBy, java.lang.String pageToken)
    Return runs from provided list of experiments that satisfy the search query.
    java.lang.String sendGet​(java.lang.String path)
    :: Experimental :: This API may change or be removed in a future release without warning.
    java.lang.String sendPatch​(java.lang.String path, java.lang.String json)  
    java.lang.String sendPost​(java.lang.String path, java.lang.String json)
    :: Experimental :: This API may change or be removed in a future release without warning.
    void setExperimentTag​(java.lang.String experimentId, java.lang.String key, java.lang.String value)
    Log a new tag against the given experiment as a key-value pair.
    void setTag​(java.lang.String runId, java.lang.String key, java.lang.String value)
    Log a new tag against the given run, as a key-value pair.
    void setTerminated​(java.lang.String runId)
    Set the status of a run to be FINISHED at the current time.
    void setTerminated​(java.lang.String runId, Service.RunStatus status)
    Set the status of a run to be completed at the current time.
    void setTerminated​(java.lang.String runId, Service.RunStatus status, long endTime)
    Set the status of a run to be completed at the given endTime.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

  • Method Details

    • getRun

      public Service.Run getRun​(java.lang.String runId)
      Get metadata, params, tags, and metrics for a run. A single value is returned for each metric key: the most recently logged metric value at the largest step.
      Returns:
      Run associated with the ID.

    • getMetricHistory

      public java.util.List<Service.Metric> getMetricHistory​(java.lang.String runId, java.lang.String key)

    • createRun

      public Service.RunInfo createRun()
      Create a new run under the default experiment with no application name.
      Returns:
      RunInfo created by the server.

    • createRun

      public Service.RunInfo createRun​(java.lang.String experimentId)
      Create a new run under the given experiment.
      Returns:
      RunInfo created by the server.

    • createRun

      public Service.RunInfo createRun​(Service.CreateRun request)
      Create a new run. This method allows providing all possible fields of CreateRun, and can be invoked as follows: 
         import org.mlflow.api.proto.Service.CreateRun;
   CreateRun.Builder request = CreateRun.newBuilder();
   request.setExperimentId(experimentId);
   request.setSourceVersion("my-version");
   createRun(request.build());
      Returns:
      RunInfo created by the server.

    • listRunInfos

      public java.util.List<Service.RunInfo> listRunInfos​(java.lang.String experimentId)
      Returns:
      A list of all RunInfos associated with the given experiment.

    • searchRuns

      public java.util.List<Service.RunInfo> searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter)
      Deprecated.
      As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
      Return RunInfos from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc = 0.9" If null, the result will be equivalent to having an empty search filter.
      Returns:
      A list of all RunInfos that satisfy search filter.

    • searchRuns

      public java.util.List<Service.RunInfo> searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType)
      Deprecated.
      As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
      Return RunInfos from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      Returns:
      A list of all RunInfos that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      Returns:
      A list of all Runs that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults, java.util.List<java.lang.String> orderBy)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      Returns:
      A list of all Runs that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns​(java.util.List<java.lang.String> experimentIds, java.lang.String searchFilter, Service.ViewType runViewType, int maxResults, java.util.List<java.lang.String> orderBy, java.lang.String pageToken)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      pageToken - String token specifying the next page of results. It should be obtained from a call to searchRuns(List, String).
      Returns:
      A page of Runs that satisfy the search filter.

    • listExperiments

      public java.util.List<Service.Experiment> listExperiments()
      Returns:
      A list of all experiments.

    • getExperiment

      public Service.GetExperiment.Response getExperiment​(java.lang.String experimentId)
      Returns:
      An experiment with the given ID.

    • getExperimentByName

      public java.util.Optional<Service.Experiment> getExperimentByName​(java.lang.String experimentName)
      Returns:
      The experiment associated with the given name or Optional.empty if none exists.

    • createExperiment

      public java.lang.String createExperiment​(java.lang.String experimentName)
      Create a new experiment using the default artifact location provided by the server.
      Parameters:
      experimentName - Name of the experiment. This must be unique across all experiments.
      Returns:
      Experiment ID of the newly created experiment.

    • deleteExperiment

      public void deleteExperiment​(java.lang.String experimentId)
      Mark an experiment and associated runs, params, metrics, etc. for deletion.

    • restoreExperiment

      public void restoreExperiment​(java.lang.String experimentId)
      Restore an experiment marked for deletion.

    • renameExperiment

      public void renameExperiment​(java.lang.String experimentId, java.lang.String newName)
      Update an experiment's name. The new name must be unique.

    • deleteRun

      public void deleteRun​(java.lang.String runId)
      Delete a run with the given ID.

    • restoreRun

      public void restoreRun​(java.lang.String runId)
      Restore a deleted run with the given ID.

    • logParam

      public void logParam​(java.lang.String runId, java.lang.String key, java.lang.String value)
      Log a parameter against the given run, as a key-value pair. This cannot be called against the same parameter key more than once.

    • logMetric

      public void logMetric​(java.lang.String runId, java.lang.String key, double value)
      Log a new metric against the given run, as a key-value pair. Metrics are recorded against two axes: timestamp and step. This method uses the number of milliseconds since the Unix epoch for the timestamp, and it uses the default step of zero.
      Parameters:
      runId - The ID of the run in which to record the metric.
      key - The key identifying the metric for which to record the specified value.
      value - The value of the metric.

    • logMetric

      public void logMetric​(java.lang.String runId, java.lang.String key, double value, long timestamp, long step)
      Log a new metric against the given run, as a key-value pair. Metrics are recorded against two axes: timestamp and step.
      Parameters:
      runId - The ID of the run in which to record the metric.
      key - The key identifying the metric for which to record the specified value.
      value - The value of the metric.
      timestamp - The timestamp at which to record the metric value.
      step - The step at which to record the metric value.

    • setExperimentTag

      public void setExperimentTag​(java.lang.String experimentId, java.lang.String key, java.lang.String value)
      Log a new tag against the given experiment as a key-value pair.
      Parameters:
      experimentId - The ID of the experiment on which to set the tag
      key - The key used to identify the tag.
      value - The value of the tag.

    • setTag

      public void setTag​(java.lang.String runId, java.lang.String key, java.lang.String value)
      Log a new tag against the given run, as a key-value pair.
      Parameters:
      runId - The ID of the run on which to set the tag
      key - The key used to identify the tag.
      value - The value of the tag.

    • deleteTag

      public void deleteTag​(java.lang.String runId, java.lang.String key)
      Delete a tag on the run ID with a specific key. This is irreversible.
      Parameters:
      runId - String ID of the run
      key - Name of the tag

    • logBatch

      public void logBatch​(java.lang.String runId, java.lang.Iterable<Service.Metric> metrics, java.lang.Iterable<Service.Param> params, java.lang.Iterable<Service.RunTag> tags)
      Log multiple metrics, params, and/or tags against a given run (argument runId). Argument metrics, params, and tag iterables can be nulls.

    • setTerminated

      public void setTerminated​(java.lang.String runId)
      Set the status of a run to be FINISHED at the current time.

    • setTerminated

      public void setTerminated​(java.lang.String runId, Service.RunStatus status)
      Set the status of a run to be completed at the current time.

    • setTerminated

      public void setTerminated​(java.lang.String runId, Service.RunStatus status, long endTime)
      Set the status of a run to be completed at the given endTime.

    • sendGet

      public java.lang.String sendGet​(java.lang.String path)
      :: Experimental :: This API may change or be removed in a future release without warning. Send a GET to the following path, including query parameters. This is mostly an internal API, but allows making lower-level or unsupported requests.
      Returns:
      JSON response from the server.

    • sendPost

      public java.lang.String sendPost​(java.lang.String path, java.lang.String json)
      :: Experimental :: This API may change or be removed in a future release without warning. Send a POST to the following path, with a String-encoded JSON body. This is mostly an internal API, but allows making lower-level or unsupported requests.
      Returns:
      JSON response from the server.

    • sendPatch

      public java.lang.String sendPatch​(java.lang.String path, java.lang.String json)

    • logArtifact

      public void logArtifact​(java.lang.String runId, java.io.File localFile)
      Upload the given local file or directory to the run's root artifact directory. For example, 
         logArtifact(runId, "/my/localModel")
   listArtifacts(runId) // returns "localModel"
      Parameters:
      runId - Run ID of an existing MLflow run.
      localFile - File or directory to upload. Must exist.

    • logArtifact

      public void logArtifact​(java.lang.String runId, java.io.File localFile, java.lang.String artifactPath)
      Upload the given local file or directory to an artifactPath within the run's root directory. For example, 
         logArtifact(runId, "/my/localModel", "model")
   listArtifacts(runId, "model") // returns "model/localModel"
      (i.e., the localModel file is now available in model/localModel). If logging a directory, the directory is renamed to artifactPath.
      Parameters:
      runId - Run ID of an existing MLflow run.
      localFile - File or directory to upload. Must exist.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • logArtifacts

      public void logArtifacts​(java.lang.String runId, java.io.File localDir)
      Upload all files within the given local directory the run's root artifact directory. For example, if /my/local/dir/ contains two files "file1" and "file2", then 
         logArtifacts(runId, "/my/local/dir")
   listArtifacts(runId) // returns "file1" and "file2"
      Parameters:
      runId - Run ID of an existing MLflow run.
      localDir - Directory to upload. Must exist, and must be a directory (not a simple file).

    • logArtifacts

      public void logArtifacts​(java.lang.String runId, java.io.File localDir, java.lang.String artifactPath)
      Upload all files within the given local director an artifactPath within the run's root artifact directory. For example, if /my/local/dir/ contains two files "file1" and "file2", then 
         logArtifacts(runId, "/my/local/dir", "model")
   listArtifacts(runId, "model") // returns "model/file1" and "model/file2"
      (i.e., the contents of the local directory are now available in model/).
      Parameters:
      runId - Run ID of an existing MLflow run.
      localDir - Directory to upload. Must exist, and must be a directory (not a simple file).
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • listArtifacts

      public java.util.List<Service.FileInfo> listArtifacts​(java.lang.String runId)
      List the artifacts immediately under the run's root artifact directory. This does not recursively list; instead, it will return FileInfos with isDir=true where further listing may be done.
      Parameters:
      runId - Run ID of an existing MLflow run.

    • listArtifacts

      public java.util.List<Service.FileInfo> listArtifacts​(java.lang.String runId, java.lang.String artifactPath)
      List the artifacts immediately under the given artifactPath within the run's root artifact directory. This does not recursively list; instead, it will return FileInfos with isDir=true where further listing may be done.
      Parameters:
      runId - Run ID of an existing MLflow run.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • downloadArtifacts

      public java.io.File downloadArtifacts​(java.lang.String runId)
      Return a local directory containing *all* artifacts within the run's artifact directory. Note that this will download the entire directory path, and so may be expensive if the directory has a lot of data.
      Parameters:
      runId - Run ID of an existing MLflow run.

    • downloadArtifacts

      public java.io.File downloadArtifacts​(java.lang.String runId, java.lang.String artifactPath)
      Return a local file or directory containing all artifacts within the given artifactPath within the run's root artifactDirectory. For example, if "model/file1" and "model/file2" exist within the artifact directory, then 
         downloadArtifacts(runId, "model") // returns a local directory containing "file1" and "file2"
   downloadArtifacts(runId, "model/file1") // returns a local *file* with the contents of file1.
      Note that this will download the entire subdirectory path, and so may be expensive if the subdirectory has a lot of data.
      Parameters:
      runId - Run ID of an existing MLflow run.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • getLatestVersions

      public java.util.List<ModelRegistry.ModelVersion> getLatestVersions​(java.lang.String modelName)
      :: Experimental :: This API may change or be removed in a future release without warning. Return the latest model version for each stage. The current available stages are: [None, Staging, Production, Archived]. 
              import org.mlflow.api.proto.ModelRegistry.ModelVersion;
        List<ModelVersion> detailsList = getLatestVersions("model");

        for (ModelVersion details : detailsList) {
            System.out.println("Model Name: " + details.getModelVersion()
                                                       .getRegisteredModel()
                                                       .getName());
            System.out.println("Model Version: " + details.getModelVersion().getVersion());
            System.out.println("Current Stage: " + details.getCurrentStage());
        }
      Parameters:
      modelName - The name of the model
      Returns:
      A collection of ModelRegistry.ModelVersion

    • getLatestVersions

      public java.util.List<ModelRegistry.ModelVersion> getLatestVersions​(java.lang.String modelName, java.lang.Iterable<java.lang.String> stages)
      :: Experimental :: This API may change or be removed in a future release without warning. Return the latest model version for each stage requested. The current available stages are: [None, Staging, Production, Archived]. 
              import org.mlflow.api.proto.ModelRegistry.ModelVersion;
        List<ModelVersion> detailsList =
          getLatestVersions("model", Lists.newArrayList<String>("Staging"));

        for (ModelVersion details : detailsList) {
            System.out.println("Model Name: " + details.getModelVersion()
                                                       .getRegisteredModel()
                                                       .getName());
            System.out.println("Model Version: " + details.getModelVersion().getVersion());
            System.out.println("Current Stage: " + details.getCurrentStage());
        }
      Parameters:
      modelName - The name of the model
      stages - A list of stages
      Returns:
      The latest model version ModelRegistry.ModelVersion

    • getModelVersionDownloadUri

      public java.lang.String getModelVersionDownloadUri​(java.lang.String modelName, java.lang.String version)
      :: Experimental :: This API may change or be removed in a future release without warning. Return the model URI containing for the given model version. The model URI can be used to download the model version artifacts. 
              String modelUri = getModelVersionDownloadUri("model", 0);
      Parameters:
      modelName - The name of the model
      version - The version number of the model
      Returns:
      The specified model version's URI.

    • downloadModelVersion

      public java.io.File downloadModelVersion​(java.lang.String modelName, java.lang.String version)
      :: Experimental :: This API may change or be removed in a future release without warning. Return a local file or directory containing all artifacts within the given registered model version. The method will download the model version artifacts to the local file system. 
              File modelVersionFile = downloadModelVersion("model", 0);
      Parameters:
      modelName - The name of the model
      version - The version number of the model
      Returns:
      A local file or directory (File) containing model artifacts

    • downloadLatestModelVersion

      public java.io.File downloadLatestModelVersion​(java.lang.String modelName, java.lang.String stage)
      :: experimental :: this api may change or be removed in a future release without warning. Return a local file or directory containing all artifacts within the latest registered model version in the given stage. The method will download the model version artifacts to the local file system. 
              File modelVersionFile = downloadLatestModelVersion("model", "Staging");
      (i.e., the contents of the local directory are now available).
      Parameters:
      modelName - The name of the model
      stage - The name of the stage
      Returns:
      A local file or directory (File) containing model artifacts