xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Edited "Vertex AI: Track artifacts and metrics across Vertex AI Pipelines runs using Vertex ML Metadata. No impact to code (#2749)

Browse Source
This commit is contained in:
kittyabs
2024-02-27 21:23:48 +00:00
committed by GitHub
parent 3fb37b5f35
commit d7a9ea8eed
1 changed files with 40 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/official/ml_metadata/vertex-pipelines-ml-metadata.ipynb
+40 -40
View File
@@ -61,7 +61,7 @@
"source": [
"## Overview\n",
"\n",
"This notebook demonstrates how to track metrics and artifacts across Vertex AI Pipelines runs, and analyze this metadata using the Vertex AI SDK. If you'd prefer to follow a step-by-step tutorial, check out the [codelab version](https://codelabs.developers.google.com/vertex-mlmd-pipelines#0) of this notebook.\n",
"This notebook demonstrates how to track metrics and artifacts across Vertex AI Pipelines runs, and analyze this metadata using the Vertex AI SDK for Pyrhon. If you'd prefer to follow a step-by-step tutorial, check out the [codelab version](https://codelabs.developers.google.com/vertex-mlmd-pipelines#0) of this notebook.\n",
"\n",
"Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
]
@@ -74,7 +74,7 @@
"source": [
"### Objective\n",
"\n",
"In this notebook, you learn how to track artifacts and metrics with `Vertex ML Metadata` in `Vertex AI Pipeline` runs.\n",
"In this notebook, you learn how to track artifacts and metrics with Vertex ML Metadata in Vertex AI Pipeline runs.\n",
"\n",
"This tutorial uses the following Google Cloud ML services and resources:\n",
"\n",
@@ -83,12 +83,12 @@
"\n",
"The steps performed include:\n",
"\n",
"* Use the Kubeflow Pipelines SDK to build an ML pipeline that runs on Vertex AI\n",
"* The pipeline will create a dataset, train a scikit-learn model, and deploy the model to an endpoint\n",
"* Write custom pipeline components that generate artifacts and metadata\n",
"* Compare Vertex Pipelines runs, both in the Cloud console and programmatically\n",
"* Trace the lineage for pipeline-generated artifacts\n",
"* Query your pipeline run metadata"
"* Use the Kubeflow Pipelines SDK to build an ML pipeline that runs on Vertex AI.\n",
"* The pipeline creates a dataset, trains a scikit-learn model, and deploys the model to an endpoint.\n",
"* Write custom pipeline components that generate artifacts and metadata.\n",
"* Compare Vertex AI Pipelines runs, both in the Google Cloud console and programmatically.\n",
"* Trace the lineage for pipeline-generated artifacts.\n",
"* Query your pipeline run metadata."
]
},
{
@@ -99,7 +99,7 @@
"source": [
"### Dataset\n",
"\n",
"In this notebook, we will train a model using scikit-learn to classify bean types using the [Dry Beans Dataset](https://archive.ics.uci.edu/ml/datasets/Dry+Bean+Dataset) from UCI Machine Learning. This is a tabular dataset that includes measurements and characteristics of seven different types of beans taken from images."
"In this notebook, you'll train a model using scikit-learn to classify bean types using the [Dry Beans Dataset](https://archive.ics.uci.edu/ml/datasets/Dry+Bean+Dataset) from UCI Machine Learning. This is a tabular dataset that includes measurements and characteristics of seven different types of beans taken from images."
]
},
{
@@ -132,7 +132,7 @@
"source": [
"### Install additional packages\n",
"\n",
"Run the following commands to install the Vertex AI SDK and packages used in this notebook."
"Run the following commands to install the Vertex AI SDK for Python and packages used in this notebook."
]
},
{
@@ -355,9 +355,9 @@
"source": [
"### Service Account\n",
"\n",
"You use a service account to create Vertex AI Pipeline jobs.\n",
"Use a service account to create Vertex AI Pipeline jobs.\n",
"\n",
"If you do not want to use your project's Compute Engine service account, set `SERVICE_ACCOUNT` to another service account ID."
"If you don't want to use your project's Compute Engine service account, set `SERVICE_ACCOUNT` to another service account ID."
]
},
{
@@ -513,7 +513,7 @@
"source": [
"## Concepts\n",
"\n",
"To better understand [Vertex Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata), we'd like to introduce the following concepts:\n"
"To better understand [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [Vertex AI ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata), here’re some relevant concepts:\n"
]
},
{
@@ -523,7 +523,7 @@
},
"source": [
"### Pipeline Run\n",
"When we use the term run, we're referring to a single execution of your pipeline in Vertex Pipelines. Each run generates artifacts, metrics, and associated metadata."
"The term “run” refers to a single execution of your pipeline in Vertex AI Pipelines. Each run generates artifacts, metrics, and associated metadata."
]
},
{
@@ -534,7 +534,7 @@
"source": [
"### Artifact\n",
"\n",
"An artifact is a resource generated by your pipeline. Artifacts could include datasets, models, endpoints, or custom resources defined in your pipeline."
"An artifact is a resource generated by your pipeline. Artifacts could datasets, models, endpoints, or custom resources defined in your pipeline."
]
},
{
@@ -567,11 +567,11 @@
"source": [
"## Creating a 3-step pipeline with custom components\n",
"\n",
"The focus of this lab is on understanding *metadata* from pipeline runs. In order to do that, we'll need a pipeline to run on Vertex Pipelines, which is where we'll start. Here we'll define a 3-step pipeline with the following custom components:\n",
"The focus of this lab is on understanding metadata from pipeline runs. In order to do that, you'll need a pipeline to run on Vertex AI Pipelines, which is where you’ll start. Here you’ll define a 3-step pipeline with the following custom components:\n",
"\n",
"* `get_dataframe`: Retrieve data from a BigQuery table and convert it into a pandas DataFrame\n",
"* `train_sklearn_model`: Use the pandas DataFrame to train and export a scikit-learn model, along with some metrics\n",
"* `deploy_model`: Deploy the exported scikit-learn model to an endpoint in Vertex AI"
"* `get_dataframe`: Retrieve data from a BigQuery table and convert it into a pandas DataFrame.\n",
"* `train_sklearn_model`: Use the pandas DataFrame to train and export a scikit-learn model, along with some metrics.\n",
"* `deploy_model`: Deploy the exported scikit-learn model to an endpoint in Vertex AI."
]
},
{
@@ -628,11 +628,11 @@
},
"source": [
"Next, create a component to train a scikit-learn model. This component does the following:\n",
"* Imports a CSV as a pandas DataFrame\n",
"* Splits the DataFrame into train and test sets\n",
"* Trains a scikit-learn model\n",
"* Logs metrics from the model\n",
"* Saves the model artifacts as a local `model.joblib` file"
"* Imports a CSV as a pandas DataFrame.\n",
"* Splits the DataFrame into train and test sets.\n",
"* Trains a scikit-learn model.\n",
"* Logs metrics from the model.\n",
"* Saves the model artifacts as a local `model.joblib` file."
]
},
{
@@ -678,7 +678,7 @@
"id": "gaNNTFPaU7KT"
},
"source": [
"Finally, our last component will take the trained model from the previous step, upload it to Vertex AI, and deploy it to an endpoint:"
"Finally, the last component takes the trained model from the previous step, uploads the model to Vertex AI, and deploys it to an endpoint:"
]
},
{
@@ -759,7 +759,7 @@
"id": "910541af051c"
},
"source": [
"The following will generate a JSON file that you'll use to run the pipeline:"
"The following generates a JSON file that you'll use to run the pipeline:"
]
},
{
@@ -781,7 +781,7 @@
"source": [
"### Start two pipeline runs\n",
"\n",
"Next we'll kick off **two** runs of our pipeline. First let's define a timestamp to use for our pipeline job IDs:"
"Next you'll kick ofg **two** runs of our pipeline. First, define a timestamp to use for our pipeline job IDs:"
]
},
{
@@ -803,7 +803,7 @@
"id": "3d380ed72490"
},
"source": [
"Our pipeline takes one parameter when we run it: the `bq_table` we want to use for training data. This pipeline run will use a smaller version of the beans dataset:"
"The pipeline takes one parameter when you run it: the `bq_table` we want to use for training data you’ll use for training data. This pipeline run uses a smaller version of the beans dataset:"
]
},
{
@@ -895,7 +895,7 @@
"id": "cc15017be48e"
},
"source": [
"After running this cell, you'll see a link to view each pipeline in the Vertex AI console. Open that link to see more details on your pipeline.\n",
"After running this cell, you'll see a link to view each pipeline in the Google Cloud console. Open that link to see more details on your pipeline.\n",
"\n",
"**These pipeline runs will take 10-15 minutes to complete.**"
]
@@ -915,9 +915,9 @@
"id": "A1PqKxlpOZa2"
},
"source": [
"Now that you have two pipeline completed pipeline runs, we're ready to take a closer look at pipeline metrics using the Vertex AI SDK.\n",
"Now that you have two pipeline completed pipeline runs, you're ready to take a closer look at pipeline metrics using the Vertex AI SDK for Python.\n",
"\n",
"**For guidance on inspecting pipeline artifacts and metadata in the Vertex AI Console, see [this codelab](https://codelabs.developers.google.com/vertex-mlmd-pipelines#5).**"
"**For guidance on inspecting pipeline artifacts and metadata in the Google Cloud console, see [this codelab](https://codelabs.developers.google.com/vertex-mlmd-pipelines#5).**"
]
},
{
@@ -926,7 +926,7 @@
"id": "jbRf1WoH_vbY"
},
"source": [
"You can use the `aiplatform.get_pipeline_df()` method to access run metadata. Here, we'll get metadata for the last two runs of the same pipeline and load it into a Pandas DataFrame. The `mlmd-pipeline` parameter here refers to the name we gave our pipeline in our pipeline definition:"
"You can use the `aiplatform.get_pipeline_df()` method to access run metadata. Here, you'll get metadata for the last two runs of the same pipeline and load it into a Pandas DataFrame. The `mlmd-pipeline` parameter here refers to the name you gave your pipeline in the pipeline definition:"
]
},
{
@@ -947,7 +947,7 @@
"id": "d23e2cb66265"
},
"source": [
"We've only executed our pipeline twice here, but you can imagine how many metrics you'd have with more executions. Next, we'll create a custom visualization with matplotlib to see the relationship between our model's accuracy and the amount of data used for training. Run the following to generate a graph:"
"You’ve only executed the pipeline twice here, but you can imagine how many metrics you'd have with more executions. Next, create a custom visualization with matplotlib to see the relationship between the model's accuracy and the amount of data used for training. Run the following to generate a graph:"
]
},
{
@@ -979,7 +979,7 @@
"id": "4431b5d062f3"
},
"source": [
"In addition to getting a DataFrame of all pipeline metrics, you may want to programmatically query artifacts created in your ML system. From there you could create a custom dashboard or let others in your organizaiton get details on specific artifacts."
"In addition to getting a DataFrame of all pipeline metrics, you may want to programmatically query artifacts created in your ML system. From there you can create a custom dashboard or let others in your organizaiton get details on specific artifacts."
]
},
{
@@ -990,7 +990,7 @@
"source": [
"### Getting all Model artifacts\n",
"\n",
"To query artifacts in this way, we'll create a `MetadataServiceClient`:"
"To query artifacts in this way, you'll create a `MetadataServiceClient`:"
]
},
{
@@ -1013,7 +1013,7 @@
"id": "e5aee9cdc5bd"
},
"source": [
"Next, we'll make a `list_artifacts` request to that endpoint and pass a filter indicating which artifacts we'd like in our response. First, let's get all the artifacts in our project that are **models**. To do that, run the following in your notebook:"
"Next, make a `list_artifacts` request to that endpoint and pass a filter indicating which artifacts you'd like in your response. First, let's get all the artifacts in the project that are **models**. To do that, run the following in your notebook:"
]
},
{
@@ -1056,7 +1056,7 @@
"id": "F19_5lw0MqXv"
},
"source": [
"It would be handy if we could more easily visualize the resulting artifact query. Next, let's get all artifacts created after August 10, 2021 with a `LIVE` state. After we run this request, we'll display the results in a pandas DataFrame. First, execute the request:"
"Next, get all artifacts created after August 10, 2021 with a `LIVE` state. After you run this request, display the results in a pandas DataFrame. First, execute the request:"
]
},
{
@@ -1118,11 +1118,11 @@
"\n",
"If you don't want to delete the project, do the following to clean up the resources you used:\n",
"\n",
"* If you used Google Cloud Notebooks to run this, stop or delete the notebook instance\n",
"* If you used Vertex AI Workbench notebooks to run this, stop or delete the notebook instance.\n",
"\n",
"* The pipeline runs we executed deployed endpoints in Vertex AI. Navigate to the [Vertex AI console](https://console.cloud.google.com/vertex-ai/endpoints) to delete those endpoints\n",
"* The pipeline runs you executed deployed endpoints in Vertex AI. Navigate to the [Google Cloud console](https://console.cloud.google.com/vertex-ai/endpoints) to delete those endpoints.\n",
"\n",
"* Delete the [Cloud Storage bucket](https://console.cloud.google.com/storage/browser/) you created"
"* Delete the [Cloud Storage bucket](https://console.cloud.google.com/storage/browser/) you created."
]
},
{