mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
metrics_viz_run_compare_kfp (#847)
* added new cell for is_colab condition * added new cell for is_colab condition * changes andrew comments * changes andrew comments * review changes * review changes Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
This commit is contained in:
udaypunna
co-authored by
Andrew Ferlitsch
Andrew Ferlitsch
parent
8ab8ef9ca9
commit
976ed94cf2
@@ -64,30 +64,6 @@
|
||||
"This notebook shows how to use [the Kubeflow Pipelines (KFP) SDK](https://www.kubeflow.org/docs/components/pipelines/) to build [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines) that generate model metrics and metrics visualizations, and comparing pipeline runs."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dataset:wine,lcn,sklearn"
|
||||
},
|
||||
"source": [
|
||||
"### Datasets\n",
|
||||
"\n",
|
||||
"The dataset used for this tutorial is the [Wine dataset](https://archive.ics.uci.edu/ml/datasets/wine) from [Scikit-learn builtin datasets](https://scikit-learn.org/stable/datasets.html).\n",
|
||||
"\n",
|
||||
"The dataset predicts the origin of a wine."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dataset:iris,lcn,sklearn"
|
||||
},
|
||||
"source": [
|
||||
"The dataset used for this tutorial is the [Iris dataset](https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html) from [Scikit-learn builtin datasets](https://scikit-learn.org/stable/datasets.html).\n",
|
||||
"\n",
|
||||
"The dataset predicts the type of Iris flower species from a class of three species: setosa, virginica, or versicolor."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -112,6 +88,30 @@
|
||||
"- Compare metrics across pipeline runs"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dataset:wine,lcn,sklearn"
|
||||
},
|
||||
"source": [
|
||||
"### Datasets\n",
|
||||
"\n",
|
||||
"The dataset used for this tutorial is the [Wine dataset](https://archive.ics.uci.edu/ml/datasets/wine) from [Scikit-learn builtin datasets](https://scikit-learn.org/stable/datasets.html).\n",
|
||||
"\n",
|
||||
"The dataset predicts the origin of a wine."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dataset:iris,lcn,sklearn"
|
||||
},
|
||||
"source": [
|
||||
"The dataset used for this tutorial is the [Iris dataset](https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html) from [Scikit-learn builtin datasets](https://scikit-learn.org/stable/datasets.html).\n",
|
||||
"\n",
|
||||
"The dataset predicts the type of Iris flower species from a class of three species: setosa, virginica, or versicolor."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -202,7 +202,7 @@
|
||||
"! pip3 install {USER_FLAG} kfp google-cloud-pipeline-components --upgrade -q\n",
|
||||
"\n",
|
||||
"if os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! pip3 install --upgrade matplotlib $USER_FLAG"
|
||||
" ! pip3 install --upgrade matplotlib $USER_FLAG -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -240,6 +240,8 @@
|
||||
"id": "check_versions"
|
||||
},
|
||||
"source": [
|
||||
"### KFP SDK version\n",
|
||||
"\n",
|
||||
"Check the versions of the packages you installed. The KFP SDK version should be >=1.6."
|
||||
]
|
||||
},
|
||||
@@ -349,7 +351,10 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
"REGION = \"[your-region]\" # @param {type: \"string\"}\n",
|
||||
"\n",
|
||||
"if REGION == \"[your-region]\":\n",
|
||||
" REGION = \"us-central1\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -358,9 +363,9 @@
|
||||
"id": "timestamp"
|
||||
},
|
||||
"source": [
|
||||
"#### Timestamp\n",
|
||||
"#### UUID\n",
|
||||
"\n",
|
||||
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial."
|
||||
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -371,9 +376,16 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from datetime import datetime\n",
|
||||
"import random\n",
|
||||
"import string\n",
|
||||
"\n",
|
||||
"TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
|
||||
"\n",
|
||||
"# Generate a uuid of a specifed length(default=8)\n",
|
||||
"def generate_uuid(length: int = 8) -> str:\n",
|
||||
" return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"UUID = generate_uuid()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -384,7 +396,7 @@
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"**If you are using Vertex AI Workbench Notebook**, your environment is already authenticated. Skip this step.\n",
|
||||
"**If you are using Vertex AI Workbench Notebook**, your environment is already authenticated.\n",
|
||||
"\n",
|
||||
"**If you are using Colab**, run the cell below and follow the instructions\n",
|
||||
"when prompted to authenticate your account via oAuth.\n",
|
||||
@@ -479,7 +491,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
|
||||
" BUCKET_NAME = PROJECT_ID + \"aip-\" + TIMESTAMP\n",
|
||||
" BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n",
|
||||
" BUCKET_URI = \"gs://\" + BUCKET_NAME"
|
||||
]
|
||||
},
|
||||
@@ -553,6 +565,10 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"IS_COLAB = \"google.colab\" in sys.modules\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" SERVICE_ACCOUNT == \"\"\n",
|
||||
" or SERVICE_ACCOUNT is None\n",
|
||||
@@ -912,12 +928,12 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"DISPLAY_NAME = \"iris_\" + TIMESTAMP\n",
|
||||
"DISPLAY_NAME = \"iris_\" + UUID\n",
|
||||
"\n",
|
||||
"job = aip.PipelineJob(\n",
|
||||
" display_name=DISPLAY_NAME,\n",
|
||||
" template_path=\"tabular classification_pipeline.json\".replace(\" \", \"_\"),\n",
|
||||
" job_id=f\"tabular classification-v2{TIMESTAMP}-1\".replace(\" \", \"\"),\n",
|
||||
" job_id=f\"tabular classification-v2{UUID}-1\".replace(\" \", \"\"),\n",
|
||||
" pipeline_root=PIPELINE_ROOT,\n",
|
||||
" parameter_values={\"seed\": 7, \"splits\": 10},\n",
|
||||
")\n",
|
||||
@@ -950,7 +966,18 @@
|
||||
"\n",
|
||||
"Next, generate another pipeline run that uses a different `seed` and `split` for the `iris_logregression` step.\n",
|
||||
"\n",
|
||||
"Submit the new pipeline run:"
|
||||
"Submit the new pipeline run:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"**pipeline_root :** Specify a Cloud Storage URI that your pipelines service account can access. The artifacts of your pipeline runs are stored within the pipeline root. \n",
|
||||
"\n",
|
||||
"**display_name :** The name of the pipeline, this will show up in the Google Cloud console. \n",
|
||||
"\n",
|
||||
"**parameter_values :** The pipeline parameters to pass to this run. For example, create a dict() with the parameter names as the dictionary keys and the parameter values as the dictionary values. \n",
|
||||
"\n",
|
||||
"**job_id :** A unique identifier for this pipeline run. If the job ID is not specified, Vertex AI Pipelines creates a job ID for you using the pipeline name and the timestamp of when the pipeline run was started. \n",
|
||||
"\n",
|
||||
"**template_path :** complete pipeline path"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -962,9 +989,9 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"job = aip.PipelineJob(\n",
|
||||
" display_name=\"iris_\" + TIMESTAMP,\n",
|
||||
" display_name=\"iris_\" + UUID,\n",
|
||||
" template_path=\"tabular classification_pipeline.json\".replace(\" \", \"_\"),\n",
|
||||
" job_id=f\"tabular classification-pipeline-v2{TIMESTAMP}-2\".replace(\" \", \"\"),\n",
|
||||
" job_id=f\"tabular classification-pipeline-v2{UUID}-2\".replace(\" \", \"\"),\n",
|
||||
" pipeline_root=PIPELINE_ROOT,\n",
|
||||
" parameter_values={\"seed\": 5, \"splits\": 7},\n",
|
||||
")\n",
|
||||
@@ -1081,16 +1108,7 @@
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial -- *Note:* this is auto-generated and not all resources may be applicable for this tutorial:\n",
|
||||
"\n",
|
||||
"- Dataset\n",
|
||||
"- Pipeline\n",
|
||||
"- Model\n",
|
||||
"- Endpoint\n",
|
||||
"- Batch Job\n",
|
||||
"- Custom Job\n",
|
||||
"- Hyperparameter Tuning Job\n",
|
||||
"- Cloud Storage Bucket"
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial -- *Note:* this is auto-generated and not all resources may be applicable for this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1101,94 +1119,9 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"delete_dataset = True\n",
|
||||
"delete_pipeline = True\n",
|
||||
"delete_model = True\n",
|
||||
"delete_endpoint = True\n",
|
||||
"delete_batchjob = True\n",
|
||||
"delete_customjob = True\n",
|
||||
"delete_hptjob = True\n",
|
||||
"delete_bucket = True\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" if delete_model and \"DISPLAY_NAME\" in globals():\n",
|
||||
" models = aip.Model.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" model = models[0]\n",
|
||||
" aip.Model.delete(model)\n",
|
||||
" print(\"Deleted model:\", model)\n",
|
||||
"except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" if delete_endpoint and \"DISPLAY_NAME\" in globals():\n",
|
||||
" endpoints = aip.Endpoint.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}_endpoint\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" endpoint = endpoints[0]\n",
|
||||
" endpoint.undeploy_all()\n",
|
||||
" aip.Endpoint.delete(endpoint.resource_name)\n",
|
||||
" print(\"Deleted endpoint:\", endpoint)\n",
|
||||
"except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
"if delete_dataset and \"DISPLAY_NAME\" in globals():\n",
|
||||
" if \"tabular\" == \"tabular\":\n",
|
||||
" try:\n",
|
||||
" datasets = aip.TabularDataset.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" dataset = datasets[0]\n",
|
||||
" aip.TabularDataset.delete(dataset.resource_name)\n",
|
||||
" print(\"Deleted dataset:\", dataset)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
" if \"tabular\" == \"image\":\n",
|
||||
" try:\n",
|
||||
" datasets = aip.ImageDataset.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" dataset = datasets[0]\n",
|
||||
" aip.ImageDataset.delete(dataset.resource_name)\n",
|
||||
" print(\"Deleted dataset:\", dataset)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
" if \"tabular\" == \"text\":\n",
|
||||
" try:\n",
|
||||
" datasets = aip.TextDataset.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" dataset = datasets[0]\n",
|
||||
" aip.TextDataset.delete(dataset.resource_name)\n",
|
||||
" print(\"Deleted dataset:\", dataset)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
" if \"tabular\" == \"video\":\n",
|
||||
" try:\n",
|
||||
" datasets = aip.VideoDataset.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" dataset = datasets[0]\n",
|
||||
" aip.VideoDataset.delete(dataset.resource_name)\n",
|
||||
" print(\"Deleted dataset:\", dataset)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" if delete_pipeline and \"DISPLAY_NAME\" in globals():\n",
|
||||
" pipelines = aip.PipelineJob.list(\n",
|
||||
" filter=f\"display_name={DISPLAY_NAME}\", order_by=\"create_time\"\n",
|
||||
" )\n",
|
||||
" pipeline = pipelines[0]\n",
|
||||
" aip.PipelineJob.delete(pipeline.resource_name)\n",
|
||||
" print(\"Deleted pipeline:\", pipeline)\n",
|
||||
"except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
"delete_bucket = False\n",
|
||||
"\n",
|
||||
"job.delete()\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil rm -r $BUCKET_URI"
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user