xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

fi: Issue 850 (#863)

Browse Source 
* fix: add missing end_run() for experiment

* fix: add missing end_run() for experiment
This commit is contained in:
Andrew Ferlitsch
2022-08-15 08:57:07 -07:00
committed by GitHub
parent b129c0bf43
commit 578cfb7da7
1 changed files with 55 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/official/ml_metadata/sdk-metric-parameter-tracking-for-custom-jobs.ipynb
+55 -28
View File
@@ -29,6 +29,8 @@
"id": "JAPoU8Sm5E6e"
},
"source": [
"# Vertex AI: Track parameters and metrics for custom training jobs\n",
"\n",
"<table align=\"left\">\n",
"\n",
" <td>\n",
@@ -51,15 +53,6 @@
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j9gUDU_3vV9d"
},
"source": [
"# Vertex AI: Track parameters and metrics for custom training jobs"
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -68,11 +61,15 @@
"source": [
"## Overview\n",
"\n",
"This notebook demonstrates how to track metrics and parameters for `Vertex AI` custom training jobs, and how to perform detailed analysis using this data.\n",
"\n",
"### Dataset\n",
"\n",
"This example uses the Abalone Dataset. For more information about this dataset please visit: https://archive.ics.uci.edu/ml/datasets/abalone\n",
"This notebook demonstrates how to track metrics and parameters for `Vertex AI` custom training jobs, and how to perform detailed analysis using this data."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "37147bd9c3c4"
},
"source": [
"### Objective\n",
"\n",
"In this notebook, you learn how to use `Vertex ML Metadata` to track training parameters and evaluation metrics.\n",
@@ -85,8 +82,26 @@
"The steps performed include:\n",
"\n",
"- Track parameters and metrics for a `Vertex AI` custom trained model.\n",
"- Extract and perform analysis for all parameters and metrics within an Experiment.\n",
"- Extract and perform analysis for all parameters and metrics within an Experiment."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "96cb18467417"
},
"source": [
"### Dataset\n",
"\n",
"This example uses the Abalone Dataset. For more information about this dataset please visit: https://archive.ics.uci.edu/ml/datasets/abalone"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c831245dc1d5"
},
"source": [
"### Costs \n",
"\n",
"\n",
@@ -285,8 +300,7 @@
},
"outputs": [],
"source": [
"if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
" PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"if PROJECT_ID == \"[your-project-id]\" or PROJECT_ID == \"\" or PROJECT_ID is None:\n",
" # Get your GCP project id from gcloud\n",
" shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
" PROJECT_ID = shell_output[0]\n",
@@ -332,7 +346,10 @@
},
"outputs": [],
"source": [
"REGION = \"us-central1\" # @param {type: \"string\"}"
"REGION = \"[your-region]\" # @param {type: \"string\"}\n",
"\n",
"if REGION == \"[your-region]\":\n",
" REGION = \"us-central1\""
]
},
{
@@ -368,7 +385,7 @@
"### Authenticate your Google Cloud account\n",
"\n",
"**If you are using Vertex AI Workbench**, your environment is already\n",
"authenticated. Skip this step."
"authenticated. "
]
},
{
@@ -664,9 +681,9 @@
"id": "35QVNhACqcTJ"
},
"source": [
"### Create a managed tabular dataset from a CSV\n",
"### Create a Vertex AI Dataset from a CSV\n",
"\n",
"A Managed dataset can be used to create an AutoML model or a custom model. "
"A Vertex AI Dataset can be used to create an AutoML model or a custom model. "
]
},
{
@@ -679,7 +696,7 @@
"source": [
"ds = aiplatform.TabularDataset.create(display_name=\"abalone\", gcs_source=[gcs_csv_path])\n",
"\n",
"ds.resource_name"
"print(ds.resource_name)"
]
},
{
@@ -780,7 +797,11 @@
"id": "k_QorXXztzPH"
},
"source": [
"Start a new experiment run to track training parameters and start the training job. Note that this operation will take around 10 mins."
"Start a new experiment run to track training parameters and start the training job. \n",
"\n",
"Prior to executing the training job, you call the `start_run()` method to initialize the start of the experiment, and then use the `log_params()` to log the parameters used in the experiment.\n",
"\n",
"*Note:* This operation will take around 10 mins."
]
},
{
@@ -818,7 +839,7 @@
"id": "O-uCOL3Naap4"
},
"source": [
"Deploy model to Google Cloud. This operation will take 10-20 mins."
"Deploy model to Google Cloud. This operation may take a few minutes."
]
},
{
@@ -899,7 +920,7 @@
"id": "_HphZ38obJeB"
},
"source": [
"Perform online prediction."
"### Perform online prediction"
]
},
{
@@ -911,7 +932,7 @@
"outputs": [],
"source": [
"prediction = endpoint.predict(test_dataset.tolist())\n",
"prediction"
"print(prediction)"
]
},
{
@@ -920,7 +941,11 @@
"id": "TDKiv_O7bNwE"
},
"source": [
"Calculate and track prediction evaluation metrics."
"### Calculate and track prediction evaluation metrics.\n",
"\n",
"Next, log the evaluation metrics for your experiment.\n",
"\n",
"Once the experiment is completed, you call the `end_run()` method to indicate the end of tracking for the experiment."
]
},
{
@@ -934,7 +959,9 @@
"mse = mean_squared_error(test_labels, prediction.predictions)\n",
"mae = mean_absolute_error(test_labels, prediction.predictions)\n",
"\n",
"aiplatform.log_metrics({\"mse\": mse, \"mae\": mae})"
"aiplatform.log_metrics({\"mse\": mse, \"mae\": mae})\n",
"\n",
"aiplatform.end_run()"
]
},
{