mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-27 15:42:05 +00:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Andrew Ferlitsch
|
d6a59b5a7a |
@@ -11,3 +11,4 @@ google-cloud-storage
|
||||
google-cloud-build
|
||||
ratemate
|
||||
GitPython
|
||||
google-api-core==2.10
|
||||
|
||||
@@ -5,6 +5,6 @@ nbconvert
|
||||
black==22.10.0
|
||||
pyupgrade==2.38.4
|
||||
isort==5.10.1
|
||||
flake8==6.0.0
|
||||
flake8==4.0.1
|
||||
nbqa==1.5.3
|
||||
|
||||
|
||||
@@ -40,4 +40,4 @@
|
||||
/notebooks/community/pipelines/google_cloud_pipeline_components_bqml_pipeline_anomaly_detection.ipynb @inardini
|
||||
/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb @Narwhalprime
|
||||
/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb @Narwhalprime
|
||||
/notebooks/community/feature_store/get_started_vertex_feature_store.ipynb @junkourata
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1028,9 +1028,6 @@
|
||||
"deployment_resource_pool.dedicated_resources.min_replica_count = MIN_NODES\n",
|
||||
"deployment_resource_pool.dedicated_resources.max_replica_count = MAX_NODES\n",
|
||||
"deployment_resource_pool.dedicated_resources.machine_spec.machine_type = DEPLOY_COMPUTE\n",
|
||||
"if DEPLOY_NGPU:\n",
|
||||
" deployment_resource_pool.dedicated_resources.machine_spec.accelerator_type = DEPLOY_GPU\n",
|
||||
" deployment_resource_pool.dedicated_resources.machine_spec.accelerator_count = DEPLOY_NGPU\n",
|
||||
"\n",
|
||||
"request = aip_beta.CreateDeploymentResourcePoolRequest(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}\",\n",
|
||||
|
||||
+41
-95
@@ -54,20 +54,18 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "239ba71252d3"
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "25c28706c23e"
|
||||
},
|
||||
"source": [
|
||||
"This notebook shows how to use `Vertex AI Pipelines` and `BigQuery ML pipeline components` to train and evaluate a demand forecasting model.\n",
|
||||
"\n",
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
|
||||
"](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n",
|
||||
"\n",
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"In this tutorial, you learn how to train and evaluate a BigQuery ML model using Vertex AI Pipelines and BigQuery ML pipeline components. \n",
|
||||
@@ -89,27 +87,8 @@
|
||||
" - Generate the ARIMA Plus forecasts\n",
|
||||
" - Generate the ARIMA PLUS forecast explainations\n",
|
||||
"- Compile the pipeline.\n",
|
||||
"- Execute the pipeline."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "586acfa9b502"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"- Execute the pipeline.\n",
|
||||
"\n",
|
||||
"The dataset is a modified version of the dataset in [Build and visualize demand forecast predictions using Datastream, Dataflow, BigQuery ML, and Looker\n",
|
||||
"](https://cloud.google.com/architecture/build-visualize-demand-forecast-prediction-datastream-dataflow-bigqueryml-looker) solution architecture\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"### Costs \n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
@@ -373,8 +352,9 @@
|
||||
"id": "06571eb4063b"
|
||||
},
|
||||
"source": [
|
||||
"#### UUID\n",
|
||||
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
|
||||
"#### Timestamp\n",
|
||||
"\n",
|
||||
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -385,16 +365,9 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"import string\n",
|
||||
"from datetime import datetime\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Generate a uuid of a specifed length(default=8)\n",
|
||||
"def generate_uuid(length: int = 8) -> str:\n",
|
||||
" return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"UUID = generate_uuid()"
|
||||
"TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -512,7 +485,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
|
||||
" BUCKET_NAME = PROJECT_ID + \"-aip-\" + UUID\n",
|
||||
" BUCKET_NAME = PROJECT_ID + \"-aip-\" + TIMESTAMP\n",
|
||||
" BUCKET_URI = f\"gs://{BUCKET_NAME}\""
|
||||
]
|
||||
},
|
||||
@@ -733,7 +706,6 @@
|
||||
"KFP_COMPONENTS_PATH = \"components\"\n",
|
||||
"PIPELINES_PATH = \"pipelines\"\n",
|
||||
"\n",
|
||||
"! mkdir -m 777 -p {DATA_PATH}\n",
|
||||
"! mkdir -m 777 -p {KFP_COMPONENTS_PATH}\n",
|
||||
"! mkdir -m 777 -p {PIPELINES_PATH}"
|
||||
]
|
||||
@@ -799,7 +771,7 @@
|
||||
" --location={LOCATION} \\\n",
|
||||
" --source_format=CSV \\\n",
|
||||
" --skip_leading_rows=1\\\n",
|
||||
" fast_fresh.orders_{UUID} \\\n",
|
||||
" fast_fresh.orders_{TIMESTAMP} \\\n",
|
||||
" {RAW_DATA_URI} \\\n",
|
||||
" time_of_sale:DATETIME,order_id:INTEGER,product_name:STRING,price:NUMERIC,quantity:NUMERIC,payment_method:STRING,store_id:INTEGER,user_id:INTEGER"
|
||||
]
|
||||
@@ -810,7 +782,7 @@
|
||||
"id": "ZrgOD30o7HcL"
|
||||
},
|
||||
"source": [
|
||||
"## BigQuery ML Training Formalization\n",
|
||||
"## BQML Training Formalization\n",
|
||||
"\n",
|
||||
"In the next cells, you build the components and pipeline to train and evaluate the BQML demand forecasting model."
|
||||
]
|
||||
@@ -848,13 +820,13 @@
|
||||
"BQ_EVALUATE_MODEL_TABLE_PREFIX = \"orders_arima_model_evaluate\"\n",
|
||||
"BQ_FORECAST_TABLE_PREFIX = \"orders_arima_forecast\"\n",
|
||||
"BQ_EXPLAIN_FORECAST_TABLE_PREFIX = \"orders_arima_explain_forecast\"\n",
|
||||
"BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{UUID}\"\n",
|
||||
"BQ_ORDERS_TABLE = f\"{BQ_ORDERS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_TRAINING_TABLE = f\"{BQ_TRAINING_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_MODEL_TABLE = f\"{BQ_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_EVALUATE_TS_TABLE = f\"{BQ_EVALUATE_TS_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_EVALUATE_MODEL_TABLE = f\"{BQ_EVALUATE_MODEL_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_FORECAST_TABLE = f\"{BQ_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"BQ_EXPLAIN_FORECAST_TABLE = f\"{BQ_EXPLAIN_FORECAST_TABLE_PREFIX}_{TIMESTAMP}\"\n",
|
||||
"\n",
|
||||
"BQ_TRAIN_CONFIGURATION = {\n",
|
||||
" \"destinationTable\": {\n",
|
||||
@@ -1050,7 +1022,7 @@
|
||||
"id": "pcSL1FHk69KT"
|
||||
},
|
||||
"source": [
|
||||
"### Build the BigQuery ML training pipeline\n",
|
||||
"### Build the BQML training pipeline\n",
|
||||
"\n",
|
||||
"Define your workflow using Kubeflow Pipelines DSL package. \n",
|
||||
"\n",
|
||||
@@ -1122,8 +1094,8 @@
|
||||
" location=location,\n",
|
||||
" ).set_display_name(\"get train data\")\n",
|
||||
"\n",
|
||||
" # Run an ARIMA PLUS experiment\n",
|
||||
" bq_arima_model_exp_op = (\n",
|
||||
" # Train the ARIMA PLUS model\n",
|
||||
" bq_arima_model_op = (\n",
|
||||
" BigqueryCreateModelJobOp(\n",
|
||||
" query=f\"\"\"\n",
|
||||
" -- create model table\n",
|
||||
@@ -1132,7 +1104,10 @@
|
||||
" MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
|
||||
" TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
|
||||
" TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
|
||||
" TIME_SERIES_ID_COL = [\\'product_name\\']\n",
|
||||
" TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
|
||||
" MODEL_REGISTRY = \\'vertex_ai\\',\n",
|
||||
" VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
|
||||
" VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
|
||||
" ) AS\n",
|
||||
" SELECT\n",
|
||||
" hourly_timestamp,\n",
|
||||
@@ -1144,7 +1119,7 @@
|
||||
" project=project,\n",
|
||||
" location=location,\n",
|
||||
" )\n",
|
||||
" .set_display_name(\"run arima+ model experiment\")\n",
|
||||
" .set_display_name(\"train arima plus model\")\n",
|
||||
" .after(create_training_dataset_op)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
@@ -1153,12 +1128,12 @@
|
||||
" BigqueryMLArimaEvaluateJobOp(\n",
|
||||
" project=project,\n",
|
||||
" location=location,\n",
|
||||
" model=bq_arima_model_exp_op.outputs[\"model\"],\n",
|
||||
" model=bq_arima_model_op.outputs[\"model\"],\n",
|
||||
" show_all_candidate_models=False,\n",
|
||||
" job_configuration_query=bq_evaluate_time_series_configuration,\n",
|
||||
" )\n",
|
||||
" .set_display_name(\"evaluate arima plus time series\")\n",
|
||||
" .after(bq_arima_model_exp_op)\n",
|
||||
" .after(bq_arima_model_op)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Evaluate ARIMA Plus model\n",
|
||||
@@ -1166,12 +1141,12 @@
|
||||
" BigqueryEvaluateModelJobOp(\n",
|
||||
" project=project,\n",
|
||||
" location=location,\n",
|
||||
" model=bq_arima_model_exp_op.outputs[\"model\"],\n",
|
||||
" model=bq_arima_model_op.outputs[\"model\"],\n",
|
||||
" query_statement=f\"\"\"SELECT * FROM `{project}.{bq_dataset}.{bq_training_table}` WHERE split='TEST'\"\"\",\n",
|
||||
" job_configuration_query=bq_evaluate_model_configuration,\n",
|
||||
" )\n",
|
||||
" .set_display_name(\"evaluate arima plus model\")\n",
|
||||
" .after(bq_arima_model_exp_op)\n",
|
||||
" .after(bq_arima_model_op)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Plot model metrics\n",
|
||||
@@ -1189,34 +1164,6 @@
|
||||
" < PERF_THRESHOLD,\n",
|
||||
" name=\"avg. mae good\",\n",
|
||||
" ):\n",
|
||||
" # Train the ARIMA PLUS model\n",
|
||||
" bq_arima_model_op = (\n",
|
||||
" BigqueryCreateModelJobOp(\n",
|
||||
" query=f\"\"\"\n",
|
||||
" -- create model table\n",
|
||||
" CREATE OR REPLACE MODEL `{project}.{bq_dataset}.{bq_model_table}`\n",
|
||||
" OPTIONS(\n",
|
||||
" MODEL_TYPE = \\'ARIMA_PLUS\\',\n",
|
||||
" TIME_SERIES_TIMESTAMP_COL = \\'hourly_timestamp\\',\n",
|
||||
" TIME_SERIES_DATA_COL = \\'total_sold\\',\n",
|
||||
" TIME_SERIES_ID_COL = [\\'product_name\\'],\n",
|
||||
" MODEL_REGISTRY = \\'vertex_ai\\',\n",
|
||||
" VERTEX_AI_MODEL_ID = \\'order_demand_forecasting\\',\n",
|
||||
" VERTEX_AI_MODEL_VERSION_ALIASES = [\\'staging\\']\n",
|
||||
" ) AS\n",
|
||||
" SELECT\n",
|
||||
" DATETIME_TRUNC(time_of_sale, HOUR) as hourly_timestamp,\n",
|
||||
" product_name,\n",
|
||||
" SUM(quantity) AS total_sold,\n",
|
||||
" FROM `{project}.{bq_dataset}.{bq_orders_table}`\n",
|
||||
" GROUP BY hourly_timestamp, product_name;\n",
|
||||
" \"\"\",\n",
|
||||
" project=project,\n",
|
||||
" location=location,\n",
|
||||
" )\n",
|
||||
" .set_display_name(\"train arima+ model\")\n",
|
||||
" .after(get_evaluation_model_metrics_op)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Generate the ARIMA PLUS forecasts\n",
|
||||
" bq_arima_forecast_op = (\n",
|
||||
@@ -1277,7 +1224,7 @@
|
||||
"source": [
|
||||
"### Execute your pipeline\n",
|
||||
"\n",
|
||||
"Next, we execute the pipeline. It takes the following parameters which we set as default:\n",
|
||||
"Next, you execute the pipeline. It takes the following parameters which we set as default:\n",
|
||||
"\n",
|
||||
"- `bq_dataset`: The BigQuery dataset to train on.\n",
|
||||
"- `bq_orders_table` : The BigQuery table of raw data.\n",
|
||||
@@ -1319,7 +1266,7 @@
|
||||
"source": [
|
||||
"### View BigQuery ML training pipeline results\n",
|
||||
"\n",
|
||||
"Finally, you view the artifact outputs of each task in the pipeline."
|
||||
"Finally, you will view the artifact outputs of each task in the pipeline."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1395,8 +1342,8 @@
|
||||
"print(\"bigquery-ml-arima-evaluate-job\")\n",
|
||||
"artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-ml-arima-evaluate-job\")\n",
|
||||
"print(\"\\n\\n\")\n",
|
||||
"print(\"bigquery-evaluate-model-job\")\n",
|
||||
"artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-evaluate-model-job\")\n",
|
||||
"print(\"get-model-evaluation-metrics\")\n",
|
||||
"artifacts = print_pipeline_output(bqml_pipeline, \"get-model-evaluation-metrics\")\n",
|
||||
"print(\"\\n\\n\")\n",
|
||||
"print(\"bigquery-forecast-model-job\")\n",
|
||||
"artifacts = print_pipeline_output(bqml_pipeline, \"bigquery-forecast-model-job\")\n",
|
||||
@@ -1460,8 +1407,7 @@
|
||||
"\n",
|
||||
"# Remove local resorces\n",
|
||||
"! rm -rf {KFP_COMPONENTS_PATH}\n",
|
||||
"! rm -rf {PIPELINES_PATH}\n",
|
||||
"! rm -rf {DATA_PATH}"
|
||||
"! rm -rf {PIPELINES_PATH}"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
+5
-5
@@ -42,12 +42,12 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_cloud_natural_language_pipeline.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/natural_language/cloud_natural_language_pipeline.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
@@ -328,7 +328,7 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"your-project-id\" # @param {type:\"string\"}\n",
|
||||
"PROJECT_ID = \"cloud-ml-language-test\" # @param {type:\"string\"}\n",
|
||||
"if PROJECT_ID == \"\" or PROJECT_ID is None or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" # Get your GCP project id from gcloud\n",
|
||||
" shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
|
||||
@@ -368,7 +368,7 @@
|
||||
"source": [
|
||||
"REGION = \"us\" # @param {type:\"string\"}\n",
|
||||
"LOCATION = \"us-central1\" # @param {type:\"string\"}\n",
|
||||
"TRAINING_DATA_LOCATION = \"gs://your-training-data-location\" # @param {type:\"string\"}\n",
|
||||
"TRAINING_DATA_LOCATION = \"gs://dougchen-20221130-pipeline-colab-test/data-00001-of-00001.jsonl\" # @param {type:\"string\"}\n",
|
||||
"TASK_TYPE = \"CLASSIFICATION\" # @param [\"CLASSIFICATION\", \"MULTILABEL_CLASSIFICATION\"]"
|
||||
]
|
||||
},
|
||||
@@ -740,7 +740,7 @@
|
||||
"\n",
|
||||
"This sends a create pipeline job request to Vertex Pipelines. Note that this task run synchronously and may take a while to complete.\n",
|
||||
"\n",
|
||||
"You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline."
|
||||
"You may view the progress of the job at any time by clicking on the generated links (after \"View Pipeline Job\" in the console output of the cell below). Once the pipeline finishes, you may examine the artifacts produced from this pipeline. See "
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
+3
-3
@@ -42,12 +42,12 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/pipelines/google_cloud_pipeline_components_ready_to_go_text_classification_pipeline.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/ready_to_go_text_classification_pipeline/ready_to_go_text_classification_pipeline.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
@@ -706,7 +706,7 @@
|
||||
")\n",
|
||||
"\n",
|
||||
"# The GCS directory for keeping staging files for model evaluation.\n",
|
||||
"ROOT_DIR = \"'f\\\"{BASE_OUTPUT_DIR}/root\\\"'\" # @param {type:\"string\"}"
|
||||
"ROOT_DIR = 'f\"{BASE_OUTPUT_DIR}/root\"' # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -112,7 +112,7 @@ def benchmark(
|
||||
|
||||
results = []
|
||||
for qps in qps_list:
|
||||
num_requests = int(max(qps * duration_sec, 10))
|
||||
num_requests = max(qps * duration_sec, 10)
|
||||
requests_for_qps = list(
|
||||
itertools.islice(itertools.cycle(requests), num_requests)
|
||||
)
|
||||
|
||||
-1569
File diff suppressed because it is too large
Load Diff
@@ -12,10 +12,6 @@
|
||||
--errors-codes: A list of error codes to report errors. Otherwise, all errors are reported.
|
||||
--errors-csv: Report errors in CSV format
|
||||
|
||||
# options for automatic fixing
|
||||
--fix: Automatic fix
|
||||
--fix-codes: A list of fix codes to fix. Otherwise, all fix codes are enabled.
|
||||
|
||||
# index generatation
|
||||
--repo: Generate index in markdown format
|
||||
--web: Generate index in HTML format
|
||||
@@ -23,7 +19,6 @@
|
||||
--desc: Add description to index
|
||||
--steps: Add steps to index
|
||||
--uses: Add "resources" used to index
|
||||
--linkback: Add linkback to index
|
||||
|
||||
Format of CSV file for notebooks to review:
|
||||
|
||||
|
||||
@@ -30,7 +30,6 @@
|
||||
/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb @TheMichaelHu
|
||||
/automl/automl_tabular_on_vertex_pipelines.ipynb @helinwang
|
||||
/custom/custom_training_tensorboard_profiler.ipynb @itseric
|
||||
/custom/get_started_with_vertex_endpoint_and_shared_vm.ipynb @andrewferlitsch @xingziye
|
||||
/workbench/spark/spark_sample_notebook.ipynb @bradmiro
|
||||
/workbench/spark/spark_ml.ipynb @bradmiro
|
||||
/model_registry/bqml_vertexai_model_registry.ipynb @soheilazangeneh
|
||||
|
||||
@@ -14,7 +14,7 @@ The steps performed include the following:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
Learn more about [Tabular classification](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
|
||||
|
||||
[Create, train, and deploy an AutoML text classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl-text-classification.ipynb)
|
||||
@@ -34,7 +34,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Classification for text data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_text).
|
||||
Learn more about [AutoML Text classification](https://cloud.google.com/vertex-ai/docs/text-data/classification/train-model).
|
||||
|
||||
|
||||
[Compare Vertex AI Forecasting and BigQuery ML ARIMA_PLUS](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb)
|
||||
@@ -54,7 +54,9 @@ The steps performed are:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [BQML ARIMA+ forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-arima/overview).
|
||||
Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).
|
||||
|
||||
Learn more about [BQML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-arima/overview).
|
||||
|
||||
|
||||
[AutoML Tabular Workflow pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_tabular_on_vertex_pipelines.ipynb)
|
||||
@@ -69,26 +71,7 @@ The steps performed are:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for E2E AutoML](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl).
|
||||
|
||||
|
||||
[Get started with AutoML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/get_started_automl_training.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `AutoML` for training with `Vertex AI`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Train an image model
|
||||
- Export the image model as an edge model
|
||||
- Train a tabular model
|
||||
- Export the tabular model as a cloud model
|
||||
- Train a text model
|
||||
- Train a video model
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI for AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users).
|
||||
Learn more about [AutoML Tabular Workflows](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl).
|
||||
|
||||
|
||||
[AutoML training hierarchical forecasting for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_forecasting_hierarchical_batch.ipynb)
|
||||
@@ -107,7 +90,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Hierarchical forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/hierarchical).
|
||||
Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).
|
||||
|
||||
|
||||
[AutoML training image object detection model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_image_object_detection_batch.ipynb)
|
||||
@@ -124,7 +107,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).
|
||||
Learn more about [AutoML Image](https://cloud.google.com/vertex-ai/docs/image-data/object-detection/train-model).
|
||||
|
||||
|
||||
[AutoML tabular forecasting model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_forecasting_batch.ipynb)
|
||||
@@ -141,7 +124,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/overview).
|
||||
Learn more about [AutoML Forecasting](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting/tutorials-samples).
|
||||
|
||||
|
||||
[AutoML training tabular regression model for batch prediction using BigQuery](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_regression_batch_bq.ipynb)
|
||||
@@ -160,7 +143,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/training-overview#tabular_data).
|
||||
|
||||
|
||||
[AutoML training tabular regression model for online prediction using BigQuery](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_regression_online_bq.ipynb)
|
||||
@@ -179,7 +162,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/training-overview#tabular_data).
|
||||
|
||||
|
||||
[AutoML training text entity extraction model for online prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_text_entity_extraction_online.ipynb)
|
||||
@@ -198,7 +181,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Entity extraction for text data](https://cloud.google.com/vertex-ai/docs/training-overview#entity_extraction_for_text).
|
||||
Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/entity-extraction/train-model).
|
||||
|
||||
|
||||
[Training an AutoML text sentiment analysis model for online predictions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_text_sentiment_analysis_online.ipynb)
|
||||
@@ -218,7 +201,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text).
|
||||
Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/sentiment-analysis/train-model).
|
||||
|
||||
|
||||
[AutoML training video action recognition model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_action_recognition_batch.ipynb)
|
||||
@@ -235,7 +218,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Action recognition for video data](https://cloud.google.com/vertex-ai/docs/training-overview#action_recognition_for_videos).
|
||||
Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/action-recognition/train-model).
|
||||
|
||||
|
||||
[AutoML training video classification model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_classification_batch.ipynb)
|
||||
@@ -252,7 +235,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos).
|
||||
Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/classification/train-model).
|
||||
|
||||
|
||||
[AutoML training video object tracking model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_object_tracking_batch.ipynb)
|
||||
@@ -269,5 +252,5 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Object tracking for video data](https://cloud.google.com/vertex-ai/docs/training-overview#object_tracking_for_videos).
|
||||
Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/object-tracking/train-model).
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -15,25 +15,5 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
|
||||
|
||||
[Get started with BigQuery ML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery_ml/get_started_with_bqml_training.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `BigQueryML` for training with `Vertex AI`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a local BigQuery table in your project
|
||||
- Train a BigQuery ML model
|
||||
- Evaluate the BigQuery ML model
|
||||
- Export the BigQuery ML model as a cloud model
|
||||
- Upload the exported model as a `Vertex AI Model` resource
|
||||
- Hyperparameter tune a BigQuery ML model with `Vertex AI Vizier`
|
||||
- Automatically register a BigQuery ML model to `Vertex AI Model Registry`
|
||||
|
||||
```
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction).
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
"\n",
|
||||
"This notebook is aimed at data analysts and data scientists who have data in BigQuery, want to train a model using BigQuery ML, register the model to Vertex AI Model Registry, and deploy it to an endpoint for real-time prediction. \n",
|
||||
"\n",
|
||||
"Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
|
||||
"Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -29,7 +29,7 @@
|
||||
"id": "title:generic,gcp"
|
||||
},
|
||||
"source": [
|
||||
"# Get started with BigQuery datasets\n",
|
||||
"# E2E ML on GCP: MLOps stage 1 : data management: get started with BigQuery datasets\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
@@ -61,7 +61,7 @@
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with BigQuery datasets.\n",
|
||||
"This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with BigQuery datasets.\n",
|
||||
"\n",
|
||||
"Learn more about [BigQuery Datasets](https://cloud.google.com/bigquery/docs/datasets-intro)."
|
||||
]
|
||||
|
||||
@@ -29,7 +29,7 @@
|
||||
"id": "title:generic,gcp"
|
||||
},
|
||||
"source": [
|
||||
"# Get started with Vertex AI Data Labeling\n",
|
||||
"# E2E ML on GCP: MLOps stage 1 : formalization: get started with Vertex AI Data Labeling\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
@@ -62,7 +62,7 @@
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with Vertex AI Data Labeling service.\n",
|
||||
"This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with Vertex AI Data Labeling service.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Data Labeling](https://cloud.google.com/vertex-ai/docs/datasets/data-labeling-job)."
|
||||
]
|
||||
@@ -352,10 +352,7 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"EMAIL = \"[your-email-address]\" # @param {type: \"string\"}\n",
|
||||
"\n",
|
||||
"if os.getenv(\"IS_TESTING\"):\n",
|
||||
" EMAIL = \"noreply@google.com\""
|
||||
"EMAIL = \"[your-email-address]\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -55,36 +55,3 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
[Get started with Vertex AI Experiments](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_vertex_experiments.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Experiments` when training with `Vertex AI`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Local (notebook) Training
|
||||
- Create an experiment
|
||||
- Create a first run in the experiment
|
||||
- Log parameters and metrics
|
||||
- Create artifact lineage
|
||||
- Visualize the experiment results
|
||||
- Execute a second run
|
||||
- Compare the two runs in the experiment
|
||||
- Cloud (`Vertex AI`) Training
|
||||
- Within the training script:
|
||||
- Create an experiment
|
||||
- Log parameters and metrics
|
||||
- Create artifact lineage
|
||||
- Create a `Vertex AI Training` custom job
|
||||
- Execute the custom job
|
||||
- Visualize the experiment results
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Experiments](https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments).
|
||||
|
||||
Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata).
|
||||
|
||||
Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -15,128 +15,6 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Model Monitoring for batch predictions](https://cloud.google.com/vertex-ai/docs/model-monitoring/model-monitoring-batch-predictions).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for AutoML tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_automl.ipynb)
|
||||
|
||||
```
|
||||
Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for AutoML tabular models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Train an `AutoML` model.
|
||||
- Deploy the `Model` resource to the `Endpoint` resource.
|
||||
- Configure the `Endpoint` resource for model monitoring.
|
||||
- Generate synthetic prediction requests for skew.
|
||||
- Wait for email alert notification.
|
||||
- Generate synthetic prediction requests for drift.
|
||||
- Wait for email alert notification.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for batch prediction in AutoML image models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_automl_image_batch.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Model Monitoring` with `Vertex AI Batch Prediction` with an AutoML image classification model to detect an out of distribution image.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
1. Train an AutoML image classification model.
|
||||
2. Submit a batch prediction containing both in and out of distribution images.
|
||||
3. Use Model Monitoring to calculate anomaly score on each image.
|
||||
4. Identify the images in the batch prediction request that are out of distribution.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for custom tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_custom.ipynb)
|
||||
|
||||
```
|
||||
Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for custom tabular models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download a pre-trained custom tabular model.
|
||||
- Upload the pre-trained model as a `Model` resource.
|
||||
- Deploy the `Model` resource to the `Endpoint` resource.
|
||||
- Configure the `Endpoint` resource for model monitoring.
|
||||
- Generate synthetic prediction requests for skew.
|
||||
- Wait for email alert notification.
|
||||
- Generate synthetic prediction requests for drift.
|
||||
- Wait for email alert notification.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for custom tabular models with TensorFlow Serving container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_custom_tf_serving.ipynb)
|
||||
|
||||
```
|
||||
Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests, for custom tabular models, using a custom deployment container.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download a pre-trained custom tabular model.
|
||||
- Upload the pre-trained model as a `Model` resource.
|
||||
- Deploying the `Model` resource to an `Endpoint` resource with `TensorFlow Serving` serving binary.
|
||||
- Configure the `Endpoint` resource for model monitoring.
|
||||
- Generate synthetic prediction requests for skew.
|
||||
- Wait for email alert notification.
|
||||
- Generate synthetic prediction requests for drift.
|
||||
- Wait for email alert notification.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for setup for tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_setup.ipynb)
|
||||
|
||||
```
|
||||
Learn to setup the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download a pre-trained custom tabular model.
|
||||
- Upload the pre-trained model as a `Model` resource.
|
||||
- Deploy the `Model` resource to the `Endpoint` resource.
|
||||
- Configure the `Endpoint` resource for model monitoring.
|
||||
- Skew and drift detection for feature inputs.
|
||||
- Skew and drift detection for feature attributions.
|
||||
- Automatic generation of the `input schema` by sending 1000 prediction request.
|
||||
- List, pause, resume and delete monitoring jobs.
|
||||
- Restart monitoring job with predefined `input schema`.
|
||||
- View logged monitored data.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring for XGBoost models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_xgboost.ipynb)
|
||||
|
||||
```
|
||||
Learn to use the `Vertex AI Model Monitoring` service to detect feature skew and drift in the input predict requests for XGBoost models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download a pre-trained XGBoost model.
|
||||
- Upload the pre-trained model as a `Model` resource.
|
||||
- Deploy the `Model` resource to the `Endpoint` resource.
|
||||
- Configure the `Endpoint` resource for model monitoring:
|
||||
- drift detection only -- no access to training data.
|
||||
- predefine the input schema to map feature alias names to the unnamed array input to the model.
|
||||
- Generate synthetic prediction requests for drift.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring).
|
||||
|
||||
|
||||
[Vertex AI Model Monitoring with Explainable AI Feature Attributions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/model_monitoring.ipynb)
|
||||
|
||||
```
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
-1331
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
-1878
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -17,5 +17,5 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction).
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -285,24 +285,3 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
[BQML and AutoML - Experimenting with Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/rapid_prototyping_bqml_automl.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Predictions` for rapid prototyping a model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Creating a BigQuery and Vertex AI training dataset.
|
||||
- Training a BigQuery ML and AutoML model.
|
||||
- Extracting evaluation metrics from the BigQueryML and AutoML models.
|
||||
- Selecting the best trained model.
|
||||
- Deploying the best trained model.
|
||||
- Testing the deployed model infrastructure.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component).
|
||||
|
||||
Learn more about [BigQuery ML components](https://cloud.google.com/vertex-ai/docs/pipelines/bigqueryml-component).
|
||||
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
|
||||
[Training, tuning and deploying a PyTorch stable diffusion model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/stable_diffusion/dreambooth_stablediffusion.ipynb)
|
||||
|
||||
```
|
||||
Learn to build, train, tune and deploy a stable diffusion 2.0 model using HuggingFace and PyTorch on
|
||||
[Vertex AI](https://cloud.google.com/vertex/).
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Fine tune the stable diffusion 2.0 model with your own images.
|
||||
- Upload the model artifacts to Google Cloud Storage.
|
||||
- Check the created model artifacts.
|
||||
- Upload the model in the PyTorch prebuilt container.
|
||||
- Create an endpoint on Vertex AI.
|
||||
- Deploy the model to the endpoint.
|
||||
- Generate a new image.
|
||||
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -12,7 +12,9 @@ The steps performed are:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview).
|
||||
|
||||
|
||||
[Vertex AI TabNet](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/tabnet_vertex_tutorial.ipynb)
|
||||
@@ -30,5 +32,7 @@ The steps performed are:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
|
||||
@@ -65,7 +65,7 @@
|
||||
"\n",
|
||||
"The goal of the tutorial is to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm.\n",
|
||||
"\n",
|
||||
"Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
|
||||
"Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -67,7 +67,7 @@
|
||||
"\n",
|
||||
"TabNet uses a machine learning technique called sequential attention to select which model features to reason from at each step in the model. This mechanism makes it possible to explain how the model arrives at its predictions and helps it learn more accurate models. Thanks to this design, TabNet not only outperforms other neural networks and decision trees but also provides interpretable feature attributions. Releasing TabNet as a First Party Trainer in Vertex AI means you'll be able to easily take advantage of TabNet's architecture and explainability and use it to train models on your own data. \n",
|
||||
"\n",
|
||||
"Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
|
||||
"Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -11,7 +11,9 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
[Wide & Deep Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/wide_and_deep_on_vertex_pipelines.ipynb)
|
||||
@@ -26,5 +28,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep).
|
||||
Learn more about [Vertex AI Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep).
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
"\n",
|
||||
"This notebook showcases how to run the TabNet algorithm using Vertex AI Tabular Workflows.\n",
|
||||
"\n",
|
||||
"Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
|
||||
"Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
"\n",
|
||||
"This notebook showcases how to run the Wide & Deep algorithm using Vertex AI Tabular Workflows.\n",
|
||||
"\n",
|
||||
"Learn more about [Tabular Workflow for Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep)."
|
||||
"Learn more about [Vertex AI Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -1,794 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2022 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI TensorBoard Hyperparameter Tuning with the HParams Dashboard\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "24743cf4a1e1"
|
||||
},
|
||||
"source": [
|
||||
"**_NOTE_**: This notebook has been tested in the following environments:\n",
|
||||
"\n",
|
||||
"* Python version = 3.8"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"### What is Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"[Open source TensorBoard](https://www.tensorflow.org/tensorboard/get_started)\n",
|
||||
"(TB) is a Google open source project for machine learning experiment\n",
|
||||
"visualization. Vertex AI TensorBoard is an enterprise-ready managed\n",
|
||||
"version of TensorBoard.\n",
|
||||
"\n",
|
||||
"Vertex AI TensorBoard provides various detailed visualizations, including the following:\n",
|
||||
"\n",
|
||||
"* Tracking and visualizing metrics, such as loss and accuracy over time.\n",
|
||||
"* Visualizing model computational graphs (ops and layers).\n",
|
||||
"* Viewing histograms of weights, biases, or other tensors as they change over time.\n",
|
||||
"* Projecting embeddings to a lower dimensional space.\n",
|
||||
"* Displaying image, text, and audio samples.\n",
|
||||
"\n",
|
||||
"In addition to the powerful visualizations from\n",
|
||||
"TensorBoard, Vertex AI TensorBoard provides the following benefits:\n",
|
||||
"\n",
|
||||
"* A persistent, shareable link to your experiment's dashboard.\n",
|
||||
"\n",
|
||||
"* A searchable list of all experiments in a project.\n",
|
||||
"\n",
|
||||
"* Integrations with Vertex AI services for model training.\n",
|
||||
"\n",
|
||||
"* Enterprise-grade security, privacy, and compliance.\n",
|
||||
"\n",
|
||||
"With Vertex AI TensorBoard, you can track, visualize, and compare\n",
|
||||
"ML experiments and share them with your team.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d975e698c9a4"
|
||||
},
|
||||
"source": [
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"This tutorial shows you how to log hyperparameter experiment results in TensorFlow and visualize the results in TensorBoard's Hparams dashboard.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Vertex AI services and resources:\n",
|
||||
"\n",
|
||||
"- Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"\n",
|
||||
"* Adapt TensorFlow runs to log hyperparameters and metrics.\n",
|
||||
"* Start runs and log them all under one parent directory.\n",
|
||||
"* Visualize the results in TensorBoard's HParams dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "08d289fa873f"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"This tutorial uses the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aed92deeb4a0"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses the following billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
|
||||
"and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1fD9UZaygyPG"
|
||||
},
|
||||
"source": [
|
||||
"## Set up your local development environment\n",
|
||||
"\n",
|
||||
"**If you are using Colab or Vertex AI Workbench**, your environment already meets all the requirements to run this notebook. You can skip this step.\n",
|
||||
"\n",
|
||||
"Otherwise, make sure your environment meets this notebook's requirements. You need the following:\n",
|
||||
"\n",
|
||||
"- Git\n",
|
||||
"- Python 3\n",
|
||||
"- virtualenv\n",
|
||||
"- Jupyter notebook running in a virtual environment with Python 3\n",
|
||||
"\n",
|
||||
"To quickly set up your environment to meet the requirements of this tutorial, perform the following:\n",
|
||||
"\n",
|
||||
"1. [Install and initialize the SDK](https://cloud.google.com/sdk/docs/).\n",
|
||||
"\n",
|
||||
"2. [Install Python 3](https://cloud.google.com/python/setup#installing_python).\n",
|
||||
"\n",
|
||||
"3. [Install virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv) and create a virtual environment that uses Python 3 and activate the virtual environment.\n",
|
||||
"\n",
|
||||
"4. Install Jupyter by running the following command in a terminal shell:\n",
|
||||
"<br> `pip3 install jupyter`\n",
|
||||
"\n",
|
||||
"5. Launch Jupyter by running the following command in a terminal shell: <br> `jupyter notebook`\n",
|
||||
"\n",
|
||||
"6. Open this tutorial notebook in the Jupyter Notebook Dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "i7EUnXsZhAGF"
|
||||
},
|
||||
"source": [
|
||||
"## Install dependencies\n",
|
||||
"\n",
|
||||
"Install the following packages required to run this tutorial notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "th7tWguZiSN2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# The Vertex AI Workbench Notebook product has specific requirements\n",
|
||||
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
|
||||
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
|
||||
" \"/opt/deeplearning/metadata/env_version\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
|
||||
"USER_FLAG = \"\"\n",
|
||||
"if IS_WORKBENCH_NOTEBOOK:\n",
|
||||
" USER_FLAG = \"--user\"\n",
|
||||
"\n",
|
||||
"! pip3 install --upgrade google-cloud-aiplatform[tensorboard] tensorflow==2.7 {USER_FLAG} -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "58707a750154"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "f200f10a1da3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BF1j6f9HApxa"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"4. If you are running this notebook locally, install the [Cloud SDK](https://cloud.google.com/sdk)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WReHDGG5g0XY"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "oM1iC_MfAts1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"source": [
|
||||
"#### Set the region\n",
|
||||
"\n",
|
||||
"**Optional**: Update the 'REGION' variable to specify the region that you want to use. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nsN5NJKSu-GU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sBCra4QMA2wR"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"To authenticate your Google Cloud account, follow the instructions for your Jupyter environment:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "74ccc9e52986"
|
||||
},
|
||||
"source": [
|
||||
"* **Vertex AI Workbench**\n",
|
||||
"<br>You are already authenticated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "de775a3773ba"
|
||||
},
|
||||
"source": [
|
||||
"* **Local JupyterLab instance**\n",
|
||||
"<br>Uncomment and run the following code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "254614fa0c46"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ef21552ccea8"
|
||||
},
|
||||
"source": [
|
||||
"* **Colab**\n",
|
||||
"<br>Uncomment and run the following code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "603adbbf0532"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "960505627ddf"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PyQmSRbKA8r-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk,all"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize the Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KllitKlIu-GW"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WjWD61gONRkw"
|
||||
},
|
||||
"source": [
|
||||
"### Load TensorBoard and TensorFlow components\n",
|
||||
"\n",
|
||||
"Load the TensorBoard notebook extension and import TensorFlow and the TensorBoard HParams plugin.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KSayPNqxfJC_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the TensorBoard notebook extension\n",
|
||||
"%load_ext tensorboard\n",
|
||||
"\n",
|
||||
"# Clear any logs from previous runs\n",
|
||||
"!rm -rf ./logs/\n",
|
||||
"\n",
|
||||
"# Import TensorFlow and the TensorBoard HParams plugin\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorboard.plugins.hparams import api as hp"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KJ4zE7rYfcvb"
|
||||
},
|
||||
"source": [
|
||||
"### Download dataset\n",
|
||||
"\n",
|
||||
"Download the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset and scale it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "vHME9wnnfiMr"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"fashion_mnist = tf.keras.datasets.fashion_mnist\n",
|
||||
"\n",
|
||||
"(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
|
||||
"x_train, x_test = x_train / 255.0, x_test / 255.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ofGSMru5r4kP"
|
||||
},
|
||||
"source": [
|
||||
"## Set up the experiment\n",
|
||||
"\n",
|
||||
"Run an experiment by specifying values for the following hyperparameters:\n",
|
||||
"\n",
|
||||
"* Number of units in the first dense layer\n",
|
||||
"* Dropout rate in the dropout layer\n",
|
||||
"* Optimizer\n",
|
||||
"\n",
|
||||
"Specify the hyperparameter values for the experiment in TensorBoard.\n",
|
||||
"\n",
|
||||
"*Optional*: For more fine grained filtering of hyperparameters in the UI, provide domain information and specify which metrics should be displayed."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "IG5sPLBAcDRy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"HP_NUM_UNITS = hp.HParam(\"num_units\", hp.Discrete([16, 32]))\n",
|
||||
"HP_DROPOUT = hp.HParam(\"dropout\", hp.RealInterval(0.1, 0.2))\n",
|
||||
"HP_OPTIMIZER = hp.HParam(\"optimizer\", hp.Discrete([\"adam\", \"sgd\"]))\n",
|
||||
"\n",
|
||||
"METRIC_ACCURACY = \"accuracy\"\n",
|
||||
"\n",
|
||||
"with tf.summary.create_file_writer(\"logs/hparam_tuning\").as_default():\n",
|
||||
" hp.hparams_config(\n",
|
||||
" hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],\n",
|
||||
" metrics=[hp.Metric(METRIC_ACCURACY, display_name=\"Accuracy\")],\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cLNgBNA6srlk"
|
||||
},
|
||||
"source": [
|
||||
"## Adapt TensorFlow runs to log hyperparameters and metrics\n",
|
||||
"\n",
|
||||
"The model will be quite simple: two dense layers with a dropout layer between them. The training code will look familiar, although the hyperparameters are no longer hardcoded. Instead, the hyperparameters are provided in an `hparams` dictionary and used throughout the training function:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "C-RSsrF4u-Fq"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_test_model(hparams):\n",
|
||||
" model = tf.keras.models.Sequential(\n",
|
||||
" [\n",
|
||||
" tf.keras.layers.Flatten(),\n",
|
||||
" tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),\n",
|
||||
" tf.keras.layers.Dropout(hparams[HP_DROPOUT]),\n",
|
||||
" tf.keras.layers.Dense(10, activation=tf.nn.softmax),\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" model.compile(\n",
|
||||
" optimizer=hparams[HP_OPTIMIZER],\n",
|
||||
" loss=\"sparse_categorical_crossentropy\",\n",
|
||||
" metrics=[\"accuracy\"],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" model.fit(\n",
|
||||
" x_train, y_train, epochs=1\n",
|
||||
" ) # Run with 1 epoch to speed things up for demo purposes\n",
|
||||
" _, accuracy = model.evaluate(x_test, y_test)\n",
|
||||
" return accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Esz3uqqCvLoK"
|
||||
},
|
||||
"source": [
|
||||
"For each run, log an hparams summary with the hyperparameters and final accuracy:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HwR1PAv1vPER"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def run(run_dir, hparams):\n",
|
||||
" with tf.summary.create_file_writer(run_dir).as_default():\n",
|
||||
" hp.hparams(hparams) # record the values used in this trial\n",
|
||||
" accuracy = train_test_model(hparams)\n",
|
||||
" tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0V_8soFFvU7b"
|
||||
},
|
||||
"source": [
|
||||
"## Start runs and log them all under one parent directory\n",
|
||||
"\n",
|
||||
"You can now try multiple experiments, training each one with a different set of hyperparameters.\n",
|
||||
"\n",
|
||||
"For simplicity, use a grid search: try all combinations of the discrete parameters and just the lower and upper bounds of the real-valued parameter. For more complex scenarios, it might be more effective to choose each hyperparameter value randomly (this is called a random search). There are more advanced methods that can be used.\n",
|
||||
"\n",
|
||||
"Run a few experiments, which will take a few minutes:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6r2oO_PVvbdL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"session_num = 0\n",
|
||||
"\n",
|
||||
"for num_units in HP_NUM_UNITS.domain.values:\n",
|
||||
" for dropout_rate in (HP_DROPOUT.domain.min_value, HP_DROPOUT.domain.max_value):\n",
|
||||
" for optimizer in HP_OPTIMIZER.domain.values:\n",
|
||||
" hparams = {\n",
|
||||
" HP_NUM_UNITS: num_units,\n",
|
||||
" HP_DROPOUT: dropout_rate,\n",
|
||||
" HP_OPTIMIZER: optimizer,\n",
|
||||
" }\n",
|
||||
" run_name = \"run-%d\" % session_num\n",
|
||||
" print(\"--- Starting trial: %s\" % run_name)\n",
|
||||
" print({h.name: hparams[h] for h in hparams})\n",
|
||||
" run(\"logs/hparam_tuning/\" + run_name, hparams)\n",
|
||||
" session_num += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6FJJwCclvslF"
|
||||
},
|
||||
"source": [
|
||||
"## Visualize the results in Vertex AI TensorBoard's HParams tab"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BkbB5GEI3Ge3"
|
||||
},
|
||||
"source": [
|
||||
"### Create Vertex AI Tensorboard\n",
|
||||
"A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. [documentation instructions](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).\n",
|
||||
"\n",
|
||||
"Create a TensorBoard instance to be used by the training job."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lQ-d3j-I3ZWV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"TENSORBOARD_NAME = \"[your-tensorboard-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" TENSORBOARD_NAME == \"\"\n",
|
||||
" or TENSORBOARD_NAME is None\n",
|
||||
" or TENSORBOARD_NAME == \"[your-tensorboard-name]\"\n",
|
||||
"):\n",
|
||||
" TENSORBOARD_NAME = PROJECT_ID + \"-tb-\"\n",
|
||||
"\n",
|
||||
"tensorboard = aiplatform.Tensorboard.create(\n",
|
||||
" display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
|
||||
")\n",
|
||||
"TENSORBOARD_RESOURCE_NAME = tensorboard.gca_resource.name\n",
|
||||
"print(\"TensorBoard resource name:\", TENSORBOARD_RESOURCE_NAME)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27rERDqeJ2nE"
|
||||
},
|
||||
"source": [
|
||||
"Set your TensorBoard Experiment name."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4OU4TMtFCn0_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from datetime import datetime\n",
|
||||
"\n",
|
||||
"EXPERIMENT_NAME = \"[your-experiment-run-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" EXPERIMENT_NAME == \"\"\n",
|
||||
" or EXPERIMENT_NAME is None\n",
|
||||
" or EXPERIMENT_NAME == \"[your-experiment-run-name]\"\n",
|
||||
"):\n",
|
||||
" EXPERIMENT_NAME = \"experiment\" + datetime.now().strftime(\"%H-%M-%S\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f1D2oU3K8Ys0"
|
||||
},
|
||||
"source": [
|
||||
"Upload the log to your Vertex AI TensorBoard"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "TyXFVQuRv0-X"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!tb-gcp-uploader --one_shot=True --tensorboard_resource_name=$TENSORBOARD_RESOURCE_NAME --logdir=\"logs/hparam_tuning/\" --experiment_name=$EXPERIMENT_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "OFe3qRyh9Wjl"
|
||||
},
|
||||
"source": [
|
||||
"Click the generated TensorBoard link and click on \"HParams\" at the top.\n",
|
||||
"\n",
|
||||
"The left pane of the dashboard provides filtering capabilities that are active across all the views in the HParams dashboard:\n",
|
||||
"\n",
|
||||
"- Filter which hyperparameters/metrics are shown in the dashboard\n",
|
||||
"- Filter which hyperparameter/metrics values are shown in the dashboard\n",
|
||||
"- Filter on run status (running, success, ...)\n",
|
||||
"- Sort by hyperparameter/metric in the table view\n",
|
||||
"- Number of session groups to show (useful for performance when there are many experiments)\n",
|
||||
"\n",
|
||||
"The HParams dashboard has three different views, with various useful information:\n",
|
||||
"\n",
|
||||
"* The **Table View** lists the runs, their hyperparameters, and their metrics.\n",
|
||||
"* The **Parallel Coordinates View** shows each run as a line going through an axis for each hyperparemeter and metric. Click and drag the mouse on any axis to mark a region which will highlight only the runs that pass through it. This can be useful for identifying which groups of hyperparameters are most important. The axes themselves can be re-ordered by dragging them.\n",
|
||||
"* The **Scatter Plot View** shows plots comparing each hyperparameter/metric with each metric. This can help identify correlations. Click and drag to select a region in a specific plot and highlight those sessions across the other plots.\n",
|
||||
"\n",
|
||||
"A table row, a parallel coordinates line, and a scatter plot market can be clicked to see a plot of the metrics as a function of training steps for that session (although in this tutorial only one step is used for each run)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TpV-iwP9qw9c"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sx_vKniMq9ZX"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# Delete endpoint resource\n",
|
||||
"# e.g. `endpoint.delete()`\n",
|
||||
"\n",
|
||||
"# Delete model resource\n",
|
||||
"# e.g. `model.delete()`\n",
|
||||
"\n",
|
||||
"# Delete Cloud Storage objects that were created\n",
|
||||
"delete_bucket = False\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil -m rm -r $BUCKET_URI"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "tensorboard_hyperparameter_tuning_with_hparams.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -1,807 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2022 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI TensorBoard Hyperparameter Tuning\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "24743cf4a1e1"
|
||||
},
|
||||
"source": [
|
||||
"**_NOTE_**: This notebook has been tested in the following environment:\n",
|
||||
"\n",
|
||||
"* Python version = 3.8"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"### What is Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"[Open source TensorBoard](https://www.tensorflow.org/tensorboard/get_started)\n",
|
||||
"(TB) is a Google open source project for machine learning experiment\n",
|
||||
"visualization. Vertex AI TensorBoard is an enterprise-ready managed\n",
|
||||
"version of TensorBoard.\n",
|
||||
"\n",
|
||||
"Vertex AI TensorBoard provides various detailed visualizations, including:\n",
|
||||
"\n",
|
||||
"* Tracking and visualizing metrics, such as loss and accuracy over time.\n",
|
||||
"* Visualizing model computational graphs (ops and layers).\n",
|
||||
"* Viewing histograms of weights, biases, or other tensors as they change over time.\n",
|
||||
"* Projecting embeddings to a lower dimensional space.\n",
|
||||
"* Displaying image, text, and audio samples.\n",
|
||||
"\n",
|
||||
"In addition to the powerful visualizations from\n",
|
||||
"TensorBoard, Vertex AI TensorBoard provides the following benefits:\n",
|
||||
"\n",
|
||||
"* A persistent, shareable link to your experiment's dashboard.\n",
|
||||
"\n",
|
||||
"* A searchable list of all experiments in a project.\n",
|
||||
"\n",
|
||||
"* Tight integrations with Vertex AI services for model training.\n",
|
||||
"\n",
|
||||
"* Enterprise-grade security, privacy, and compliance.\n",
|
||||
"\n",
|
||||
"With Vertex AI TensorBoard, you can track, visualize, and compare\n",
|
||||
"ML experiments and share them with your team.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d975e698c9a4"
|
||||
},
|
||||
"source": [
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"In this tutorial, you will experiment how to adapt TensorFlow runs to log hyperparameters and metrics and subsequently visualize the results in TensorBoard's HParams dashboard.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Google Cloud ML services and resources:\n",
|
||||
"\n",
|
||||
"- Vertex AI Training\n",
|
||||
"- Vertex AI TensorBoard\n",
|
||||
"- Vertex AI Pipelines\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"\n",
|
||||
"* Setup a service account and Google Cloud Storage buckets.\n",
|
||||
"* Construct a KFP pipeline with your custom training code.\n",
|
||||
"* Compile and execute the KFP pipeline in Vertex AI Pipelines with Tensorboard enabled for near real time monitorning."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "08d289fa873f"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"Dataset used in this tutorial is the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aed92deeb4a0"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
|
||||
"and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1fD9UZaygyPG"
|
||||
},
|
||||
"source": [
|
||||
"### Set up your local development environment\n",
|
||||
"\n",
|
||||
"**If you are using Colab or Vertex AI Workbench**, your environment already meets all the requirements to run this notebook. You can skip this step.\n",
|
||||
"\n",
|
||||
"Otherwise, make sure your environment meets this notebook's requirements. You need the following:\n",
|
||||
"\n",
|
||||
"- The Cloud Storage SDK\n",
|
||||
"- Git\n",
|
||||
"- Python 3\n",
|
||||
"- virtualenv\n",
|
||||
"- Jupyter notebook running in a virtual environment with Python 3\n",
|
||||
"\n",
|
||||
"The Cloud Storage guide to [Setting up a Python development environment](https://cloud.google.com/python/setup) and the [Jupyter installation guide](https://jupyter.org/install) provide detailed instructions for meeting these requirements. The following steps provide a condensed set of instructions:\n",
|
||||
"\n",
|
||||
"1. [Install and initialize the SDK](https://cloud.google.com/sdk/docs/).\n",
|
||||
"\n",
|
||||
"2. [Install Python 3](https://cloud.google.com/python/setup#installing_python).\n",
|
||||
"\n",
|
||||
"3. [Install virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv) and create a virtual environment that uses Python 3. Activate the virtual environment.\n",
|
||||
"\n",
|
||||
"4. To install Jupyter, run `pip3 install jupyter` on the command-line in a terminal shell.\n",
|
||||
"\n",
|
||||
"5. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n",
|
||||
"\n",
|
||||
"6. Open this notebook in the Jupyter Notebook Dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "i7EUnXsZhAGF"
|
||||
},
|
||||
"source": [
|
||||
"## Installation\n",
|
||||
"\n",
|
||||
"Install the following packages required to execute this notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "th7tWguZiSN2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# The Vertex AI Workbench Notebook product has specific requirements\n",
|
||||
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
|
||||
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
|
||||
" \"/opt/deeplearning/metadata/env_version\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
|
||||
"USER_FLAG = \"\"\n",
|
||||
"if IS_WORKBENCH_NOTEBOOK:\n",
|
||||
" USER_FLAG = \"--user\"\n",
|
||||
"\n",
|
||||
"! pip3 install {USER_FLAG} --force-reinstall google-cloud-aiplatform[tensorboard] tensorflow==2.7 \"shapely<2\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "58707a750154"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "f200f10a1da3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BF1j6f9HApxa"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). {TODO: Update the APIs needed for your tutorial. Edit the API names, and update the link to append the API IDs, separating each one with a comma. For example, container.googleapis.com,cloudbuild.googleapis.com}\n",
|
||||
"\n",
|
||||
"4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WReHDGG5g0XY"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "oM1iC_MfAts1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"source": [
|
||||
"#### Region\n",
|
||||
"\n",
|
||||
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nsN5NJKSu-GU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sBCra4QMA2wR"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "74ccc9e52986"
|
||||
},
|
||||
"source": [
|
||||
"**1. Vertex AI Workbench**\n",
|
||||
"* Do nothing as you are already authenticated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "de775a3773ba"
|
||||
},
|
||||
"source": [
|
||||
"**2. Local JupyterLab instance, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "254614fa0c46"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ef21552ccea8"
|
||||
},
|
||||
"source": [
|
||||
"**3. Colab, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "603adbbf0532"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f6b2ccc891ed"
|
||||
},
|
||||
"source": [
|
||||
"**4. Service account or other**\n",
|
||||
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "960505627ddf"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PyQmSRbKA8r-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "goitVEQmnz2u"
|
||||
},
|
||||
"source": [
|
||||
"If you run into ImportError: cannot import name 'WKBWriter' from 'shapely.geos', try the following and then restart runtime:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk,all"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KllitKlIu-GW"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Yj41fZkkfE0b"
|
||||
},
|
||||
"source": [
|
||||
"Start by loading the TensorBoard notebook extension and importing TensorFlow and the TensorBoard HParams plugin:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KSayPNqxfJC_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the TensorBoard notebook extension\n",
|
||||
"%load_ext tensorboard\n",
|
||||
"\n",
|
||||
"# Clear any logs from previous runs\n",
|
||||
"!rm -rf ./logs/\n",
|
||||
"\n",
|
||||
"# Import TensorFlow and the TensorBoard HParams plugin\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorboard.plugins.hparams import api as hp"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KJ4zE7rYfcvb"
|
||||
},
|
||||
"source": [
|
||||
"Download the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset and scale it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "vHME9wnnfiMr"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"fashion_mnist = tf.keras.datasets.fashion_mnist\n",
|
||||
"\n",
|
||||
"(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
|
||||
"x_train, x_test = x_train / 255.0, x_test / 255.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ofGSMru5r4kP"
|
||||
},
|
||||
"source": [
|
||||
"## Experiment setup and the HParams experiment summary\n",
|
||||
"\n",
|
||||
"Experiment with three hyperparameters in the model:\n",
|
||||
"\n",
|
||||
"1. Number of units in the first dense layer\n",
|
||||
"2. Dropout rate in the dropout layer\n",
|
||||
"3. Optimizer\n",
|
||||
"\n",
|
||||
"List the values to try, and log an experiment configuration to TensorBoard. This step is optional: you can provide domain information to enable more precise filtering of hyperparameters in the UI, and you can specify which metrics should be displayed."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "IG5sPLBAcDRy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"HP_NUM_UNITS = hp.HParam(\"num_units\", hp.Discrete([16, 32]))\n",
|
||||
"HP_DROPOUT = hp.HParam(\"dropout\", hp.RealInterval(0.1, 0.2))\n",
|
||||
"HP_OPTIMIZER = hp.HParam(\"optimizer\", hp.Discrete([\"adam\", \"sgd\"]))\n",
|
||||
"\n",
|
||||
"METRIC_ACCURACY = \"accuracy\"\n",
|
||||
"\n",
|
||||
"with tf.summary.create_file_writer(\"logs/hparam_tuning\").as_default():\n",
|
||||
" hp.hparams_config(\n",
|
||||
" hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],\n",
|
||||
" metrics=[hp.Metric(METRIC_ACCURACY, display_name=\"Accuracy\")],\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cLNgBNA6srlk"
|
||||
},
|
||||
"source": [
|
||||
"## Adapt TensorFlow runs to log hyperparameters and metrics\n",
|
||||
"\n",
|
||||
"The model will be quite simple: two dense layers with a dropout layer between them. The training code will look familiar, although the hyperparameters are no longer hardcoded. Instead, the hyperparameters are provided in an `hparams` dictionary and used throughout the training function:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "C-RSsrF4u-Fq"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_test_model(hparams):\n",
|
||||
" model = tf.keras.models.Sequential(\n",
|
||||
" [\n",
|
||||
" tf.keras.layers.Flatten(),\n",
|
||||
" tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),\n",
|
||||
" tf.keras.layers.Dropout(hparams[HP_DROPOUT]),\n",
|
||||
" tf.keras.layers.Dense(10, activation=tf.nn.softmax),\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" model.compile(\n",
|
||||
" optimizer=hparams[HP_OPTIMIZER],\n",
|
||||
" loss=\"sparse_categorical_crossentropy\",\n",
|
||||
" metrics=[\"accuracy\"],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" model.fit(\n",
|
||||
" x_train, y_train, epochs=1\n",
|
||||
" ) # Run with 1 epoch to speed things up for demo purposes\n",
|
||||
" _, accuracy = model.evaluate(x_test, y_test)\n",
|
||||
" return accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Esz3uqqCvLoK"
|
||||
},
|
||||
"source": [
|
||||
"For each run, log an hparams summary with the hyperparameters and final accuracy:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HwR1PAv1vPER"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def run(run_dir, hparams):\n",
|
||||
" with tf.summary.create_file_writer(run_dir).as_default():\n",
|
||||
" hp.hparams(hparams) # record the values used in this trial\n",
|
||||
" accuracy = train_test_model(hparams)\n",
|
||||
" tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0V_8soFFvU7b"
|
||||
},
|
||||
"source": [
|
||||
"## Start runs and log them all under one parent directory\n",
|
||||
"\n",
|
||||
"You can now try multiple experiments, training each one with a different set of hyperparameters.\n",
|
||||
"\n",
|
||||
"For simplicity, use a grid search: try all combinations of the discrete parameters and just the lower and upper bounds of the real-valued parameter. For more complex scenarios, it might be more effective to choose each hyperparameter value randomly (this is called a random search). There are more advanced methods that can be used.\n",
|
||||
"\n",
|
||||
"Run a few experiments, which will take a few minutes:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6r2oO_PVvbdL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"session_num = 0\n",
|
||||
"\n",
|
||||
"for num_units in HP_NUM_UNITS.domain.values:\n",
|
||||
" for dropout_rate in (HP_DROPOUT.domain.min_value, HP_DROPOUT.domain.max_value):\n",
|
||||
" for optimizer in HP_OPTIMIZER.domain.values:\n",
|
||||
" hparams = {\n",
|
||||
" HP_NUM_UNITS: num_units,\n",
|
||||
" HP_DROPOUT: dropout_rate,\n",
|
||||
" HP_OPTIMIZER: optimizer,\n",
|
||||
" }\n",
|
||||
" run_name = \"run-%d\" % session_num\n",
|
||||
" print(\"--- Starting trial: %s\" % run_name)\n",
|
||||
" print({h.name: hparams[h] for h in hparams})\n",
|
||||
" run(\"logs/hparam_tuning/\" + run_name, hparams)\n",
|
||||
" session_num += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6FJJwCclvslF"
|
||||
},
|
||||
"source": [
|
||||
"## Visualize the results in Vertex AI TensorBoard's HParams tab"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BkbB5GEI3Ge3"
|
||||
},
|
||||
"source": [
|
||||
"### Create Vertex AI Tensorboard\n",
|
||||
"A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. [documentation instructions](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).\n",
|
||||
"\n",
|
||||
"Create a TensorBoard instance to be used by the training job."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lQ-d3j-I3ZWV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"TENSORBOARD_NAME = \"[your-tensorboard-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" TENSORBOARD_NAME == \"\"\n",
|
||||
" or TENSORBOARD_NAME is None\n",
|
||||
" or TENSORBOARD_NAME == \"[your-tensorboard-name]\"\n",
|
||||
"):\n",
|
||||
" TENSORBOARD_NAME = PROJECT_ID + \"-tb-\"\n",
|
||||
"\n",
|
||||
"tensorboard = aiplatform.Tensorboard.create(\n",
|
||||
" display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
|
||||
")\n",
|
||||
"TENSORBOARD_RESOURCE_NAME = tensorboard.gca_resource.name\n",
|
||||
"print(\"TensorBoard resource name:\", TENSORBOARD_RESOURCE_NAME)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27rERDqeJ2nE"
|
||||
},
|
||||
"source": [
|
||||
"Set your TensorBoard Experiment name."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4OU4TMtFCn0_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from datetime import datetime\n",
|
||||
"\n",
|
||||
"EXPERIMENT_NAME = \"[your-experiment-run-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" EXPERIMENT_NAME == \"\"\n",
|
||||
" or EXPERIMENT_NAME is None\n",
|
||||
" or EXPERIMENT_NAME == \"[your-experiment-run-name]\"\n",
|
||||
"):\n",
|
||||
" EXPERIMENT_NAME = \"experiment\" + datetime.now().strftime(\"%H-%M-%S\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f1D2oU3K8Ys0"
|
||||
},
|
||||
"source": [
|
||||
"Upload the log to your Vertex AI TensorBoard"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "TyXFVQuRv0-X"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!tb-gcp-uploader --one_shot=True --tensorboard_resource_name=$TENSORBOARD_RESOURCE_NAME --logdir=\"logs/hparam_tuning/\" --experiment_name=$EXPERIMENT_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "OFe3qRyh9Wjl"
|
||||
},
|
||||
"source": [
|
||||
"Click the generated TensorBoard link and click on \"HParams\" at the top.\n",
|
||||
"\n",
|
||||
"The left pane of the dashboard provides filtering capabilities that are active across all the views in the HParams dashboard:\n",
|
||||
"\n",
|
||||
"- Filter which hyperparameters/metrics are shown in the dashboard\n",
|
||||
"- Filter which hyperparameter/metrics values are shown in the dashboard\n",
|
||||
"- Filter on run status (running, success, ...)\n",
|
||||
"- Sort by hyperparameter/metric in the table view\n",
|
||||
"- Number of session groups to show (useful for performance when there are many experiments)\n",
|
||||
"\n",
|
||||
"The HParams dashboard has three different views, with various useful information:\n",
|
||||
"\n",
|
||||
"* The **Table View** lists the runs, their hyperparameters, and their metrics.\n",
|
||||
"* The **Parallel Coordinates View** shows each run as a line going through an axis for each hyperparemeter and metric. Click and drag the mouse on any axis to mark a region which will highlight only the runs that pass through it. This can be useful for identifying which groups of hyperparameters are most important. The axes themselves can be re-ordered by dragging them.\n",
|
||||
"* The **Scatter Plot View** shows plots comparing each hyperparameter/metric with each metric. This can help identify correlations. Click and drag to select a region in a specific plot and highlight those sessions across the other plots.\n",
|
||||
"\n",
|
||||
"A table row, a parallel coordinates line, and a scatter plot market can be clicked to see a plot of the metrics as a function of training steps for that session (although in this tutorial only one step is used for each run)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TpV-iwP9qw9c"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sx_vKniMq9ZX"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# Delete endpoint resource\n",
|
||||
"# e.g. `endpoint.delete()`\n",
|
||||
"\n",
|
||||
"# Delete model resource\n",
|
||||
"# e.g. `model.delete()`\n",
|
||||
"\n",
|
||||
"# Delete Cloud Storage objects that were created\n",
|
||||
"delete_bucket = False\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil -m rm -r $BUCKET_URI"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "tensorboard_vertex_ai_hyperparameter_tuning.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -1,22 +1,4 @@
|
||||
|
||||
[Get started with Vertex AI Distributed Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/get_started_with_vertex_distributed_training.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Distributed Training` for when training with `Vertex AI`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- `MirroredStrategy`: Train on a single VM with multiple GPUs.
|
||||
- `MultiWorkerMirroredStrategy`: Train on multiple VMs with automatic setup of replicas.
|
||||
- `MultiWorkerMirroredStrategy`: Train on multiple VMs with fine grain control of replicas.
|
||||
- `ReductionServer`: Train on multiple VMS and sync updates across VMS with `Vertex AI Reduction Server`.
|
||||
- `TPUTraining`: Train with multiple Cloud TPUs.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Distributed Training](https://cloud.google.com/vertex-ai/docs/training/distributed-training).
|
||||
|
||||
|
||||
[Run hyperparameter tuning for a TensorFlow model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/hyperparameter_tuning_tensorflow.ipynb)
|
||||
|
||||
```
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -83,7 +83,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery](https://cloud.google.com/bigquery).
|
||||
|
||||
|
||||
[Build a fraud detection model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/fraud_detection/fraud-detection-model.ipynb)
|
||||
@@ -127,7 +127,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex).
|
||||
|
||||
|
||||
[Inventory prediction on ecommerce data using Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/inventory-prediction/inventory_prediction.ipynb)
|
||||
@@ -194,7 +194,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex).
|
||||
|
||||
|
||||
[Sentiment Analysis using AutoML Natural Language and Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/sentiment_analysis/Sentiment_Analysis.ipynb)
|
||||
|
||||
+1
-1
@@ -63,7 +63,7 @@
|
||||
"\n",
|
||||
"This notebook is written for data analysts and data scientists who have data in BigQuery and want to perform exploratory data analysis to gather insights from that data in an interactive environment.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery](https://cloud.google.com/bigquery)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
+1
-1
@@ -92,7 +92,7 @@
|
||||
"\n",
|
||||
"This tutorial shows you how to train, evaluate a propensity model in BigQuery ML to predict user retention on a mobile game, based on app measurement data from Google Analytics 4.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -87,7 +87,7 @@
|
||||
"\n",
|
||||
"*Note: This notebook file was developed to run in a [Vertex AI Workbench managed notebooks](https://console.cloud.google.com/vertex-ai/workbench/list/managed) instance using the Python (Local) kernel. Some components of this notebook may not work in other notebook environments.*\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
|
||||
"Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user