mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Andrew Ferlitsch
|
98277d0497 | ||
|
Andrew Ferlitsch
|
cb5dbbb055 | ||
|
|
fd16e39f91 | ||
|
Andrew FerlitschandGitHub
|
1551ca9435 | ||
|
Andrew FerlitschandGitHub
|
80fcd2904f |
@@ -61,7 +61,6 @@
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"This tutorial demonstrates how to use Vertex AI in production. This tutorial covers get started with BigQuery ML training.\n",
|
||||
"\n",
|
||||
"Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
|
||||
@@ -155,7 +154,8 @@
|
||||
"! pip3 install --upgrade pyarrow \\\n",
|
||||
" google-cloud-aiplatform \\\n",
|
||||
" google-cloud-bigquery \\\n",
|
||||
" google-cloud-bigquery-storage $USER_FLAG -q"
|
||||
" google-cloud-bigquery-storage \\\n",
|
||||
" db-dtypes $USER_FLAG -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -192,8 +192,7 @@
|
||||
"if IS_WORKBENCH_NOTEBOOK:\n",
|
||||
" USER_FLAG = \"--user\"\n",
|
||||
"\n",
|
||||
"! pip3 install --upgrade google-cloud-aiplatform {USER_FLAG} -q\n",
|
||||
"! pip3 install {USER_FLAG} --upgrade gsutil -q"
|
||||
"! pip3 install --upgrade google-cloud-aiplatform {USER_FLAG} -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -504,6 +503,55 @@
|
||||
"! gsutil ls -al $BUCKET_URI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "set_service_account"
|
||||
},
|
||||
"source": [
|
||||
"#### Service Account\n",
|
||||
"\n",
|
||||
"**If you don't know your service account**, try to get your service account using `gcloud` command by executing the second cell below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "set_service_account"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"SERVICE_ACCOUNT = \"[your-service-account]\" # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "autoset_service_account"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if (\n",
|
||||
" SERVICE_ACCOUNT == \"\"\n",
|
||||
" or SERVICE_ACCOUNT is None\n",
|
||||
" or SERVICE_ACCOUNT == \"[your-service-account]\"\n",
|
||||
"):\n",
|
||||
" # Get your service account from gcloud\n",
|
||||
" if not IS_COLAB:\n",
|
||||
" shell_output = !gcloud auth list 2>/dev/null\n",
|
||||
" SERVICE_ACCOUNT = shell_output[2].replace(\"*\", \"\").strip()\n",
|
||||
"\n",
|
||||
" if IS_COLAB:\n",
|
||||
" shell_output = ! gcloud projects describe $PROJECT_ID\n",
|
||||
" # print(\"shell_output=\", shell_output)\n",
|
||||
" project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n",
|
||||
" SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n",
|
||||
"\n",
|
||||
" print(\"Service Account:\", SERVICE_ACCOUNT)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -683,7 +731,7 @@
|
||||
"source": [
|
||||
"## Introduction to `Vertex AI Experiments`\n",
|
||||
"\n",
|
||||
"With `Vertex AI Experiments` you can log and track the following when experimenting/devloping your model architecture and model training:\n",
|
||||
"With `Vertex AI Experiments` you can log and track the following when experimenting/developing your model architecture and model training:\n",
|
||||
"\n",
|
||||
"- Log the metaparameters for the model architecture.\n",
|
||||
"- Log the hyperparameters for training.\n",
|
||||
@@ -698,9 +746,9 @@
|
||||
"- Cloud development in `Vertex AI Training`\n",
|
||||
"- Operationalizing development in `Vertex AI Pipelines`\n",
|
||||
"\n",
|
||||
"Learn more about [Experiments](https://cloud.google.com/vertex-ai/docs/experiments/).\n",
|
||||
"Learn more about [Experiments]( https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments).\n",
|
||||
"\n",
|
||||
"Learn more about [Introduction to Vertex AI ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata/introduction)."
|
||||
"Learn more about [Introduction to Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -717,7 +765,7 @@
|
||||
"- Instantiate a run per training run in the experiment.\n",
|
||||
"- Within the local training run, log the corresponding parameters and results.\n",
|
||||
"- Create lineage to the artifacts and experiment data.\n",
|
||||
"- Retreive the experiment data."
|
||||
"- Retrieve the experiment data."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -981,7 +1029,7 @@
|
||||
"source": [
|
||||
"#### Create a dataset and model artifacts\n",
|
||||
"\n",
|
||||
"Next, you create synthetic artifacts in the Vertex AI ML Metadata to associated with this run in the experiment, as lineage. You will create:\n",
|
||||
"Next, you create synthetic artifacts in the Vertex ML Metadata to associated with this run in the experiment, as lineage. You will create:\n",
|
||||
"\n",
|
||||
"- `dataset_artifact`: A dataset that is the input to the experiment run.\n",
|
||||
"- `model_artifact`: A model that is the output from the experiment run."
|
||||
@@ -1438,7 +1486,13 @@
|
||||
" \"--run=run-1\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"job.run(args=CMDARGS, replica_count=1, machine_type=TRAIN_COMPUTE, sync=True)"
|
||||
"job.run(\n",
|
||||
" args=CMDARGS,\n",
|
||||
" replica_count=1,\n",
|
||||
" machine_type=TRAIN_COMPUTE,\n",
|
||||
" service_account=SERVICE_ACCOUNT,\n",
|
||||
" sync=True,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -0,0 +1,794 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2022 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI TensorBoard Hyperparameter Tuning with the HParams Dashboard\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "24743cf4a1e1"
|
||||
},
|
||||
"source": [
|
||||
"**_NOTE_**: This notebook has been tested in the following environments:\n",
|
||||
"\n",
|
||||
"* Python version = 3.8"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"### What is Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"[Open source TensorBoard](https://www.tensorflow.org/tensorboard/get_started)\n",
|
||||
"(TB) is a Google open source project for machine learning experiment\n",
|
||||
"visualization. Vertex AI TensorBoard is an enterprise-ready managed\n",
|
||||
"version of TensorBoard.\n",
|
||||
"\n",
|
||||
"Vertex AI TensorBoard provides various detailed visualizations, including the following:\n",
|
||||
"\n",
|
||||
"* Tracking and visualizing metrics, such as loss and accuracy over time.\n",
|
||||
"* Visualizing model computational graphs (ops and layers).\n",
|
||||
"* Viewing histograms of weights, biases, or other tensors as they change over time.\n",
|
||||
"* Projecting embeddings to a lower dimensional space.\n",
|
||||
"* Displaying image, text, and audio samples.\n",
|
||||
"\n",
|
||||
"In addition to the powerful visualizations from\n",
|
||||
"TensorBoard, Vertex AI TensorBoard provides the following benefits:\n",
|
||||
"\n",
|
||||
"* A persistent, shareable link to your experiment's dashboard.\n",
|
||||
"\n",
|
||||
"* A searchable list of all experiments in a project.\n",
|
||||
"\n",
|
||||
"* Integrations with Vertex AI services for model training.\n",
|
||||
"\n",
|
||||
"* Enterprise-grade security, privacy, and compliance.\n",
|
||||
"\n",
|
||||
"With Vertex AI TensorBoard, you can track, visualize, and compare\n",
|
||||
"ML experiments and share them with your team.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d975e698c9a4"
|
||||
},
|
||||
"source": [
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"This tutorial shows you how to log hyperparameter experiment results in TensorFlow and visualize the results in TensorBoard's Hparams dashboard.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Vertex AI services and resources:\n",
|
||||
"\n",
|
||||
"- Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"\n",
|
||||
"* Adapt TensorFlow runs to log hyperparameters and metrics.\n",
|
||||
"* Start runs and log them all under one parent directory.\n",
|
||||
"* Visualize the results in TensorBoard's HParams dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "08d289fa873f"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"This tutorial uses the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aed92deeb4a0"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses the following billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
|
||||
"and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1fD9UZaygyPG"
|
||||
},
|
||||
"source": [
|
||||
"## Set up your local development environment\n",
|
||||
"\n",
|
||||
"**If you are using Colab or Vertex AI Workbench**, your environment already meets all the requirements to run this notebook. You can skip this step.\n",
|
||||
"\n",
|
||||
"Otherwise, make sure your environment meets this notebook's requirements. You need the following:\n",
|
||||
"\n",
|
||||
"- Git\n",
|
||||
"- Python 3\n",
|
||||
"- virtualenv\n",
|
||||
"- Jupyter notebook running in a virtual environment with Python 3\n",
|
||||
"\n",
|
||||
"To quickly set up your environment to meet the requirements of this tutorial, perform the following:\n",
|
||||
"\n",
|
||||
"1. [Install and initialize the SDK](https://cloud.google.com/sdk/docs/).\n",
|
||||
"\n",
|
||||
"2. [Install Python 3](https://cloud.google.com/python/setup#installing_python).\n",
|
||||
"\n",
|
||||
"3. [Install virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv) and create a virtual environment that uses Python 3 and activate the virtual environment.\n",
|
||||
"\n",
|
||||
"4. Install Jupyter by running the following command in a terminal shell:\n",
|
||||
"<br> `pip3 install jupyter`\n",
|
||||
"\n",
|
||||
"5. Launch Jupyter by running the following command in a terminal shell: <br> `jupyter notebook`\n",
|
||||
"\n",
|
||||
"6. Open this tutorial notebook in the Jupyter Notebook Dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "i7EUnXsZhAGF"
|
||||
},
|
||||
"source": [
|
||||
"## Install dependencies\n",
|
||||
"\n",
|
||||
"Install the following packages required to run this tutorial notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "th7tWguZiSN2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# The Vertex AI Workbench Notebook product has specific requirements\n",
|
||||
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
|
||||
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
|
||||
" \"/opt/deeplearning/metadata/env_version\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
|
||||
"USER_FLAG = \"\"\n",
|
||||
"if IS_WORKBENCH_NOTEBOOK:\n",
|
||||
" USER_FLAG = \"--user\"\n",
|
||||
"\n",
|
||||
"! pip3 install --upgrade google-cloud-aiplatform[tensorboard] tensorflow==2.7 {USER_FLAG} -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "58707a750154"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "f200f10a1da3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BF1j6f9HApxa"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"4. If you are running this notebook locally, install the [Cloud SDK](https://cloud.google.com/sdk)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WReHDGG5g0XY"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "oM1iC_MfAts1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"source": [
|
||||
"#### Set the region\n",
|
||||
"\n",
|
||||
"**Optional**: Update the 'REGION' variable to specify the region that you want to use. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nsN5NJKSu-GU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sBCra4QMA2wR"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"To authenticate your Google Cloud account, follow the instructions for your Jupyter environment:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "74ccc9e52986"
|
||||
},
|
||||
"source": [
|
||||
"* **Vertex AI Workbench**\n",
|
||||
"<br>You are already authenticated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "de775a3773ba"
|
||||
},
|
||||
"source": [
|
||||
"* **Local JupyterLab instance**\n",
|
||||
"<br>Uncomment and run the following code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "254614fa0c46"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ef21552ccea8"
|
||||
},
|
||||
"source": [
|
||||
"* **Colab**\n",
|
||||
"<br>Uncomment and run the following code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "603adbbf0532"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "960505627ddf"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PyQmSRbKA8r-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk,all"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize the Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KllitKlIu-GW"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WjWD61gONRkw"
|
||||
},
|
||||
"source": [
|
||||
"### Load TensorBoard and TensorFlow components\n",
|
||||
"\n",
|
||||
"Load the TensorBoard notebook extension and import TensorFlow and the TensorBoard HParams plugin.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KSayPNqxfJC_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the TensorBoard notebook extension\n",
|
||||
"%load_ext tensorboard\n",
|
||||
"\n",
|
||||
"# Clear any logs from previous runs\n",
|
||||
"!rm -rf ./logs/\n",
|
||||
"\n",
|
||||
"# Import TensorFlow and the TensorBoard HParams plugin\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorboard.plugins.hparams import api as hp"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KJ4zE7rYfcvb"
|
||||
},
|
||||
"source": [
|
||||
"### Download dataset\n",
|
||||
"\n",
|
||||
"Download the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset and scale it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "vHME9wnnfiMr"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"fashion_mnist = tf.keras.datasets.fashion_mnist\n",
|
||||
"\n",
|
||||
"(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
|
||||
"x_train, x_test = x_train / 255.0, x_test / 255.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ofGSMru5r4kP"
|
||||
},
|
||||
"source": [
|
||||
"## Set up the experiment\n",
|
||||
"\n",
|
||||
"Run an experiment by specifying values for the following hyperparameters:\n",
|
||||
"\n",
|
||||
"* Number of units in the first dense layer\n",
|
||||
"* Dropout rate in the dropout layer\n",
|
||||
"* Optimizer\n",
|
||||
"\n",
|
||||
"Specify the hyperparameter values for the experiment in TensorBoard.\n",
|
||||
"\n",
|
||||
"*Optional*: For more fine grained filtering of hyperparameters in the UI, provide domain information and specify which metrics should be displayed."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "IG5sPLBAcDRy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"HP_NUM_UNITS = hp.HParam(\"num_units\", hp.Discrete([16, 32]))\n",
|
||||
"HP_DROPOUT = hp.HParam(\"dropout\", hp.RealInterval(0.1, 0.2))\n",
|
||||
"HP_OPTIMIZER = hp.HParam(\"optimizer\", hp.Discrete([\"adam\", \"sgd\"]))\n",
|
||||
"\n",
|
||||
"METRIC_ACCURACY = \"accuracy\"\n",
|
||||
"\n",
|
||||
"with tf.summary.create_file_writer(\"logs/hparam_tuning\").as_default():\n",
|
||||
" hp.hparams_config(\n",
|
||||
" hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],\n",
|
||||
" metrics=[hp.Metric(METRIC_ACCURACY, display_name=\"Accuracy\")],\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cLNgBNA6srlk"
|
||||
},
|
||||
"source": [
|
||||
"## Adapt TensorFlow runs to log hyperparameters and metrics\n",
|
||||
"\n",
|
||||
"The model will be quite simple: two dense layers with a dropout layer between them. The training code will look familiar, although the hyperparameters are no longer hardcoded. Instead, the hyperparameters are provided in an `hparams` dictionary and used throughout the training function:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "C-RSsrF4u-Fq"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_test_model(hparams):\n",
|
||||
" model = tf.keras.models.Sequential(\n",
|
||||
" [\n",
|
||||
" tf.keras.layers.Flatten(),\n",
|
||||
" tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),\n",
|
||||
" tf.keras.layers.Dropout(hparams[HP_DROPOUT]),\n",
|
||||
" tf.keras.layers.Dense(10, activation=tf.nn.softmax),\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" model.compile(\n",
|
||||
" optimizer=hparams[HP_OPTIMIZER],\n",
|
||||
" loss=\"sparse_categorical_crossentropy\",\n",
|
||||
" metrics=[\"accuracy\"],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" model.fit(\n",
|
||||
" x_train, y_train, epochs=1\n",
|
||||
" ) # Run with 1 epoch to speed things up for demo purposes\n",
|
||||
" _, accuracy = model.evaluate(x_test, y_test)\n",
|
||||
" return accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Esz3uqqCvLoK"
|
||||
},
|
||||
"source": [
|
||||
"For each run, log an hparams summary with the hyperparameters and final accuracy:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HwR1PAv1vPER"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def run(run_dir, hparams):\n",
|
||||
" with tf.summary.create_file_writer(run_dir).as_default():\n",
|
||||
" hp.hparams(hparams) # record the values used in this trial\n",
|
||||
" accuracy = train_test_model(hparams)\n",
|
||||
" tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0V_8soFFvU7b"
|
||||
},
|
||||
"source": [
|
||||
"## Start runs and log them all under one parent directory\n",
|
||||
"\n",
|
||||
"You can now try multiple experiments, training each one with a different set of hyperparameters.\n",
|
||||
"\n",
|
||||
"For simplicity, use a grid search: try all combinations of the discrete parameters and just the lower and upper bounds of the real-valued parameter. For more complex scenarios, it might be more effective to choose each hyperparameter value randomly (this is called a random search). There are more advanced methods that can be used.\n",
|
||||
"\n",
|
||||
"Run a few experiments, which will take a few minutes:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6r2oO_PVvbdL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"session_num = 0\n",
|
||||
"\n",
|
||||
"for num_units in HP_NUM_UNITS.domain.values:\n",
|
||||
" for dropout_rate in (HP_DROPOUT.domain.min_value, HP_DROPOUT.domain.max_value):\n",
|
||||
" for optimizer in HP_OPTIMIZER.domain.values:\n",
|
||||
" hparams = {\n",
|
||||
" HP_NUM_UNITS: num_units,\n",
|
||||
" HP_DROPOUT: dropout_rate,\n",
|
||||
" HP_OPTIMIZER: optimizer,\n",
|
||||
" }\n",
|
||||
" run_name = \"run-%d\" % session_num\n",
|
||||
" print(\"--- Starting trial: %s\" % run_name)\n",
|
||||
" print({h.name: hparams[h] for h in hparams})\n",
|
||||
" run(\"logs/hparam_tuning/\" + run_name, hparams)\n",
|
||||
" session_num += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6FJJwCclvslF"
|
||||
},
|
||||
"source": [
|
||||
"## Visualize the results in Vertex AI TensorBoard's HParams tab"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BkbB5GEI3Ge3"
|
||||
},
|
||||
"source": [
|
||||
"### Create Vertex AI Tensorboard\n",
|
||||
"A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. [documentation instructions](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).\n",
|
||||
"\n",
|
||||
"Create a TensorBoard instance to be used by the training job."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lQ-d3j-I3ZWV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"TENSORBOARD_NAME = \"[your-tensorboard-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" TENSORBOARD_NAME == \"\"\n",
|
||||
" or TENSORBOARD_NAME is None\n",
|
||||
" or TENSORBOARD_NAME == \"[your-tensorboard-name]\"\n",
|
||||
"):\n",
|
||||
" TENSORBOARD_NAME = PROJECT_ID + \"-tb-\"\n",
|
||||
"\n",
|
||||
"tensorboard = aiplatform.Tensorboard.create(\n",
|
||||
" display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
|
||||
")\n",
|
||||
"TENSORBOARD_RESOURCE_NAME = tensorboard.gca_resource.name\n",
|
||||
"print(\"TensorBoard resource name:\", TENSORBOARD_RESOURCE_NAME)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27rERDqeJ2nE"
|
||||
},
|
||||
"source": [
|
||||
"Set your TensorBoard Experiment name."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4OU4TMtFCn0_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from datetime import datetime\n",
|
||||
"\n",
|
||||
"EXPERIMENT_NAME = \"[your-experiment-run-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" EXPERIMENT_NAME == \"\"\n",
|
||||
" or EXPERIMENT_NAME is None\n",
|
||||
" or EXPERIMENT_NAME == \"[your-experiment-run-name]\"\n",
|
||||
"):\n",
|
||||
" EXPERIMENT_NAME = \"experiment\" + datetime.now().strftime(\"%H-%M-%S\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f1D2oU3K8Ys0"
|
||||
},
|
||||
"source": [
|
||||
"Upload the log to your Vertex AI TensorBoard"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "TyXFVQuRv0-X"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!tb-gcp-uploader --one_shot=True --tensorboard_resource_name=$TENSORBOARD_RESOURCE_NAME --logdir=\"logs/hparam_tuning/\" --experiment_name=$EXPERIMENT_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "OFe3qRyh9Wjl"
|
||||
},
|
||||
"source": [
|
||||
"Click the generated TensorBoard link and click on \"HParams\" at the top.\n",
|
||||
"\n",
|
||||
"The left pane of the dashboard provides filtering capabilities that are active across all the views in the HParams dashboard:\n",
|
||||
"\n",
|
||||
"- Filter which hyperparameters/metrics are shown in the dashboard\n",
|
||||
"- Filter which hyperparameter/metrics values are shown in the dashboard\n",
|
||||
"- Filter on run status (running, success, ...)\n",
|
||||
"- Sort by hyperparameter/metric in the table view\n",
|
||||
"- Number of session groups to show (useful for performance when there are many experiments)\n",
|
||||
"\n",
|
||||
"The HParams dashboard has three different views, with various useful information:\n",
|
||||
"\n",
|
||||
"* The **Table View** lists the runs, their hyperparameters, and their metrics.\n",
|
||||
"* The **Parallel Coordinates View** shows each run as a line going through an axis for each hyperparemeter and metric. Click and drag the mouse on any axis to mark a region which will highlight only the runs that pass through it. This can be useful for identifying which groups of hyperparameters are most important. The axes themselves can be re-ordered by dragging them.\n",
|
||||
"* The **Scatter Plot View** shows plots comparing each hyperparameter/metric with each metric. This can help identify correlations. Click and drag to select a region in a specific plot and highlight those sessions across the other plots.\n",
|
||||
"\n",
|
||||
"A table row, a parallel coordinates line, and a scatter plot market can be clicked to see a plot of the metrics as a function of training steps for that session (although in this tutorial only one step is used for each run)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TpV-iwP9qw9c"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sx_vKniMq9ZX"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# Delete endpoint resource\n",
|
||||
"# e.g. `endpoint.delete()`\n",
|
||||
"\n",
|
||||
"# Delete model resource\n",
|
||||
"# e.g. `model.delete()`\n",
|
||||
"\n",
|
||||
"# Delete Cloud Storage objects that were created\n",
|
||||
"delete_bucket = False\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil -m rm -r $BUCKET_URI"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "tensorboard_hyperparameter_tuning_with_hparams.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -0,0 +1,807 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2022 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI TensorBoard Hyperparameter Tuning\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tensorboard/tensorboard_vertex_ai_hyperparameter_tuning.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "24743cf4a1e1"
|
||||
},
|
||||
"source": [
|
||||
"**_NOTE_**: This notebook has been tested in the following environment:\n",
|
||||
"\n",
|
||||
"* Python version = 3.8"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"### What is Vertex AI TensorBoard\n",
|
||||
"\n",
|
||||
"[Open source TensorBoard](https://www.tensorflow.org/tensorboard/get_started)\n",
|
||||
"(TB) is a Google open source project for machine learning experiment\n",
|
||||
"visualization. Vertex AI TensorBoard is an enterprise-ready managed\n",
|
||||
"version of TensorBoard.\n",
|
||||
"\n",
|
||||
"Vertex AI TensorBoard provides various detailed visualizations, including:\n",
|
||||
"\n",
|
||||
"* Tracking and visualizing metrics, such as loss and accuracy over time.\n",
|
||||
"* Visualizing model computational graphs (ops and layers).\n",
|
||||
"* Viewing histograms of weights, biases, or other tensors as they change over time.\n",
|
||||
"* Projecting embeddings to a lower dimensional space.\n",
|
||||
"* Displaying image, text, and audio samples.\n",
|
||||
"\n",
|
||||
"In addition to the powerful visualizations from\n",
|
||||
"TensorBoard, Vertex AI TensorBoard provides the following benefits:\n",
|
||||
"\n",
|
||||
"* A persistent, shareable link to your experiment's dashboard.\n",
|
||||
"\n",
|
||||
"* A searchable list of all experiments in a project.\n",
|
||||
"\n",
|
||||
"* Tight integrations with Vertex AI services for model training.\n",
|
||||
"\n",
|
||||
"* Enterprise-grade security, privacy, and compliance.\n",
|
||||
"\n",
|
||||
"With Vertex AI TensorBoard, you can track, visualize, and compare\n",
|
||||
"ML experiments and share them with your team.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d975e698c9a4"
|
||||
},
|
||||
"source": [
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"In this tutorial, you will experiment how to adapt TensorFlow runs to log hyperparameters and metrics and subsequently visualize the results in TensorBoard's HParams dashboard.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Google Cloud ML services and resources:\n",
|
||||
"\n",
|
||||
"- Vertex AI Training\n",
|
||||
"- Vertex AI TensorBoard\n",
|
||||
"- Vertex AI Pipelines\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"\n",
|
||||
"* Setup a service account and Google Cloud Storage buckets.\n",
|
||||
"* Construct a KFP pipeline with your custom training code.\n",
|
||||
"* Compile and execute the KFP pipeline in Vertex AI Pipelines with Tensorboard enabled for near real time monitorning."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "08d289fa873f"
|
||||
},
|
||||
"source": [
|
||||
"### Dataset\n",
|
||||
"\n",
|
||||
"Dataset used in this tutorial is the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aed92deeb4a0"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
|
||||
"and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1fD9UZaygyPG"
|
||||
},
|
||||
"source": [
|
||||
"### Set up your local development environment\n",
|
||||
"\n",
|
||||
"**If you are using Colab or Vertex AI Workbench**, your environment already meets all the requirements to run this notebook. You can skip this step.\n",
|
||||
"\n",
|
||||
"Otherwise, make sure your environment meets this notebook's requirements. You need the following:\n",
|
||||
"\n",
|
||||
"- The Cloud Storage SDK\n",
|
||||
"- Git\n",
|
||||
"- Python 3\n",
|
||||
"- virtualenv\n",
|
||||
"- Jupyter notebook running in a virtual environment with Python 3\n",
|
||||
"\n",
|
||||
"The Cloud Storage guide to [Setting up a Python development environment](https://cloud.google.com/python/setup) and the [Jupyter installation guide](https://jupyter.org/install) provide detailed instructions for meeting these requirements. The following steps provide a condensed set of instructions:\n",
|
||||
"\n",
|
||||
"1. [Install and initialize the SDK](https://cloud.google.com/sdk/docs/).\n",
|
||||
"\n",
|
||||
"2. [Install Python 3](https://cloud.google.com/python/setup#installing_python).\n",
|
||||
"\n",
|
||||
"3. [Install virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv) and create a virtual environment that uses Python 3. Activate the virtual environment.\n",
|
||||
"\n",
|
||||
"4. To install Jupyter, run `pip3 install jupyter` on the command-line in a terminal shell.\n",
|
||||
"\n",
|
||||
"5. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n",
|
||||
"\n",
|
||||
"6. Open this notebook in the Jupyter Notebook Dashboard."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "i7EUnXsZhAGF"
|
||||
},
|
||||
"source": [
|
||||
"## Installation\n",
|
||||
"\n",
|
||||
"Install the following packages required to execute this notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "th7tWguZiSN2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# The Vertex AI Workbench Notebook product has specific requirements\n",
|
||||
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
|
||||
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
|
||||
" \"/opt/deeplearning/metadata/env_version\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
|
||||
"USER_FLAG = \"\"\n",
|
||||
"if IS_WORKBENCH_NOTEBOOK:\n",
|
||||
" USER_FLAG = \"--user\"\n",
|
||||
"\n",
|
||||
"! pip3 install {USER_FLAG} --force-reinstall google-cloud-aiplatform[tensorboard] tensorflow==2.7 \"shapely<2\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "58707a750154"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "f200f10a1da3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BF1j6f9HApxa"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). {TODO: Update the APIs needed for your tutorial. Edit the API names, and update the link to append the API IDs, separating each one with a comma. For example, container.googleapis.com,cloudbuild.googleapis.com}\n",
|
||||
"\n",
|
||||
"4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WReHDGG5g0XY"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "oM1iC_MfAts1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "region"
|
||||
},
|
||||
"source": [
|
||||
"#### Region\n",
|
||||
"\n",
|
||||
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nsN5NJKSu-GU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sBCra4QMA2wR"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"\n",
|
||||
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "74ccc9e52986"
|
||||
},
|
||||
"source": [
|
||||
"**1. Vertex AI Workbench**\n",
|
||||
"* Do nothing as you are already authenticated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "de775a3773ba"
|
||||
},
|
||||
"source": [
|
||||
"**2. Local JupyterLab instance, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "254614fa0c46"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ef21552ccea8"
|
||||
},
|
||||
"source": [
|
||||
"**3. Colab, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "603adbbf0532"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f6b2ccc891ed"
|
||||
},
|
||||
"source": [
|
||||
"**4. Service account or other**\n",
|
||||
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "960505627ddf"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PyQmSRbKA8r-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "goitVEQmnz2u"
|
||||
},
|
||||
"source": [
|
||||
"If you run into ImportError: cannot import name 'WKBWriter' from 'shapely.geos', try the following and then restart runtime:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "init_aip:mbsdk,all"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KllitKlIu-GW"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Yj41fZkkfE0b"
|
||||
},
|
||||
"source": [
|
||||
"Start by loading the TensorBoard notebook extension and importing TensorFlow and the TensorBoard HParams plugin:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KSayPNqxfJC_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the TensorBoard notebook extension\n",
|
||||
"%load_ext tensorboard\n",
|
||||
"\n",
|
||||
"# Clear any logs from previous runs\n",
|
||||
"!rm -rf ./logs/\n",
|
||||
"\n",
|
||||
"# Import TensorFlow and the TensorBoard HParams plugin\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorboard.plugins.hparams import api as hp"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KJ4zE7rYfcvb"
|
||||
},
|
||||
"source": [
|
||||
"Download the [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset and scale it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "vHME9wnnfiMr"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"fashion_mnist = tf.keras.datasets.fashion_mnist\n",
|
||||
"\n",
|
||||
"(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
|
||||
"x_train, x_test = x_train / 255.0, x_test / 255.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ofGSMru5r4kP"
|
||||
},
|
||||
"source": [
|
||||
"## Experiment setup and the HParams experiment summary\n",
|
||||
"\n",
|
||||
"Experiment with three hyperparameters in the model:\n",
|
||||
"\n",
|
||||
"1. Number of units in the first dense layer\n",
|
||||
"2. Dropout rate in the dropout layer\n",
|
||||
"3. Optimizer\n",
|
||||
"\n",
|
||||
"List the values to try, and log an experiment configuration to TensorBoard. This step is optional: you can provide domain information to enable more precise filtering of hyperparameters in the UI, and you can specify which metrics should be displayed."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "IG5sPLBAcDRy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"HP_NUM_UNITS = hp.HParam(\"num_units\", hp.Discrete([16, 32]))\n",
|
||||
"HP_DROPOUT = hp.HParam(\"dropout\", hp.RealInterval(0.1, 0.2))\n",
|
||||
"HP_OPTIMIZER = hp.HParam(\"optimizer\", hp.Discrete([\"adam\", \"sgd\"]))\n",
|
||||
"\n",
|
||||
"METRIC_ACCURACY = \"accuracy\"\n",
|
||||
"\n",
|
||||
"with tf.summary.create_file_writer(\"logs/hparam_tuning\").as_default():\n",
|
||||
" hp.hparams_config(\n",
|
||||
" hparams=[HP_NUM_UNITS, HP_DROPOUT, HP_OPTIMIZER],\n",
|
||||
" metrics=[hp.Metric(METRIC_ACCURACY, display_name=\"Accuracy\")],\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cLNgBNA6srlk"
|
||||
},
|
||||
"source": [
|
||||
"## Adapt TensorFlow runs to log hyperparameters and metrics\n",
|
||||
"\n",
|
||||
"The model will be quite simple: two dense layers with a dropout layer between them. The training code will look familiar, although the hyperparameters are no longer hardcoded. Instead, the hyperparameters are provided in an `hparams` dictionary and used throughout the training function:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "C-RSsrF4u-Fq"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_test_model(hparams):\n",
|
||||
" model = tf.keras.models.Sequential(\n",
|
||||
" [\n",
|
||||
" tf.keras.layers.Flatten(),\n",
|
||||
" tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),\n",
|
||||
" tf.keras.layers.Dropout(hparams[HP_DROPOUT]),\n",
|
||||
" tf.keras.layers.Dense(10, activation=tf.nn.softmax),\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" model.compile(\n",
|
||||
" optimizer=hparams[HP_OPTIMIZER],\n",
|
||||
" loss=\"sparse_categorical_crossentropy\",\n",
|
||||
" metrics=[\"accuracy\"],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" model.fit(\n",
|
||||
" x_train, y_train, epochs=1\n",
|
||||
" ) # Run with 1 epoch to speed things up for demo purposes\n",
|
||||
" _, accuracy = model.evaluate(x_test, y_test)\n",
|
||||
" return accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Esz3uqqCvLoK"
|
||||
},
|
||||
"source": [
|
||||
"For each run, log an hparams summary with the hyperparameters and final accuracy:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HwR1PAv1vPER"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def run(run_dir, hparams):\n",
|
||||
" with tf.summary.create_file_writer(run_dir).as_default():\n",
|
||||
" hp.hparams(hparams) # record the values used in this trial\n",
|
||||
" accuracy = train_test_model(hparams)\n",
|
||||
" tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0V_8soFFvU7b"
|
||||
},
|
||||
"source": [
|
||||
"## Start runs and log them all under one parent directory\n",
|
||||
"\n",
|
||||
"You can now try multiple experiments, training each one with a different set of hyperparameters.\n",
|
||||
"\n",
|
||||
"For simplicity, use a grid search: try all combinations of the discrete parameters and just the lower and upper bounds of the real-valued parameter. For more complex scenarios, it might be more effective to choose each hyperparameter value randomly (this is called a random search). There are more advanced methods that can be used.\n",
|
||||
"\n",
|
||||
"Run a few experiments, which will take a few minutes:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6r2oO_PVvbdL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"session_num = 0\n",
|
||||
"\n",
|
||||
"for num_units in HP_NUM_UNITS.domain.values:\n",
|
||||
" for dropout_rate in (HP_DROPOUT.domain.min_value, HP_DROPOUT.domain.max_value):\n",
|
||||
" for optimizer in HP_OPTIMIZER.domain.values:\n",
|
||||
" hparams = {\n",
|
||||
" HP_NUM_UNITS: num_units,\n",
|
||||
" HP_DROPOUT: dropout_rate,\n",
|
||||
" HP_OPTIMIZER: optimizer,\n",
|
||||
" }\n",
|
||||
" run_name = \"run-%d\" % session_num\n",
|
||||
" print(\"--- Starting trial: %s\" % run_name)\n",
|
||||
" print({h.name: hparams[h] for h in hparams})\n",
|
||||
" run(\"logs/hparam_tuning/\" + run_name, hparams)\n",
|
||||
" session_num += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6FJJwCclvslF"
|
||||
},
|
||||
"source": [
|
||||
"## Visualize the results in Vertex AI TensorBoard's HParams tab"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BkbB5GEI3Ge3"
|
||||
},
|
||||
"source": [
|
||||
"### Create Vertex AI Tensorboard\n",
|
||||
"A Vertex AI TensorBoard instance, which is a regionalized resource storing your Vertex AI TensorBoard experiments, must be created before the experiments can be visualized. You can create multiple instances in a project. [documentation instructions](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).\n",
|
||||
"\n",
|
||||
"Create a TensorBoard instance to be used by the training job."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lQ-d3j-I3ZWV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"TENSORBOARD_NAME = \"[your-tensorboard-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" TENSORBOARD_NAME == \"\"\n",
|
||||
" or TENSORBOARD_NAME is None\n",
|
||||
" or TENSORBOARD_NAME == \"[your-tensorboard-name]\"\n",
|
||||
"):\n",
|
||||
" TENSORBOARD_NAME = PROJECT_ID + \"-tb-\"\n",
|
||||
"\n",
|
||||
"tensorboard = aiplatform.Tensorboard.create(\n",
|
||||
" display_name=TENSORBOARD_NAME, project=PROJECT_ID, location=REGION\n",
|
||||
")\n",
|
||||
"TENSORBOARD_RESOURCE_NAME = tensorboard.gca_resource.name\n",
|
||||
"print(\"TensorBoard resource name:\", TENSORBOARD_RESOURCE_NAME)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27rERDqeJ2nE"
|
||||
},
|
||||
"source": [
|
||||
"Set your TensorBoard Experiment name."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4OU4TMtFCn0_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from datetime import datetime\n",
|
||||
"\n",
|
||||
"EXPERIMENT_NAME = \"[your-experiment-run-name]\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"if (\n",
|
||||
" EXPERIMENT_NAME == \"\"\n",
|
||||
" or EXPERIMENT_NAME is None\n",
|
||||
" or EXPERIMENT_NAME == \"[your-experiment-run-name]\"\n",
|
||||
"):\n",
|
||||
" EXPERIMENT_NAME = \"experiment\" + datetime.now().strftime(\"%H-%M-%S\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f1D2oU3K8Ys0"
|
||||
},
|
||||
"source": [
|
||||
"Upload the log to your Vertex AI TensorBoard"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "TyXFVQuRv0-X"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!tb-gcp-uploader --one_shot=True --tensorboard_resource_name=$TENSORBOARD_RESOURCE_NAME --logdir=\"logs/hparam_tuning/\" --experiment_name=$EXPERIMENT_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "OFe3qRyh9Wjl"
|
||||
},
|
||||
"source": [
|
||||
"Click the generated TensorBoard link and click on \"HParams\" at the top.\n",
|
||||
"\n",
|
||||
"The left pane of the dashboard provides filtering capabilities that are active across all the views in the HParams dashboard:\n",
|
||||
"\n",
|
||||
"- Filter which hyperparameters/metrics are shown in the dashboard\n",
|
||||
"- Filter which hyperparameter/metrics values are shown in the dashboard\n",
|
||||
"- Filter on run status (running, success, ...)\n",
|
||||
"- Sort by hyperparameter/metric in the table view\n",
|
||||
"- Number of session groups to show (useful for performance when there are many experiments)\n",
|
||||
"\n",
|
||||
"The HParams dashboard has three different views, with various useful information:\n",
|
||||
"\n",
|
||||
"* The **Table View** lists the runs, their hyperparameters, and their metrics.\n",
|
||||
"* The **Parallel Coordinates View** shows each run as a line going through an axis for each hyperparemeter and metric. Click and drag the mouse on any axis to mark a region which will highlight only the runs that pass through it. This can be useful for identifying which groups of hyperparameters are most important. The axes themselves can be re-ordered by dragging them.\n",
|
||||
"* The **Scatter Plot View** shows plots comparing each hyperparameter/metric with each metric. This can help identify correlations. Click and drag to select a region in a specific plot and highlight those sessions across the other plots.\n",
|
||||
"\n",
|
||||
"A table row, a parallel coordinates line, and a scatter plot market can be clicked to see a plot of the metrics as a function of training steps for that session (although in this tutorial only one step is used for each run)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TpV-iwP9qw9c"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
|
||||
"project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sx_vKniMq9ZX"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# Delete endpoint resource\n",
|
||||
"# e.g. `endpoint.delete()`\n",
|
||||
"\n",
|
||||
"# Delete model resource\n",
|
||||
"# e.g. `model.delete()`\n",
|
||||
"\n",
|
||||
"# Delete Cloud Storage objects that were created\n",
|
||||
"delete_bucket = False\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
" ! gsutil -m rm -r $BUCKET_URI"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "tensorboard_vertex_ai_hyperparameter_tuning.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Reference in New Issue
Block a user