diff --git a/notebooks/community/managed_notebooks/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb b/notebooks/official/workbench/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb similarity index 68% rename from notebooks/community/managed_notebooks/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb rename to notebooks/official/workbench/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb index b1d4371d9..d132b5ef7 100644 --- a/notebooks/community/managed_notebooks/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb +++ b/notebooks/official/workbench/chicago_taxi_fare_prediction/chicago_taxi_fare_prediction.ipynb @@ -1,13 +1,71 @@ { "cells": [ { - "cell_type": "markdown", + "cell_type": "code", + "execution_count": null, "metadata": { "id": "c8c4e360024a" }, + "outputs": [], "source": [ - "# Taxi fare prediction using the Chicago Taxi Trips dataset\n", + "# Copyright 2022 Google LLC\n", + "#\n", + "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d6728c7e34d2" + }, + "source": [ + "\n", "\n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \"Colab Run in Colab\n", + " \n", + " \n", + " \n", + " \"GitHub\n", + " View on GitHub\n", + " \n", + " \n", + " \n", + " \"Vertex\n", + " Open in Vertex AI Workbench\n", + " \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "28451b7c3d4b" + }, + "source": [ + "# Taxi fare prediction using the Chicago Taxi Trips dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eaf8ba4eeea8" + }, + "source": [ "## Table of contents\n", "\n", "* [Overview](#section-1)\n", @@ -24,9 +82,9 @@ "## Overview\n", "\n", "\n", - "This notebook demonstrates analysis, feature selection, model building, and deployment with Vertex Explainable AI configured on Vertex AI, using a subset of the Chicago Taxi Trips dataset for taxi-fare prediction.\n", + "This notebook demonstrates analysis, feature selection, model building, and deployment with Explainable AI configured on Vertex AI, using a subset of the Chicago Taxi Trips dataset for taxi-fare prediction.\n", "\n", - "*Note: This notebook file was developed to run in a [Vertex AI Workbench managed notebooks](https://console.cloud.google.com/vertex-ai/workbench/list/managed) instance using the Python (Local) kernel. Some components of this notebook may not work in other notebook environments.*\n", + "*Note: This notebook is developed to run in a [Vertex AI Workbench managed notebooks](https://console.cloud.google.com/vertex-ai/workbench/list/managed) instance using the Python (Local) kernel. Some components of this notebook may not work in other notebook environments.*\n", "\n", "## Dataset\n", "\n", @@ -38,7 +96,7 @@ "## Objective\n", "\n", "\n", - "The goal of this notebook is to provide an overview on the latest Vertex AI features like Explainable AI and \"BigQuery in Notebooks\" by trying to solve a taxi fare prediction problem. The steps followed in this notebook include: \n", + "The goal of this notebook is to provide an overview on the latest Vertex AI features like **Explainable AI** and **BigQuery in Notebooks** by trying to solve a taxi fare prediction problem. The steps followed in this notebook include: \n", "\n", "- Loading the dataset using \"BigQuery in Notebooks\".\n", "- Performing exploratory data analysis on the dataset.\n", @@ -66,6 +124,131 @@ "to generate a cost estimate based on your projected usage." ] }, + { + "cell_type": "markdown", + "metadata": { + "id": "5178273783dd" + }, + "source": [ + "### Set up your local development environment\n", + "\n", + "**If you are using Colab or Google Cloud Notebooks**, your environment already meets\n", + "all the requirements to run this notebook. You can skip this step." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f5494c42606e" + }, + "source": [ + "**Otherwise**, make sure your environment meets this notebook's requirements.\n", + "You need the following:\n", + "\n", + "* The Google Cloud SDK\n", + "* Git\n", + "* Python 3\n", + "* virtualenv\n", + "* Jupyter notebook running in a virtual environment with Python 3\n", + "\n", + "The Google Cloud guide to [Setting up a Python development\n", + "environment](https://cloud.google.com/python/setup) and the [Jupyter\n", + "installation guide](https://jupyter.org/install) provide detailed instructions\n", + "for meeting these requirements. The following steps provide a condensed set of\n", + "instructions:\n", + "\n", + "1. [Install and initialize the Cloud SDK.](https://cloud.google.com/sdk/docs/)\n", + "\n", + "1. [Install Python 3.](https://cloud.google.com/python/setup#installing_python)\n", + "\n", + "1. [Install\n", + " virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv)\n", + " and create a virtual environment that uses Python 3. Activate the virtual environment.\n", + "\n", + "1. To install Jupyter, run `pip3 install jupyter` on the\n", + "command-line in a terminal shell.\n", + "\n", + "1. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n", + "\n", + "1. Open this notebook in the Jupyter Notebook Dashboard." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "23976b1be293" + }, + "source": [ + "### Install additional packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1fd00fa70a2a" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# The Google Cloud Notebook product has specific requirements\n", + "IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n", + "\n", + "# Google Cloud Notebook requires dependencies to be installed with '--user'\n", + "USER_FLAG = \"\"\n", + "if IS_GOOGLE_CLOUD_NOTEBOOK:\n", + " USER_FLAG = \"--user\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "a50fd443a6ce" + }, + "outputs": [], + "source": [ + "! pip install {USER_FLAG} --upgrade google-cloud-bigquery \\\n", + " google-cloud-aiplatform \\\n", + " google-cloud-storage \\\n", + " seaborn \\\n", + " sklearn \\\n", + " pandas \\\n", + " fsspec \\\n", + " pyarrow" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d3a26cb9b19d" + }, + "source": [ + "### Restart the kernel\n", + "\n", + "After you install the additional packages, you need to restart the notebook kernel so it can find the packages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "c1464805870e" + }, + "outputs": [], + "source": [ + "# Automatically restart kernel after installs\n", + "import os\n", + "\n", + "if not os.getenv(\"IS_TESTING\"):\n", + " # Automatically restart kernel after installs\n", + " import IPython\n", + "\n", + " app = IPython.Application.instance()\n", + " app.kernel.do_shutdown(True)" + ] + }, { "cell_type": "markdown", "metadata": { @@ -74,7 +257,31 @@ "source": [ "## Before you begin\n", "\n", - "### Set your project ID\n", + "### Set up your Google Cloud project\n", + "\n", + "**The following steps are required, regardless of your notebook environment.**\n", + "\n", + "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n", + "\n", + "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n", + "\n", + "1. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", + "\n", + "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n", + "\n", + "1. Enter your project ID in the cell below. Then run the cell to make sure the\n", + "Cloud SDK uses the right project for all the commands in this notebook.\n", + "\n", + "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5aee4379e8e5" + }, + "source": [ + "#### Set your project ID\n", "\n", "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`." ] @@ -119,17 +326,175 @@ " PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "30e64c0eda41" + }, + "outputs": [], + "source": [ + "! gcloud config set project $PROJECT_ID" + ] + }, { "cell_type": "markdown", "metadata": { - "id": "fed4b24ea061" + "id": "7d43ac19ea91" }, "source": [ - "## Select or create a Cloud Storage bucket for storing the model\n", + "### Region\n", "\n", - "When you create a model resource on Vertex AI using the Cloud SDK, you need to give a Cloud Storage bucket uri of the model where the model is stored. Using the model saved, you can then create a Vertex AI model and endpoint resources in order to serve online predictions.\n", + "You can also change the `REGION` variable, which is used for operations\n", + "throughout the rest of this notebook. Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n", "\n", - "Set the name of your Cloud Storage bucket below. It must be unique across all Cloud Storage buckets. You may also change the `LOCATION` variable, which is used for operations throughout the rest of this notebook. Make sure to choose a region where Vertex AI services are available." + "- Americas: `us-central1`\n", + "- Europe: `europe-west4`\n", + "- Asia Pacific: `asia-east1`\n", + "\n", + "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n", + "\n", + "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "3281bedf6d3c" + }, + "outputs": [], + "source": [ + "REGION = \"[your-region]\" # @param {type: \"string\"}\n", + "\n", + "if REGION == \"[your-region]\":\n", + " REGION = \"us-central1\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0c9906f72b18" + }, + "source": [ + "### UUID\n", + "\n", + "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8940c46120e6" + }, + "outputs": [], + "source": [ + "import random\n", + "import string\n", + "\n", + "\n", + "# Generate a uuid of length 8\n", + "def generate_uuid():\n", + " return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=8))\n", + "\n", + "\n", + "UUID = generate_uuid()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "648aa9824ac6" + }, + "source": [ + "### Authenticate your Google Cloud account\n", + "\n", + "**If you are using Google Cloud Notebooks**, your environment is already\n", + "authenticated. Skip this step." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fc52bba17ee3" + }, + "source": [ + "**If you are using Colab**, run the cell below and follow the instructions\n", + "when prompted to authenticate your account via oAuth.\n", + "\n", + "**Otherwise**, follow these steps:\n", + "\n", + "1. In the Cloud Console, go to the [**Create service account key**\n", + " page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n", + "\n", + "2. Click **Create service account**.\n", + "\n", + "3. In the **Service account name** field, enter a name, and\n", + " click **Create**.\n", + "\n", + "4. In the **Grant this service account access to project** section, click the **Role** drop-down list. Type \"Vertex AI\"\n", + "into the filter box, and select\n", + " **Vertex AI Administrator**. Type \"Storage Object Admin\" into the filter box, and select **Storage Object Admin**.\n", + "\n", + "5. Click *Create*. A JSON file that contains your key downloads to your\n", + "local environment.\n", + "\n", + "6. Enter the path to your service account key as the\n", + "`GOOGLE_APPLICATION_CREDENTIALS` variable in the cell below and run the cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "535223fa4b84" + }, + "outputs": [], + "source": [ + "import os\n", + "import sys\n", + "\n", + "# If you are running this notebook in Colab, run this cell and follow the\n", + "# instructions to authenticate your GCP account. This provides access to your\n", + "# Cloud Storage bucket and lets you submit training jobs and prediction\n", + "# requests.\n", + "\n", + "# The Google Cloud Notebook product has specific requirements\n", + "IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n", + "\n", + "# If on Google Cloud Notebooks, then don't execute this code\n", + "if not IS_GOOGLE_CLOUD_NOTEBOOK:\n", + " if \"google.colab\" in sys.modules:\n", + " from google.colab import auth as google_auth\n", + "\n", + " google_auth.authenticate_user()\n", + "\n", + " # If you are running this notebook locally, replace the string below with the\n", + " # path to your service account key and run this cell to authenticate your GCP\n", + " # account.\n", + " elif not os.getenv(\"IS_TESTING\"):\n", + " %env GOOGLE_APPLICATION_CREDENTIALS ''" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0474cb91d91f" + }, + "source": [ + "### Create a Cloud Storage bucket\n", + "\n", + "**The following steps are required, regardless of your notebook environment.**\n", + "\n", + "\n", + "When you create a model in Vertex AI using the Cloud SDK, you give a Cloud Storage path where the trained model is saved. \n", + "In this tutorial, Vertex AI saves the trained model to a Cloud Storage bucket. Using this model artifact, you can then\n", + "create Vertex AI model and endpoint resources in order to serve\n", + "online predictions.\n", + "\n", + "Set the name of your Cloud Storage bucket below. It must be unique across all\n", + "Cloud Storage buckets." ] }, { @@ -140,9 +505,8 @@ }, "outputs": [], "source": [ - "BUCKET_NAME = \"[your-bucket-name]\"\n", - "BUCKET_URI = f\"gs://{BUCKET_NAME}\"\n", - "LOCATION = \"us-central1\"" + "BUCKET_NAME = \"[your-bucket-name]\" # @param {type:\"string\"}\n", + "BUCKET_URI = f\"gs://{BUCKET_NAME}\"" ] }, { @@ -153,13 +517,8 @@ }, "outputs": [], "source": [ - "from datetime import datetime\n", - "\n", - "# Set a default bucket name in case bucket name is not given\n", - "if BUCKET_NAME == \"\" or BUCKET_NAME == \"[your-bucket-name]\" or BUCKET_NAME is None:\n", - "\n", - " TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n", - " BUCKET_NAME = PROJECT_ID + \"aip-\" + TIMESTAMP\n", + "if BUCKET_URI == \"\" or BUCKET_URI is None or BUCKET_URI == \"gs://[your-bucket-name]\":\n", + " BUCKET_NAME = PROJECT_ID + \"aip-\" + UUID\n", " BUCKET_URI = \"gs://\" + BUCKET_NAME" ] }, @@ -180,7 +539,7 @@ }, "outputs": [], "source": [ - "! gsutil mb -l $REGION $BUCKET_NAME" + "! gsutil mb -l $REGION $BUCKET_URI" ] }, { @@ -200,7 +559,7 @@ }, "outputs": [], "source": [ - "! gsutil ls -al $BUCKET_NAME" + "! gsutil ls -al $BUCKET_URI" ] }, { @@ -209,7 +568,7 @@ "id": "2e52fd6d4854" }, "source": [ - "## Import the required libraries and define constants" + "### Import libraries" ] }, { @@ -238,11 +597,11 @@ "id": "5166f42557ad" }, "source": [ - "The dataset is quite a large and noisy one, so data from a specific date range will be used. Based on various blogs and resources that are available online, many of them seem to have used the data from around May 2018 which gave some really good results compared to the other date ranges. While there are also some complicated research models proposed for the same problem, like considering the weather data, holidays and seasons, the current notebook only explores a simple linear regression model, as our main objective is to demonstrate the model deployment with Vertex Explainable AI configured on Vertex AI.\n", + "The original dataset considered for this tutorial is a large and noisy one and so data from a specific date range will be used. Based on various online resources, the data from around May 2018 gave some really good results compared to the other date ranges. While there are also some complicated models proposed for the same problem, like considering the weather data, holidays and seasons, the current notebook only explores a simple linear regression model. Our main objective is to demonstrate the model deployment with Vertex Explainable AI configured on Vertex AI.\n", "\n", - "## Accessing the data through \"BigQuery in Notebooks\"\n", + "## Accessing the data through BigQuery Integration\n", "\n", - "The \"BigQuery in Notebooks\" feature of Vertex AI Workbench managed notebooks lets you use BigQuery and its features from the notebook itself eliminating the need to switch between tabs everytime. For every cell in the notebook, there is an option for the BigQuery integration at the top right, and selecting it enables you to compose an SQL query that can be executed in BigQuery. \n", + "The **BigQuery Integration for Notebooks** feature of Vertex AI Workbench managed notebooks lets you use BigQuery and its features from the notebook itself eliminating the need to switch between tabs everytime. For every cell in the notebook, there is an option for the BigQuery integration at the top right, and selecting it enables you to compose an SQL query that can be executed in BigQuery. \n", "\n", "The chosen dataset consists of the following fields:\n", "\n", @@ -275,7 +634,9 @@ "- Time taken for the trip > 0.\n", "- Distance covered during the trip > 0.\n", "- Total trip charges > 0 and\n", - "- Pickup and dropoff areas are valid (not empty)." + "- Pickup and dropoff areas are valid (not empty).\n", + "\n", + "Note: The below cell is a Bigquery Integration cell and can only execute on Vertex AI Workbench's managed instances. If your notebook environment is different, you can skip it." ] }, { @@ -325,7 +686,7 @@ "# Comment out otherwise for speed-up.\n", "from google.cloud.bigquery import Client\n", "\n", - "client = Client()\n", + "client = Client(project=PROJECT_ID)\n", "\n", "query = \"\"\"select \n", "taxi_id, trip_start_timestamp, \n", @@ -337,7 +698,7 @@ "where \n", "trip_start_timestamp >= '2018-05-12' and \n", "trip_end_timestamp <= '2018-05-18' and\n", - "trip_seconds > 0 and trip_seconds < 6*60*60 and\n", + "trip_seconds > 60 and trip_seconds < 6*60*60 and\n", "trip_miles > 0 and\n", "trip_total > 3 and\n", "pickup_community_area is not NULL and \n", @@ -489,7 +850,7 @@ "## Analyze numerical data\n", "\n", "\n", - "To further anaylyze the data, there are various plots that can be used on numerical and categorical fields. In case of numerical data, one can use histograms and box plots while bar charts are suited for categorical data to better understand the distribution of the data and the outliers in the data." + "To further anaylyze the data, there are various plots that can be used on numerical and categorical fields. In case of numerical data, you can use histograms and box plots. Bar charts are suited for categorical data to better understand the distribution of the data and the outliers in the data." ] }, { @@ -524,7 +885,7 @@ "id": "c3672976d67b" }, "source": [ - "The field `trip_seconds` describes the time taken for the trip in seconds. Optionally, it can be converted into hours." + "The field `trip_seconds` describes the time taken for the trip in seconds. For ease of our analysis, let us convert it into hours." ] }, { @@ -577,6 +938,7 @@ }, "outputs": [], "source": [ + "# generate a pairplot for 10K samples\n", "sns.pairplot(\n", " data=df[[\"trip_seconds\", \"trip_miles\", \"trip_total\", \"trip_speed\"]].sample(10000)\n", ")\n", @@ -589,7 +951,7 @@ "id": "b69e8094ba39" }, "source": [ - "From the box plots and the histograms visualized so far, it is evident that there are some outliers causing skewness in the data which perhaps could be removed. Also, you can see some linear relationships between the independent variables considered in the pair-plot, for example, `trip_seconds` and `trip_miles` and the dependant variable `trip_total`." + "From the box plots and the histograms visualized so far, it is evident that there are some outliers causing skewness in the data which perhaps could be removed. Also, you can see some linear relationships between the independent variables considered in the pair-plot. For example, `trip_seconds` and `trip_miles` and the dependant variable `trip_total`." ] }, { @@ -1014,6 +1376,17 @@ "Configure Vertex Explainable AI before deploying the model. For further details, see [Configuring Vertex Explainable AI in Vertex AI models](https://cloud.google.com/vertex-ai/docs/explainable-ai/configuring-explanations#scikit-learn-and-xgboost-pre-built-containers)." ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "a130721a5375" + }, + "outputs": [], + "source": [ + "MODEL_DISPLAY_NAME = \"[your-model-display-name]\" # @param {type: \"string\"}" + ] + }, { "cell_type": "code", "execution_count": null, @@ -1022,7 +1395,11 @@ }, "outputs": [], "source": [ - "MODEL_DISPLAY_NAME = \"taxi_fare_prediction_model\"\n", + "# If the model display name is not set, choose the default one\n", + "if MODEL_DISPLAY_NAME == \"[your-model-display-name]\":\n", + " MODEL_DISPLAY_NAME = \"taxi_fare_prediction_model\"\n", + "\n", + "\n", "ARTIFACT_GCS_PATH = f\"{BUCKET_URI}/{BLOB_PATH}\"\n", "\n", "# Feature-name(Inp_feature) and Output-name(Model_output) can be arbitrary\n", @@ -1052,7 +1429,7 @@ "\n", "# Create a Vertex AI model resource with support for Vertex Explainable AI\n", "\n", - "aiplatform.init(project=PROJECT, location=LOCATION)\n", + "aiplatform.init(project=PROJECT_ID, location=REGION)\n", "\n", "model = aiplatform.Model.upload(\n", " display_name=MODEL_DISPLAY_NAME,\n", @@ -1076,7 +1453,20 @@ "id": "1ed1bd9f0957" }, "source": [ - "Create an Endpoint resource for the model." + "### Create an Endpoint resource for the model\n", + "\n", + "Set a display name for the endpoint and create the endpoint resource." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "f0e5cea786b4" + }, + "outputs": [], + "source": [ + "ENDPOINT_DISPLAY_NAME = \"[your-endpoint-display-name]\" # @param {type: \"string\"}" ] }, { @@ -1087,10 +1477,12 @@ }, "outputs": [], "source": [ - "ENDPOINT_DISPLAY_NAME = \"taxi_fare_prediction_endpoint\"\n", + "# If the display name is not set, choose the default one\n", + "if ENDPOINT_DISPLAY_NAME == \"[your-endpoint-display-name]\":\n", + " ENDPOINT_DISPLAY_NAME = \"taxi_fare_prediction_endpoint\"\n", "\n", "endpoint = aiplatform.Endpoint.create(\n", - " display_name=ENDPOINT_DISPLAY_NAME, project=PROJECT, location=LOCATION\n", + " display_name=ENDPOINT_DISPLAY_NAME, project=PROJECT_ID, location=REGION\n", ")\n", "\n", "print(endpoint.display_name)\n", @@ -1100,30 +1492,23 @@ { "cell_type": "markdown", "metadata": { - "id": "2a1b280dbec6" + "id": "9eaab1c54d66" }, "source": [ - "Save the Endpoint Id for inference." + "### Deploy the model to the created endpoint with the required machine type\n", + "\n", + "Set a name for the deployment and deploy the model to the created endpoint." ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "6516bfdd5066" + "id": "6fd9517c3403" }, "outputs": [], "source": [ - "ENDPOINT_ID = \"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9eaab1c54d66" - }, - "source": [ - "Deploy the model to the created endpoint with the required machine type." + "DEPLOYED_MODEL_NAME = \"[your-deployed-model-name]\" # @param {type: \"string\"}" ] }, { @@ -1134,10 +1519,14 @@ }, "outputs": [], "source": [ - "DEPLOYED_MODEL_NAME = \"taxi_fare_prediction_deployment\"\n", + "# If the deployment name is not set, choose the default one\n", + "if DEPLOYED_MODEL_NAME == \"[your-deployed-model-name]\":\n", + " DEPLOYED_MODEL_NAME = \"taxi_fare_prediction_deployment\"\n", + "\n", + "# Set the machine type to n1-standard2\n", "MACHINE_TYPE = \"n1-standard-2\"\n", "\n", - "# deploy the model to the endpoint\n", + "# Deploy the model to the endpoint\n", "model.deploy(\n", " endpoint=endpoint,\n", " deployed_model_display_name=DEPLOYED_MODEL_NAME,\n", @@ -1156,18 +1545,18 @@ "id": "686cfdcbaef8" }, "source": [ - "Save the ID of the deployed model. The ID of the deployed model can also checked using the `endpoint.list_models()` method." + "To ensure the model is deployed, the ID of the deployed model can be checked using the `endpoint.list_models()` method." ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "018f0fdb1d60" + "id": "bab07adf5339" }, "outputs": [], "source": [ - "DEPLOYED_MODEL_ID = \"\"" + "endpoint.list_models()" ] }, { @@ -1236,7 +1625,7 @@ " \"\"\"\n", " aiplatform.init(project=project, location=location)\n", "\n", - " endpoint = aiplatform.Endpoint(endpoint_id)\n", + " # endpoint = aiplatform.Endpoint(endpoint_id)\n", "\n", " response = endpoint.explain(instances=instances)\n", " print(\"#\" * 10 + \"Explanations\" + \"#\" * 10)\n", @@ -1266,7 +1655,7 @@ "\n", "\n", "test_json = [X_test.iloc[0].tolist(), X_test.iloc[1].tolist()]\n", - "prediction = explain_tabular_sample(PROJECT, LOCATION, ENDPOINT_ID, test_json)" + "prediction = explain_tabular_sample(PROJECT_ID, REGION, endpoint, test_json)" ] }, { @@ -1277,7 +1666,7 @@ "source": [ "## Next steps\n", "\n", - "Since the Chicago Taxi Trips dataset is continuously updating, one can preform the same kind of analysis and model training every time a new set of data is available. The date range can also be increased from a week to a month or more depending on the quality of the data. Most of the steps followed in this notebook would still be valid and can be applied over the new data unless the data is too noisy. Perhaps, the notebook itself can be scheduled to run at the specified times to retrain the model using the scheduling option of [Vertex AI Workbench's executor](https://console.cloud.google.com/vertex-ai/workbench/list/executions). " + "Since the Chicago Taxi Trips dataset is continuously updating, one can preform the same kind of analysis and model training every time a new set of data is available. The date range can also be increased from a week to a month or more depending on the quality of the data. Most of the steps followed in this notebook would still be valid and can be applied over the new data unless the data is too noisy. In fact, the notebook itself can be scheduled to run at the specified times to retrain the model using the scheduling option of [Vertex AI Workbench's executor](https://console.cloud.google.com/vertex-ai/workbench/list/executions). " ] }, { @@ -1287,11 +1676,24 @@ }, "source": [ "## Clean up\n", - "\n", "\n", - "Delete the resources created in this notebook.\n", + "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n", + "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n", "\n", - "Undeploy the model by specifying the `DEPLOYED_MODEL_ID`." + "Otherwise, you can delete the individual resources you created in this tutorial:\n", + "\n", + "- Model\n", + "- Endpoint\n", + "- Cloud Storage Bucket" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f28a9843a13e" + }, + "source": [ + "Undeploy the model" ] }, { @@ -1302,7 +1704,7 @@ }, "outputs": [], "source": [ - "endpoint.undeploy(deployed_model_id=DEPLOYED_MODEL_ID)" + "endpoint.undeploy_all()" ] }, { @@ -1362,7 +1764,11 @@ }, "outputs": [], "source": [ - "! gsutil -m rm -r $BUCKET_URI" + "# Set this to true only if you'd like to delete your bucket\n", + "delete_bucket = False\n", + "\n", + "if delete_bucket or os.getenv(\"IS_TESTING\"):\n", + " ! gsutil -m rm -r $BUCKET_URI" ] } ],