model_garden/notebooks/official/prediction/sdk_private_endpoint.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ab0c6268f3a5"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ This notebook has been deprecated. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Vertex AI Private Endpoint\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/prediction/sdk_private_endpoint.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/prediction/sdk_private_endpoint.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>\n",
        "\n",
        "**Please ensure this notebook is run inside a VPC since a `PrivateEndpoint` predict method can only be executed within a private network. Create a Vertex AI Workbench instance and upload and run this notebook inside that instance. When creating a new Vertex AI Workbench instance use the `default` subnetwork or create and use a unique VPC using the steps outlined [here](https://cloud.google.com/vpc/docs/create-modify-vpc-networks).**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create and use Vertex AI `PrivateEndpoint` resources for serving models. A `PrivateEndpoint` provides a low-latency, secure, private network connection to the Vertex AI online prediction service (i.e., intranet). This eliminates the overhead of networking switching and routing of a public `Endpoint` (i.e., internet).\n",
        "\n",
        "Learn more about [`PrivateEndpoint` resources](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
        "\n",
        "### Dataset\n",
        "\n",
        "This tutorial uses the [petfinder](https://storage.googleapis.com/cloud-samples-data/ai-platform-unified/datasets/tabular/petfinder-tabular-classification-tabnet-with-header.csv) dataset in the public Cloud Storage bucket `gs://cloud-samples-data/ai-platform-unified/datasets/tabular/`, which was generated from the [PetFinder.my Adoption Prediction](https://www.kaggle.com/c/petfinder-adoption-prediction). This dataset predicts how quickly an animal will be adopted.\n",
        "\n",
        "### Objective\n",
        "\n",
        "In this notebook, you will learn how to use `Vertex AI PrivateEndpoint` resources.\n",
        "\n",
        "This tutorial uses the following Google Cloud Platform Vertex AI services and resources:\n",
        "\n",
        "- `Vertex AI TabNet`\n",
        "- `Vertex AI TrainingJob`\n",
        "- `Vertex AI Model`\n",
        "- `Vertex AI PrivateEndpoint`\n",
        "- `Vertex AI Prediction`\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Import the training data.\n",
        "- Configure training parameters for the `Vertex AI TabNet` model container.\n",
        "- Train the model with CSV data using a `Vertex AI TrainingJob`.\n",
        "- Upload the model as a `Vertex AI Model` resource.\n",
        "- Configure a VPC peering connection.\n",
        "- Create a `Vertex AI PrivateEndpoint` resource.\n",
        "- Deploy the `Vertex AI Model` resource to the `Vertex AI PrivateEndpoint` resource.\n",
        "- Send a prediction request to the `Vertex AI PrivateEndpoint`.\n",
        "- Clean up resources.\n",
        "\n",
        "### Costs \n",
        "\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7fc05eeb0918"
      },
      "source": [
        "### Installation\n",
        "Install the packages required for executing this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2b4ef9b72d43"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet tensorflow\n",
        "! pip3 install --upgrade --quiet google-cloud-aiplatform\n",
        "! gcloud components update --quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "58707a750154"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f200f10a1da3"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "before_you_begin"
      },
      "source": [
        "## Before you begin\n",
        "\n",
        "### GPU runtime\n",
        "\n",
        "*Make sure you're running this notebook in a GPU runtime if you have that option. In Colab, select* **Runtime > Change Runtime Type > GPU**\n",
        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
        "2. [Make sure that billing is enabled for your project.](https://cloud.google.com/billing/docs/how-to/modify-project)\n",
        "\n",
        "3. [Enable the following APIs: Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component,storage-component.googleapis.com)\n",
        "\n",
        "4. If you are running this notebook locally, you need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
        "\n",
        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
        "\n",
        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "project_id"
      },
      "source": [
        "#### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "region"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "region"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sBCra4QMA2wR"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated.\n",
        "\n",
        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "254614fa0c46"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcp_authenticate"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a336a05c6149"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f6b2ccc891ed"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zgPO1eR3CYjk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MzGDU7TWdts_"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gcloud storage buckets create --location={REGION} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XoEqT2Y4DJmf"
      },
      "source": [
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pRUOFELefqf1"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "import google.cloud.aiplatform as aiplatform"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bffd1dbc8a8c"
      },
      "source": [
        "### Set the training container\n",
        "\n",
        "Next, use the prebuilt `Vertex AI TabNet` container for training the model.\n",
        "\n",
        "TabNet combines the best of two worlds: it is explainable (similar to simpler tree-based models) while benefiting from high performance (similar to deep neural networks). This makes it great for retailers, finance and insurance industry applications such as predicting credit scores, fraud detection and forecasting. \n",
        "\n",
        "TabNet uses a machine learning technique called sequential attention to select which model features to reason from at each step in the model. This mechanism makes it possible to explain how the model arrives at its predictions and helps it learn more accurate models. TabNet not only outperforms other neural networks and decision trees but also provides interpretable feature attributions. \n",
        "\n",
        "Read the research paper: [TabNet: Attentive Interpretable Tabular Learning](https://arxiv.org/pdf/1908.07442.pdf)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f29b4eab8132"
      },
      "outputs": [],
      "source": [
        "TRAIN_IMAGE = \"us-docker.pkg.dev/vertex-ai-restricted/builtin-algorithm/tab_net_v2\"\n",
        "print(\"Training container:\", TRAIN_IMAGE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "container:training,prediction"
      },
      "source": [
        "### Set pre-built container for deployment\n",
        "\n",
        "Set the pre-built Docker container image for prediction.\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "container:training,prediction"
      },
      "outputs": [],
      "source": [
        "if os.getenv(\"IS_TESTING_TF\"):\n",
        "    TF = os.getenv(\"IS_TESTING_TF\")\n",
        "else:\n",
        "    TF = \"2.5\".replace(\".\", \"-\")\n",
        "\n",
        "if TF[0] == \"2\":\n",
        "    DEPLOY_VERSION = \"tf2-cpu.{}\".format(TF)\n",
        "else:\n",
        "    DEPLOY_VERSION = \"tf-cpu.{}\".format(TF)\n",
        "\n",
        "DEPLOY_IMAGE = \"{}-docker.pkg.dev/vertex-ai/prediction/{}:latest\".format(\n",
        "    REGION.split(\"-\")[0], DEPLOY_VERSION\n",
        ")\n",
        "\n",
        "print(\"Deployment container:\", DEPLOY_IMAGE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1dd2b940d35d"
      },
      "source": [
        "## Get the training data\n",
        "\n",
        "Get a copy of the training data (as a CSV file) from a public Cloud Storage bucket and copy the training data to your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BoVqAMOuedPe"
      },
      "outputs": [],
      "source": [
        "# Please note that if you use csv input, the first column is the label column.\n",
        "\n",
        "IMPORT_FILE = \"petfinder-tabular-classification-tabnet-with-header.csv\"\n",
        "TRAINING_DATA_PATH = f\"{BUCKET_URI}/data/petfinder/train.csv\"\n",
        "\n",
        "! gcloud storage cp gs://cloud-samples-data/ai-platform-unified/datasets/tabular/{IMPORT_FILE} {TRAINING_DATA_PATH}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_custom_container_training_job:mbsdk,no_model"
      },
      "source": [
        "## Train the Vertex AI TabNet model\n",
        "\n",
        "\n",
        "To train a TabNet custom model, you need to create and run a custom training job.\n",
        "\n",
        "### Create custom training job\n",
        "\n",
        "A custom training job is created with the `CustomContainerTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the custom training job.\n",
        "- `container_uri`: The training container image.\n",
        "- `model_serving_container_image_uri`: The URI of a container that can serve predictions for your model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a3307f420d27"
      },
      "outputs": [],
      "source": [
        "DATASET_NAME = \"petfinder\"  # Change to your dataset name.\n",
        "\n",
        "job = aiplatform.CustomContainerTrainingJob(\n",
        "    display_name=f\"{DATASET_NAME}\",\n",
        "    container_uri=TRAIN_IMAGE,\n",
        "    model_serving_container_image_uri=DEPLOY_IMAGE,\n",
        ")\n",
        "\n",
        "print(job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xc_I_XHz8z6B"
      },
      "source": [
        "### Configure parameter settings for TabNet training \n",
        "\n",
        "Learn more about using TabeNet with the guide [Get started with builtin TabNet algorithm](https://cloud.google.com/ai-platform/training/docs/algorithms/tab-net-start)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a987b40417de"
      },
      "outputs": [],
      "source": [
        "ALGORITHM = \"tabnet\"\n",
        "MODEL_TYPE = \"classification\"\n",
        "MODEL_NAME = f\"{DATASET_NAME}_{ALGORITHM}_{MODEL_TYPE}\"\n",
        "\n",
        "OUTPUT_DIR = f\"{BUCKET_URI}/{MODEL_NAME}\"\n",
        "print(\"Output dir: \", OUTPUT_DIR)\n",
        "\n",
        "CMDARGS = [\n",
        "    \"--preprocess\",\n",
        "    \"--data_has_header\",\n",
        "    f\"--training_data_path={TRAINING_DATA_PATH}\",\n",
        "    f\"--job-dir={OUTPUT_DIR}\",\n",
        "    f\"--model_type={MODEL_TYPE}\",\n",
        "    \"--max_steps=2000\",\n",
        "    \"--batch_size=4096\",\n",
        "    \"--learning_rate=0.01\",\n",
        "    \"--prediction_raw_inputs\",\n",
        "    \"--exclude_key\",\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "train_custom_job"
      },
      "source": [
        "### Run the custom training job and create the TabNet model\n",
        "\n",
        "Use the `run` method to start training, which takes the following parameters:\n",
        "\n",
        "- `model_display_name`: The display name of the `Model` if the script produces a managed `Model`.\n",
        "- `args`: The command line arguments to be passed to the TabNet training container.\n",
        "- `replica_count`: The number of worker replicas.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "- `base_output_dir`: GCS output directory of a job.\n",
        "- `sync`: Whether to execute this method synchronously.\n",
        "\n",
        "The `run` method creates a training pipeline that trains and creates a `Model` object. After the training pipeline completes, the `run` method returns the `Model` object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c0927403746f"
      },
      "outputs": [],
      "source": [
        "MODEL_DIR = OUTPUT_DIR\n",
        "MACHINE_TYPE = \"n1-standard-4\"\n",
        "\n",
        "model = job.run(\n",
        "    model_display_name=f\"{DATASET_NAME}\",\n",
        "    args=CMDARGS,\n",
        "    replica_count=1,\n",
        "    machine_type=MACHINE_TYPE,\n",
        "    base_output_dir=MODEL_DIR,\n",
        "    sync=True,\n",
        ")\n",
        "\n",
        "print(model.gca_resource)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "91d661ac0276"
      },
      "source": [
        "### Delete the training job\n",
        "\n",
        "Use the `delete()` method to delete the training job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3e43fc3b7fe8"
      },
      "outputs": [],
      "source": [
        "job.delete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "75ab2d5dd517"
      },
      "source": [
        "## Setup a VPC peering network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a636147aac60"
      },
      "source": [
        "To use a `PrivateEndpoint`, you need to setup a VPC peering network between your project and the Vertex AI Prediction service project that is hosting VMs running your model. This eliminates additional hops in network traffic and allows using efficient HTTP protocol.\n",
        "\n",
        "Learn more about [VPC peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering).\n",
        "\n",
        "**IMPORTANT: you can only setup one VPC peering to servicenetworking.googleapis.com per project.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dd8667882684"
      },
      "source": [
        "### Create VPC peering for `default` network\n",
        "\n",
        "For simplicity, we setup VPC peering to the `default` network that a new GCP (Google Cloud Platform) project starts with. You can also create and use a different network for your project. If you setup VPC peering with any other network, make sure that the network already exists and that your VM is running on that network."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e370da6ac004"
      },
      "outputs": [],
      "source": [
        "# This is for display only; you can name the range anything.\n",
        "PEERING_RANGE_NAME = \"vertex-ai-prediction-peering-range\"\n",
        "NETWORK = \"default\"\n",
        "\n",
        "# NOTE: `prefix-length=16` means a CIDR block with mask /16 will be\n",
        "# reserved for use by Google services, such as Vertex AI.\n",
        "! gcloud compute addresses create $PEERING_RANGE_NAME \\\n",
        "  --global \\\n",
        "  --prefix-length=16 \\\n",
        "  --description=\"peering range for Google service\" \\\n",
        "  --network=$NETWORK \\\n",
        "  --purpose=VPC_PEERING"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b232ca5e5f30"
      },
      "source": [
        "### Create the VPC connection\n",
        "\n",
        "Create the connection for VPC peering.\n",
        "\n",
        "Note: If you get a PERMISSION DENIED, you may not have the neccessary role `Compute Network Admin` set for your default service account. In the Cloud Console, do the following steps.\n",
        "\n",
        "1. Goto `IAM & Admin` in the GCP dashboard.\n",
        "2. Find your service account.\n",
        "3. Click the edit icon.\n",
        "4. Select Add Another Role.\n",
        "5. Enter `Compute Network Admin`.\n",
        "6. Select Save."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d633014964f4"
      },
      "outputs": [],
      "source": [
        "! gcloud services vpc-peerings connect \\\n",
        "  --service=servicenetworking.googleapis.com \\\n",
        "  --network=$NETWORK \\\n",
        "  --ranges=$PEERING_RANGE_NAME \\\n",
        "  --project=$PROJECT_ID"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "07c4452da03a"
      },
      "source": [
        "Check the status of your peering connections."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b87fb56c5b53"
      },
      "outputs": [],
      "source": [
        "! gcloud compute networks peerings list --network $NETWORK"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "604e1798f14a"
      },
      "source": [
        "### Construct the full network name\n",
        "\n",
        "You need to have the full network resource name when you subsequently create a `PrivateEndpoint` resource for VPC peering."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0dda2dcfe5fe"
      },
      "outputs": [],
      "source": [
        "project_number = model.resource_name.split(\"/\")[1]\n",
        "print(project_number)\n",
        "\n",
        "full_network_name = f\"projects/{project_number}/global/networks/{NETWORK}\"\n",
        "full_network_name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "69cebf72d1e6"
      },
      "source": [
        "## Create a `PrivateEndpoint` resource\n",
        "\n",
        "Create a `PrivateEndpoint` resource using the `PrivateEndpoint.create()` method.\n",
        "\n",
        "In this example, the following parameters are specified:\n",
        "\n",
        "- `display_name`: A human readable name for the `PrivateEndpoint` resource.\n",
        "- `network`: The full network resource name for the VPC peering."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4f51d5c47bdf"
      },
      "outputs": [],
      "source": [
        "if not os.getenv(\"IS_TESTING\"):\n",
        "    private_endpoint = aiplatform.PrivateEndpoint.create(\n",
        "        display_name=f\"{DATASET_NAME}_private_endpoint\",\n",
        "        network=full_network_name,\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2a720a453814"
      },
      "source": [
        "### Get details on the `PrivateEndpoint` resource\n",
        "\n",
        "View the underlying details of a `PrivateEndpoint` object with the property `gca_resource`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bf2d72ebf6c1"
      },
      "outputs": [],
      "source": [
        "if not os.getenv(\"IS_TESTING\"):\n",
        "    private_endpoint.gca_resource"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "deploy_model:dedicated"
      },
      "source": [
        "## Deploy the TabNet model to the `PrivateEndpoint`\n",
        "\n",
        "Deploy the TabNet model to the newly created `PrivateEndpoint` resource to perform predictions on incoming data samples.\n",
        "\n",
        "The function takes the following parameters:\n",
        "\n",
        "- `model`: Model to be deployed.\n",
        "- `deployed_model_display_name`: A human readable name for the deployed model.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "\n",
        "The method will block until the model is deployed and eventually return an `PrivateEndpoint` object. If this is the first time a model is deployed to the endpoint, it may take a few additional minutes to complete provisioning of resources."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4e08de0c4d6f"
      },
      "outputs": [],
      "source": [
        "DEPLOYED_NAME = f\"{DATASET_NAME}_deployed_model\"\n",
        "\n",
        "if not os.getenv(\"IS_TESTING\"):\n",
        "    response = private_endpoint.deploy(\n",
        "        model=model,\n",
        "        deployed_model_display_name=DEPLOYED_NAME,\n",
        "        machine_type=\"n1-standard-4\",\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6a7e8daa2350"
      },
      "source": [
        "### Get the serving signature\n",
        "\n",
        "Download the model locally and query the model for its serving signature. The serving signature will be of the form:\n",
        "\n",
        "    ( \"feature_name_1\",  \"feature_name_2\", ... )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "23374479e310"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "loaded = tf.saved_model.load(MODEL_DIR + \"/model\")\n",
        "loaded.signatures"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9321a211f2f7"
      },
      "source": [
        "## Make a prediction\n",
        "\n",
        "Finally, make a prediction using the `predict()` method. Each instance is specified in the following dictionary format:\n",
        "\n",
        "    { \"feature_name_1\": value, \"feature_name_2\", value, ... }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "625fc7368b65"
      },
      "outputs": [],
      "source": [
        "if not os.getenv(\"IS_TESTING\"):\n",
        "    prediction = private_endpoint.predict(\n",
        "        [\n",
        "            {\n",
        "                \"Age\": 3,\n",
        "                \"Breed1\": \"Tabby\",\n",
        "                \"Color1\": \"Black\",\n",
        "                \"Color2\": \"White\",\n",
        "                \"Fee\": 100,\n",
        "                \"FurLength\": \"Short\",\n",
        "                \"Gender\": \"Male\",\n",
        "                \"Health\": \"Healthy\",\n",
        "                \"MaturitySize\": \"Small\",\n",
        "                \"PhotoAmt\": 2,\n",
        "                \"Sterilized\": \"No\",\n",
        "                \"Type\": \"Cat\",\n",
        "                \"Vaccinated\": \"No\",\n",
        "            }\n",
        "        ]\n",
        "    )\n",
        "\n",
        "    print(prediction)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TpV-iwP9qw9c"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sx_vKniMq9ZX"
      },
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
        "try:\n",
        "    private_endpoint.delete(force=True)\n",
        "    model.delete()\n",
        "except Exception as e:\n",
        "    print(e)\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gcloud storage rm --recursive $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "sdk_private_endpoint.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}