mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 22:51:56 +00:00
Compare commits
33
Commits
uj10
...
migration_readme
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1f4c4e4390 | ||
|
|
92498e8148 | ||
|
|
ce48820cc1 | ||
|
|
b59e74be8a | ||
|
|
47579bdb10 | ||
|
|
761ffc7b17 | ||
|
|
99cd99b385 | ||
|
|
be783a2ed1 | ||
|
|
7523b575ab | ||
|
|
4f5a637cd0 | ||
|
|
0701ca01a8 | ||
|
|
78e0a67cd9 | ||
|
|
4ef75656a1 | ||
|
|
f5299da9b0 | ||
|
|
b986daa72d | ||
|
|
e987dce2ad | ||
|
|
f2f98cc8ed | ||
|
|
0661fcc121 | ||
|
|
f4c0fc7781 | ||
|
|
8e3db693cb | ||
|
|
0349a9552d | ||
|
|
ac1f03dd71 | ||
|
|
cb18a37b0a | ||
|
|
4c8ec05f22 | ||
|
|
c17df33c7c | ||
|
|
543cd66c16 | ||
|
|
f50dac63eb | ||
|
|
df979c1c65 | ||
|
|
7b49c23eb6 | ||
|
|
965835122b | ||
|
|
54a34a7122 | ||
|
|
a701f3b08c | ||
|
|
59a220cb58 |
@@ -56,3 +56,4 @@
|
||||
/notebooks/community/model_garden/model_garden_pytorch_owlvit.ipynb @xiangxu-google
|
||||
/notebooks/community/model_garden/model_garden_pytorch_layoutml_document_qa.ipynb @xiangxu-google
|
||||
/notebooks/community/model_garden/model_garden_pytorch_blip2.ipynb @xiangxu-google
|
||||
/notebooks/community/model_garden/model_garden_pytorch_timm.ipynb @xiangxu-google
|
||||
|
||||
@@ -271,7 +271,7 @@
|
||||
"\n",
|
||||
"\n",
|
||||
"def deploy_model(model_id, task):\n",
|
||||
" model_name = \"blip-image-captioning\"\n",
|
||||
" model_name = \"blip2\"\n",
|
||||
" endpoint = aiplatform.Endpoint.create(display_name=f\"{model_name}-endpoint\")\n",
|
||||
" serving_env = {\n",
|
||||
" \"MODEL_ID\": model_id,\n",
|
||||
@@ -345,10 +345,10 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "12893aa2c5af"
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. When the model is very large it could add 5~15mins additional time before the endpoint is ready for prediction."
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 10 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -409,10 +409,10 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0ac7f8d945e3"
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. When the model is very large it could add 5~15mins additional time before the endpoint is ready for prediction."
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 10 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -278,6 +278,8 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import base64\n",
|
||||
"import os\n",
|
||||
"from datetime import datetime\n",
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import cv2\n",
|
||||
@@ -410,6 +412,13 @@
|
||||
"# for a full list of training arguments.\n",
|
||||
"model = job.run(\n",
|
||||
" args=[\n",
|
||||
" f\"--num_machines={num_nodes}\",\n",
|
||||
" f\"--num_processes={num_gpus}\",\n",
|
||||
" \"--machine_rank=0\",\n",
|
||||
" \"--mixed_precision=no\",\n",
|
||||
" \"--gpu_ids=all\",\n",
|
||||
" \"--same_network\",\n",
|
||||
" \"--dynamo_backend=no\",\n",
|
||||
" \"controlnet/train_controlnet.py\",\n",
|
||||
" \"--tracker_project_name=train_controlnet\",\n",
|
||||
" f\"--pretrained_model_name_or_path={stable_diffusion_model_id}\",\n",
|
||||
@@ -477,6 +486,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
@@ -520,11 +538,11 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Delete models.\n",
|
||||
"model.delete()\n",
|
||||
"\n",
|
||||
"# Undeploy model and delete endpoint.\n",
|
||||
"endpoint.delete(force=True)"
|
||||
"endpoint.delete(force=True)\n",
|
||||
"\n",
|
||||
"# Delete models.\n",
|
||||
"model.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -558,6 +576,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -352,6 +352,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -351,6 +351,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -510,6 +510,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
@@ -585,6 +594,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
+9
@@ -510,6 +510,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -0,0 +1,556 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ApNHZJmT2AMH"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2023 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "YfxrFG052AMI"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI Model Garden - TIMM\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_timm.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_timm.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_timm.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
"Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" (a Python-3 CPU notebook is recommended)\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "76BCoQcm2AMJ"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook demonstrates finetuning the PyTorch [timm](https://github.com/rwightman/pytorch-image-models) models and deploying the models on [Vertex AI](https://cloud.google.com/vertex-ai).\n",
|
||||
"\n",
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"- Setup environment.\n",
|
||||
"- Create a custom training job on Vertex AI to train or finetune a model.\n",
|
||||
"- Deploy the model on Vertex AI for online prediction.\n",
|
||||
"\n",
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"* Cloud Storage\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "9iU1NKfh2AMJ"
|
||||
},
|
||||
"source": [
|
||||
"## Setup environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "7l3GN-QL2AMJ"
|
||||
},
|
||||
"source": [
|
||||
"### Setup cloud project\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project). Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage.\n",
|
||||
"\n",
|
||||
"1. [Enable Artifact Registry](https://cloud.google.com/artifact-registry/docs/enable-service) and [create a repository](https://cloud.google.com/artifact-registry/docs/repositories/create-repos) for storing docker images.\n",
|
||||
"\n",
|
||||
"1. [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs.\n",
|
||||
"\n",
|
||||
"1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hyAQXmgf2AMK"
|
||||
},
|
||||
"source": [
|
||||
"### Setup required libraries\n",
|
||||
"\n",
|
||||
"It's highly recommended to run this notebook on [Vertex AI workbench](https://cloud.google.com/vertex-ai-workbench), where you don't need to manually install any additional libraries.\n",
|
||||
"\n",
|
||||
"If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk) and [gsutil](https://cloud.google.com/storage/docs/gsutil_install)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-uF7Kb112AMK"
|
||||
},
|
||||
"source": [
|
||||
"### Colab Only\n",
|
||||
"Run the following commands for colab and skip this section if you use workbench."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "2lP6dnfy2AMK"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if \"google.colab\" in str(get_ipython()):\n",
|
||||
" ! pip3 install --upgrade google-cloud-aiplatform\n",
|
||||
"\n",
|
||||
" # Automatically restart kernel after installs\n",
|
||||
" import IPython\n",
|
||||
"\n",
|
||||
" app = IPython.Application.instance()\n",
|
||||
" app.kernel.do_shutdown(True)\n",
|
||||
"\n",
|
||||
" from google.colab import auth as google_auth\n",
|
||||
"\n",
|
||||
" google_auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "fo65Wg9Y2AMK"
|
||||
},
|
||||
"source": [
|
||||
"### Setup environment variables\n",
|
||||
"\n",
|
||||
"This notebook supports models in https://huggingface.co/docs/timm/models.\n",
|
||||
"\n",
|
||||
"You can also run\n",
|
||||
"`python -c \"from timm import list_models; print(list_models(pretrained=True))\"`\n",
|
||||
"locally to see all pretrained models.\n",
|
||||
"\n",
|
||||
"The following models have been manually verified to work with this notebook:\n",
|
||||
"\n",
|
||||
"* vit_tiny_patch16_224\n",
|
||||
"* beit_base_patch16_224\n",
|
||||
"* deit3_small_patch16_224\n",
|
||||
"* efficientnet_b2\n",
|
||||
"* mobilenetv2_100\n",
|
||||
"* resnet50\n",
|
||||
"* resnest50d\n",
|
||||
"* convnext_base\n",
|
||||
"* cspdarknet53\n",
|
||||
"* inception_v4"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "3msH2i5V2AMK"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The cloud project id.\n",
|
||||
"PROJECT_ID = \"\" # @param {type:\"string\"}\n",
|
||||
"# The region for running jobs.\n",
|
||||
"REGION = \"us-central1\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# The model you want to train and serve.\n",
|
||||
"# We use a ViT model as the example.\n",
|
||||
"MODEL_NAME = \"vit_tiny_patch16_224\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# The Cloud Storage bucket name without gs:// prefix for training outputs.\n",
|
||||
"# For example: test_bucket\n",
|
||||
"GCS_BUCKET = \"\" # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JlutYfxH2AMK"
|
||||
},
|
||||
"source": [
|
||||
"## Run training jobs\n",
|
||||
"\n",
|
||||
"This section runs a regular training job or a hyperparameter tuning job on Vertex AI.\n",
|
||||
"\n",
|
||||
"Before creating a training job, you need to prepare the dataset for training and evaluation.\n",
|
||||
"\n",
|
||||
"For example, you can use [ImageNet-1K](https://huggingface.co/datasets/imagenet-1k) held on a Cloud Storage bucket as the input dataset."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "qy4gKksX2AMK"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The training docker uri.\n",
|
||||
"TRAIN_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai-restricted/vertex-vision-model-garden-dockers/pytorch-timm-train\"\n",
|
||||
"\n",
|
||||
"# The path to data directory on Cloud Storage without gs:// prefix.\n",
|
||||
"# In the form of: <bucket-name>/path-to-data\n",
|
||||
"GCS_DATA_DIR = \"\" # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0DnlJhO72AMK"
|
||||
},
|
||||
"source": [
|
||||
"### Create a training job on Vertex AI\n",
|
||||
"\n",
|
||||
"This section creates a training job on Vertex AI. If you want to create a hyperparameter tuning job instead, you can skip to the next section."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1R5JWJGS2AML"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform\n",
|
||||
"\n",
|
||||
"# Init common setup.\n",
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=GCS_BUCKET)\n",
|
||||
"\n",
|
||||
"# Input and output path.\n",
|
||||
"data_dir = f\"/gcs/{GCS_DATA_DIR}\"\n",
|
||||
"output_dir = f\"/gcs/{GCS_BUCKET}/timm\"\n",
|
||||
"\n",
|
||||
"# Worker pool spec.\n",
|
||||
"# Single node with multiple GPUs.\n",
|
||||
"machine_type = \"n1-highmem-32\"\n",
|
||||
"num_nodes = 1\n",
|
||||
"gpu_type = \"NVIDIA_TESLA_P100\"\n",
|
||||
"num_gpus = 4\n",
|
||||
"\n",
|
||||
"# Model specific config.\n",
|
||||
"job_name = f\"pytorch-{MODEL_NAME}\"\n",
|
||||
"batch_size = 32\n",
|
||||
"epochs = 2\n",
|
||||
"\n",
|
||||
"job = aiplatform.CustomContainerTrainingJob(\n",
|
||||
" display_name=job_name,\n",
|
||||
" container_uri=TRAIN_DOCKER_URI,\n",
|
||||
")\n",
|
||||
"model = job.run(\n",
|
||||
" args=[\n",
|
||||
" \"--standalone\",\n",
|
||||
" f\"--nnodes={num_nodes}\",\n",
|
||||
" f\"--nproc_per_node={num_gpus}\",\n",
|
||||
" \"train.py\",\n",
|
||||
" data_dir,\n",
|
||||
" f\"--model={MODEL_NAME}\",\n",
|
||||
" \"--pretrained\",\n",
|
||||
" f\"--output={output_dir}\",\n",
|
||||
" f\"--batch-size={batch_size}\",\n",
|
||||
" f\"--epochs={epochs}\",\n",
|
||||
" ],\n",
|
||||
" replica_count=num_nodes,\n",
|
||||
" machine_type=machine_type,\n",
|
||||
" accelerator_type=gpu_type,\n",
|
||||
" accelerator_count=num_gpus,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Tf6P7ZI82AML"
|
||||
},
|
||||
"source": [
|
||||
"### Create a hyperparameter tuning job on Vertex AI\n",
|
||||
"\n",
|
||||
"You can use a [hyperparameter tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview) job to find the best configuration of your hyperparameters.\n",
|
||||
"\n",
|
||||
"You can skip this section if you already trained a model in the previous section and do not want to tune the hyperparameters."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Hy_aCff_2AML"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform\n",
|
||||
"from google.cloud.aiplatform import hyperparameter_tuning as hpt\n",
|
||||
"\n",
|
||||
"# Init common setup.\n",
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=GCS_BUCKET)\n",
|
||||
"\n",
|
||||
"# Input and output path.\n",
|
||||
"data_dir = f\"/gcs/{GCS_DATA_DIR}\"\n",
|
||||
"output_dir = f\"/gcs/{GCS_BUCKET}/timm\"\n",
|
||||
"\n",
|
||||
"# Model specific config.\n",
|
||||
"job_name = f\"pytorch-hp-{MODEL_NAME}\"\n",
|
||||
"batch_size = 32\n",
|
||||
"epochs = 2\n",
|
||||
"\n",
|
||||
"# Worker pool spec.\n",
|
||||
"machine_type = \"n1-highmem-16\"\n",
|
||||
"num_nodes = 1\n",
|
||||
"gpu_type = \"NVIDIA_TESLA_V100\"\n",
|
||||
"num_gpus = 2\n",
|
||||
"worker_pool_specs = [\n",
|
||||
" {\n",
|
||||
" \"machine_spec\": {\n",
|
||||
" \"machine_type\": machine_type,\n",
|
||||
" \"accelerator_type\": gpu_type,\n",
|
||||
" \"accelerator_count\": num_gpus,\n",
|
||||
" },\n",
|
||||
" \"replica_count\": num_nodes,\n",
|
||||
" \"container_spec\": {\n",
|
||||
" \"image_uri\": TRAIN_DOCKER_URI,\n",
|
||||
" \"args\": [\n",
|
||||
" \"--standalone\",\n",
|
||||
" f\"--nnodes={num_nodes}\",\n",
|
||||
" f\"--nproc_per_node={num_gpus}\",\n",
|
||||
" \"train.py\",\n",
|
||||
" data_dir,\n",
|
||||
" f\"--model={MODEL_NAME}\",\n",
|
||||
" \"--pretrained\",\n",
|
||||
" f\"--output={output_dir}\",\n",
|
||||
" f\"--batch-size={batch_size}\",\n",
|
||||
" f\"--epochs={epochs}\",\n",
|
||||
" ],\n",
|
||||
" },\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Hyperparameter job specs.\n",
|
||||
"metric_spec = {\"top1_accuracy\": \"maximize\"}\n",
|
||||
"parameter_spec = {\n",
|
||||
" \"lr\": hpt.DoubleParameterSpec(min=0.001, max=0.05, scale=\"log\"),\n",
|
||||
"}\n",
|
||||
"max_trial_count = 2\n",
|
||||
"parallel_trial_count = 2\n",
|
||||
"\n",
|
||||
"# Launch jobs.\n",
|
||||
"training_job = aiplatform.CustomJob(\n",
|
||||
" display_name=job_name, worker_pool_specs=worker_pool_specs\n",
|
||||
")\n",
|
||||
"hp_job = aiplatform.HyperparameterTuningJob(\n",
|
||||
" display_name=job_name,\n",
|
||||
" custom_job=training_job,\n",
|
||||
" metric_spec=metric_spec,\n",
|
||||
" parameter_spec=parameter_spec,\n",
|
||||
" max_trial_count=max_trial_count,\n",
|
||||
" parallel_trial_count=parallel_trial_count,\n",
|
||||
")\n",
|
||||
"hp_job.run()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DAyRWwqW2AML"
|
||||
},
|
||||
"source": [
|
||||
"## Deploy model for online prediction\n",
|
||||
"\n",
|
||||
"This section uploads the model to Model Registry and deploys it on an Endpoint resource.\n",
|
||||
"\n",
|
||||
"The model deployment step will take ~15 minutes to complete."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "jbNbg0yR2AML"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The serve docker uri.\n",
|
||||
"SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai-restricted/vertex-vision-model-garden-dockers/pytorch-timm-serve\"\n",
|
||||
"# The port number used by torchserve traffic.\n",
|
||||
"SERVE_PORT = 7080\n",
|
||||
"# The path to model checkpoint file, including gs:// prefix.\n",
|
||||
"MODEL_PT_PATH = \"gs://path_to_model_best.pth.tar\" # @param {type:\"string\"}\n",
|
||||
"# [Optional] the path to index_to_name.json, including gs:// prefix.\n",
|
||||
"INDEX_TO_NAME_FILE = \"gs://path_to_index_to_name.json\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Converts gs:// uris to /gcs/ to use GCS Fuse.\n",
|
||||
"# See https://github.com/GoogleCloudPlatform/gcsfuse.\n",
|
||||
"MODEL_PT_PATH = MODEL_PT_PATH.replace(\"gs://\", \"/gcs/\")\n",
|
||||
"INDEX_TO_NAME_FILE = INDEX_TO_NAME_FILE.replace(\"gs://\", \"/gcs/\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "INPri3HQ2AML"
|
||||
},
|
||||
"source": [
|
||||
"### Upload and deploy model on Vertex AI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "WOvqoAaN2AML"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Upload model.\n",
|
||||
"serving_env = {\n",
|
||||
" \"MODEL_NAME\": MODEL_NAME,\n",
|
||||
" \"MODEL_PT_PATH\": MODEL_PT_PATH,\n",
|
||||
" \"INDEX_TO_NAME_FILE\": INDEX_TO_NAME_FILE,\n",
|
||||
"}\n",
|
||||
"model = aiplatform.Model.upload(\n",
|
||||
" display_name=MODEL_NAME,\n",
|
||||
" serving_container_image_uri=SERVE_DOCKER_URI,\n",
|
||||
" serving_container_ports=[SERVE_PORT],\n",
|
||||
" serving_container_predict_route=\"/predictions/timm_serving\",\n",
|
||||
" serving_container_health_route=\"/ping\",\n",
|
||||
" serving_container_environment_variables=serving_env,\n",
|
||||
")\n",
|
||||
"# Or reuse a pre-uploaded model.\n",
|
||||
"# model = aiplatform.Model('projects/123456789/locations/us-central1/models/123456789@1')\n",
|
||||
"\n",
|
||||
"# Create an endpoint.\n",
|
||||
"endpoint = aiplatform.Endpoint.create(display_name=\"pytorch-timm-endpoint\")\n",
|
||||
"# Or reuse a pre-created endpoint.\n",
|
||||
"# endpoint = aiplatform.Endpoint('projects/123456789/locations/us-central1/endpoints/123456789')\n",
|
||||
"\n",
|
||||
"# Deploy model to endpoint.\n",
|
||||
"model.deploy(\n",
|
||||
" endpoint=endpoint,\n",
|
||||
" machine_type=\"n1-standard-8\",\n",
|
||||
" accelerator_type=\"NVIDIA_TESLA_T4\",\n",
|
||||
" accelerator_count=1,\n",
|
||||
" traffic_percentage=100,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "s2gnYFEJ2AML"
|
||||
},
|
||||
"source": [
|
||||
"You can mange your uploaded models in the [Model Registry](https://pantheon.corp.google.com/vertex-ai/models) and your endpoints in the [Endpoints](https://pantheon.corp.google.com/vertex-ai/endpoints)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5UP84Q5R2AMM"
|
||||
},
|
||||
"source": [
|
||||
"### Test online prediction\n",
|
||||
"\n",
|
||||
"You will now test the deployed endpoint. Please prepare an image to predict."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "zL2Qbm2x2AMM"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import base64\n",
|
||||
"\n",
|
||||
"# You can get the deployed endpoint object by its resource name returned by Endpoint.create(). For example:\n",
|
||||
"# endpoint = aiplatform.Endpoint('projects/816369962409/locations/us-central1/endpoints/8809168414485512192')\n",
|
||||
"\n",
|
||||
"# Please upload an image and enter its filename below.\n",
|
||||
"IMAGE_FILENAME = \"cat.jpg\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Alternatively, uncomment the follow line to download a cat image for demonstration.\n",
|
||||
"# ! wget http://images.cocodataset.org/val2017/000000039769.jpg -O cat.jpg\n",
|
||||
"\n",
|
||||
"with open(IMAGE_FILENAME, \"rb\") as f:\n",
|
||||
" image_b64 = base64.b64encode(f.read()).decode(\"utf-8\")\n",
|
||||
"instances = [{\"data\": {\"b64\": image_b64}}]\n",
|
||||
"\n",
|
||||
"prediction = endpoint.predict(instances=instances)\n",
|
||||
"print(prediction)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KqSxSyT42AMM"
|
||||
},
|
||||
"source": [
|
||||
"### Clean Up Resources"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "tMwdxiH-2AMM"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"endpoint.undeploy_all()\n",
|
||||
"model.delete()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "model_garden_pytorch_timm.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -332,6 +332,15 @@
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
||||
@@ -57,6 +57,77 @@ The steps performed are:
|
||||
Learn more about [BQML ARIMA+ forecasting for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-arima/overview).
|
||||
|
||||
|
||||
[AutoML training image classification model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_classification_batch_prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML image classification model from a Python script, and then do a batch prediction using the Vertex SDK.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Get predictions from an image classification model](https://cloud.google.com/vertex-ai/docs/image-data/classification/get-predictions).
|
||||
|
||||
|
||||
[AutoML training image classification model for online prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_classification_online_prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML image classification model and deploy for online prediction from a Python script using the Vertex SDK.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Deploy the `Model` resource to a serving `Endpoint` resource.
|
||||
- Make a prediction.
|
||||
- Undeploy the `Model`.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Get predictions from an image classification model](https://cloud.google.com/vertex-ai/docs/image-data/classification/get-predictions).
|
||||
|
||||
|
||||
[AutoML training image object detection model for export to edge](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_object_detection_export_edge.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML image object detection model from a Python script using the Vertex SDK, and then export the model as an Edge model in TFLite format.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- Export the `Edge` model from the `Model` resource to Cloud Storage.
|
||||
- Download the model locally.
|
||||
- Make a local prediction.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[AutoML training image object detection model for online prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_object_detection_online_prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML image object detection model and deploy for online prediction from a Python script using the Vertex AI SDK.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Deploy the `Model` resource to a serving `Endpoint` resource.
|
||||
- Make a prediction.
|
||||
- Undeploy the `Model`.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).
|
||||
|
||||
|
||||
[AutoML Tabular Workflow pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_tabular_on_vertex_pipelines.ipynb)
|
||||
|
||||
```
|
||||
@@ -72,6 +143,38 @@ The steps performed are:
|
||||
Learn more about [Tabular Workflow for E2E AutoML](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/e2e-automl).
|
||||
|
||||
|
||||
[AutoML training text entity extraction model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_text_entity_extraction_batch_prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML text entity extraction model from a Python script, and then do a batch prediction using the Vertex AI SDK.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Entity extraction for text data](https://cloud.google.com/vertex-ai/docs/training-overview#entity_extraction_for_text).
|
||||
|
||||
|
||||
[AutoML training text sentiment analysis model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_text_sentiment_analysis_batch_prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML text sentiment analysis model from a Python script, and then do a batch prediction using the Vertex SDK.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex `Dataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Get started with AutoML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/get_started_automl_training.ipynb)
|
||||
|
||||
```
|
||||
@@ -101,9 +204,7 @@ The steps performed include:
|
||||
- Create a Vertex AI `TimeSeriesDataset` resource.
|
||||
- Train the model.
|
||||
- View the model evaluation.
|
||||
- Deploy the `Model` resource to a serving `Endpoint` resource.
|
||||
- Make a prediction.
|
||||
- Undeploy the `Model`.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
|
||||
@@ -60,6 +60,22 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Custom training with custom training container and automatic registering of the model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/custom_training_container_and_model_registry.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create a custom model from a Python script in a custom Docker container using the Vertex AI SDK, and automatically register the model in the Vertex AI Model Registry.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex AI custom job for training a model.
|
||||
- Train and register a TensorFlow model using a custom container,
|
||||
- List the registered model from the Vertex AI Model Registry.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Profile model training performance using Profiler](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/custom_training_tensorboard_profiler.ipynb)
|
||||
|
||||
```
|
||||
@@ -77,6 +93,45 @@ The steps performed include:
|
||||
Learn more about [Vertex AI TensorBoard Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler).
|
||||
|
||||
|
||||
[Get started with Vertex AI Training for XGBoost](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/get_started_vertex_training.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Training` for training a XGBoost custom model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Training using a Python package.
|
||||
- Report accuracy when hyperparameter tuning.
|
||||
- Save the model artifacts to Cloud Storage using GCSFuse.
|
||||
- Create a `Vertex AI Model` resource.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Get started with Endpoint and shared VM](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/get_started_with_vertex_endpoint_and_shared_vm.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use deployment resource pools for deploying models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Upload a pre-trained image classification model as a `Model` resource (model A).
|
||||
- Upload a pre-trained text sentence encoder model as a `Model` resource (model B).
|
||||
- Create a shared VM deployment resource pool.
|
||||
- List shared VM deployment resource pools.
|
||||
- Create two `Endpoint` resources.
|
||||
- Deploy first model (model A) to first `Endpoint` resource using deployment resource pool.
|
||||
- Deploy second model (model B) to second `Endpoint` resource using deployment resource pool.
|
||||
- Make a prediction request with first deployed model (model A).
|
||||
- Make a prediction request with second deployed model (model B).
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Shared resources across deployments](https://cloud.google.com/vertex-ai/docs/predictions/model-co-hosting).
|
||||
|
||||
|
||||
[Custom training and batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/sdk-custom-image-classification-batch.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -18,6 +18,8 @@ The steps performed include:
|
||||
|
||||
Learn more about [BigQuery Datasets](https://cloud.google.com/bigquery/docs/datasets-intro).
|
||||
|
||||
Learn more about [Vertex AI for BigQuery users](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
|
||||
|
||||
[Get started with Vertex AI Data Labeling](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/datasets/get_started_with_data_labeling.ipynb)
|
||||
|
||||
|
||||
@@ -56,6 +56,24 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
[Delete Outdated Experiments in Vertex AI TensorBoard](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/delete_outdated_tensorboard_experiments.ipynb)
|
||||
|
||||
```
|
||||
Learn how to delete outdated TensorBoard Experiments to avoid unnecessary storage costs.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- How to delete the TB Experiment with a predefined key-value label pair `<label_key, label_value>`
|
||||
|
||||
- How to delete the TB Experiments created before the `create_time`
|
||||
|
||||
- How to delete the TB Experiments created before the `update_time`
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).
|
||||
|
||||
|
||||
[Get started with Vertex AI Experiments](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_vertex_experiments.ipynb)
|
||||
|
||||
```
|
||||
@@ -88,3 +106,18 @@ The steps performed include:
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Autologging](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/autologging.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Autologging`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Enable autologging in the Vertex AI SDK.
|
||||
- Train scikit-learn model and see the resulting experiment run with metrics and parameters autologged to Vertex AI Experiments without setting an experiment run.
|
||||
- Train Tensorflow model, check autologged metrics and parameters to Vertex AI Experiments by manually setting an experiment run with `aiplatform.start_run()` and `aiplatform.end_run()`.
|
||||
- Disable autologging in the Vertex AI SDK, train a PyTorch model and check that none of the parameters or metrics are logged.
|
||||
|
||||
```
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Batch Prediction](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions).
|
||||
|
||||
|
||||
[Custom training image classification model for online prediction with explainabilty](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb)
|
||||
[Custom training image classification model for online prediction with explainability](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Training and Explainable AI` to create a custom image classification model with explanations, and then you learn to use `Vertex AI Prediction` to make an online prediction request with explanations.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
|
||||
[AutoML Image Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ1 Vertex SDK AutoML Image Classification.ipynb)
|
||||
[AutoML Image Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-classification-batch-online.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train an image model and use `Vertex AI Prediction` and `Vertex AI Batch Prediction` to do online and batch predictions.
|
||||
@@ -18,15 +18,14 @@ The steps performed include:
|
||||
Learn more about [Classification for image data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_images).
|
||||
|
||||
|
||||
[Custom Scikit-Learn model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb)
|
||||
[AutoML image object detection](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Training` to create a custom trained model and use `Vertex AI Batch Prediction` to do a batch prediction on the trained model.
|
||||
Learn to use `AutoML` to train an image model and use `Vertex AI Prediction` and `Vertex AI Batch Prediction` to do online and batch predictions.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a `Vertex AI` custom job for training a scikit-learn model.
|
||||
- Upload the trained model artifacts as a `Model` resource.
|
||||
- Train an AutoML object detection model.
|
||||
- Make a batch prediction.
|
||||
- Deploy model to a endpoint
|
||||
- Make a online prediction
|
||||
@@ -35,40 +34,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Hyperparameter Tuning](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ11 Vertex SDK Hyperparameter Tuning.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Hyperparameter` to create and tune a custom trained model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a `Vertex AI` hyperparameter tuning job for training a TensorFlow model.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[AutoML Video Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ14 Vertex SDK AutoML Video Classification.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train a video model and use `Vertex AI Batch Prediction` to do batch predictions.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Train an AutoML video classification model.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos).
|
||||
Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).
|
||||
|
||||
|
||||
[AutoML Video Object Tracking](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb)
|
||||
@@ -88,55 +54,7 @@ The steps performed include:
|
||||
Learn more about [Object tracking for video data](https://cloud.google.com/vertex-ai/docs/training-overview#object_tracking_for_videos).
|
||||
|
||||
|
||||
[Custom Image Classification w/pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb)
|
||||
|
||||
```
|
||||
Learn how to train a tensorflow image classification model using a prebuilt container and Vertex AI training.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- *Package the training code into a python application.*
|
||||
- *Containerize the training application using Cloud Build and Artifact Registry.*
|
||||
- *Create a custom container training job in Vertex AI and run it.*
|
||||
- *Evaluate the model generated from the training job.*
|
||||
- *Create a model resource for the trained model in Vertex AI Model Registry.*
|
||||
- *Run a Vertex AI batch prediction job.*
|
||||
- *Deploy the model resource to a Vertex AI Endpoint.*
|
||||
- *Run a online prediction job on the model resource.*
|
||||
- *Clean up the resources created.*
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Custom Image Classification w/custom training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb)
|
||||
|
||||
```
|
||||
Learn how to train a tensorflow image classification model using a custom container and Vertex AI training.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- *Package the training code into a python application.*
|
||||
- *Containerize the training application using Cloud Build and Artifact Registry.*
|
||||
- *Create a custom container training job in Vertex AI and run it.*
|
||||
- *Evaluate the model generated from the training job.*
|
||||
- *Create a model resource for the trained model in Vertex AI Model Registry.*
|
||||
- *Run a Vertex AI batch prediction job.*
|
||||
- *Deploy the model resource to a Vertex AI Endpoint.*
|
||||
- *Run a online prediction job on the model resource.*
|
||||
- *Clean up the resources created.*
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[AutoML Tabular Binary Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb)
|
||||
[AutoML tabular binary classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-tabular-binary-classification-online-prediction.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you create an AutoML tabular binary classification model and deploy for online prediction from a Python script using the Vertex AI SDK.
|
||||
@@ -157,26 +75,7 @@ The steps performed include:
|
||||
Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
|
||||
|
||||
[AutoML Image Object Detection](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ5 Vertex SDK AutoML Image Object Detection.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train an image model and use `Vertex AI Prediction` and `Vertex AI Batch Prediction` to do online and batch predictions.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Train an AutoML object detection model.
|
||||
- Make a batch prediction.
|
||||
- Deploy model to a endpoint
|
||||
- Make a online prediction
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).
|
||||
|
||||
|
||||
[AutoML Text Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ6 Vertex SDK AutoML Text Classification.ipynb)
|
||||
[AutoML Text Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb)
|
||||
|
||||
```
|
||||
The objective of this notebook is to build a AutoML Text Classification Model.
|
||||
@@ -198,7 +97,7 @@ The steps performed include the following:
|
||||
Learn more about [Classification for text data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_text).
|
||||
|
||||
|
||||
[AutoML Text Entity Extraction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb)
|
||||
[AutoML Text Entity Extraction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb)
|
||||
|
||||
```
|
||||
The objective of this notebook is to build a AutoML Text Entity Extraction model.
|
||||
@@ -241,7 +140,72 @@ The steps performed include the following:
|
||||
Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text).
|
||||
|
||||
|
||||
[Custom XGBoost model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb)
|
||||
[AutoML Video Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ14 Vertex SDK AutoML Video Classification.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train a video model and use `Vertex AI Batch Prediction` to do batch predictions.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Train an AutoML video classification model.
|
||||
- Make a batch prediction.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos).
|
||||
|
||||
|
||||
[Custom image classification with a custom training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-custom-container.ipynb)
|
||||
|
||||
```
|
||||
Learn how to train a tensorflow image classification model using a custom container and Vertex AI training.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- *Package the training code into a python application.*
|
||||
- *Containerize the training application using Cloud Build and Artifact Registry.*
|
||||
- *Create a custom container training job in Vertex AI and run it.*
|
||||
- *Evaluate the model generated from the training job.*
|
||||
- *Create a model resource for the trained model in Vertex AI Model Registry.*
|
||||
- *Run a Vertex AI batch prediction job.*
|
||||
- *Deploy the model resource to a Vertex AI Endpoint.*
|
||||
- *Run a online prediction job on the model resource.*
|
||||
- *Clean up the resources created.*
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Custom image classification with a pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-prebuilt-container.ipynb)
|
||||
|
||||
```
|
||||
Learn how to train a tensorflow image classification model using a prebuilt container and Vertex AI training.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- *Package the training code into a python application.*
|
||||
- *Containerize the training application using Cloud Build and Artifact Registry.*
|
||||
- *Create a custom container training job in Vertex AI and run it.*
|
||||
- *Evaluate the model generated from the training job.*
|
||||
- *Create a model resource for the trained model in Vertex AI Model Registry.*
|
||||
- *Run a Vertex AI batch prediction job.*
|
||||
- *Deploy the model resource to a Vertex AI Endpoint.*
|
||||
- *Run a online prediction job on the model resource.*
|
||||
- *Clean up the resources created.*
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Custom Scikit-Learn model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-scikit-learn-prebuilt-container.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Training` to create a custom trained model and use `Vertex AI Batch Prediction` to do a batch prediction on the trained model.
|
||||
@@ -260,3 +224,39 @@ The steps performed include:
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Custom XGBoost model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Training` to create a custom trained model and use `Vertex AI Batch Prediction` to do a batch prediction on the trained model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a `Vertex AI` custom job for training a scikit-learn model.
|
||||
- Upload the trained model artifacts as a `Model` resource.
|
||||
- Make a batch prediction.
|
||||
- Deploy model to a endpoint
|
||||
- Make a online prediction
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Hyperparameter Tuning](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-hyperparameter-tuning.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Hyperparameter` to create and tune a custom trained model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a `Vertex AI` hyperparameter tuning job for training a TensorFlow model.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ1 Vertex SDK AutoML Image Classification.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-classification-batch-online.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ1 Vertex SDK AutoML Image Classification.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-classification-batch-online.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ1 Vertex SDK AutoML Image Classification.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-image-classification-batch-online.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1458,7 +1458,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ1 Vertex SDK AutoML Image Classification.ipynb",
|
||||
"name": "sdk-automl-image-classification-batch-online.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ5 Vertex SDK AutoML Image Object Detection.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ5 Vertex SDK AutoML Image Object Detection.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ5 Vertex SDK AutoML Image Object Detection.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-image-object-detection-batch-online.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1454,7 +1454,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ5 Vertex SDK AutoML Image Object Detection.ipynb",
|
||||
"name": "sdk-automl-image-object-detection-batch-online.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+3
-2
@@ -633,8 +633,9 @@
|
||||
" gcs_source=[IMPORT_FILE],\n",
|
||||
" import_schema_uri=aip.schema.dataset.ioformat.video.object_tracking,\n",
|
||||
")\n",
|
||||
"if os.getenv('IS_TESTING'):\n",
|
||||
"if os.getenv(\"IS_TESTING\"):\n",
|
||||
" import time\n",
|
||||
"\n",
|
||||
" time.sleep(30)\n",
|
||||
"\n",
|
||||
"print(dataset.resource_name)"
|
||||
@@ -1251,7 +1252,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ15 Vertex SDK AutoML Object Tracking.ipynb",
|
||||
"name": "sdk-automl-object-tracking-batch-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-tabular-binary-classification-online-prediction.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-tabular-binary-classification-online-prediction.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-tabular-binary-classification-online-prediction.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1410,7 +1410,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb",
|
||||
"name": "sdk-automl-tabular-binary-classification-online-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -33,18 +33,18 @@
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ6 Vertex SDK AutoML Text Classification.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ6 Vertex SDK AutoML Text Classification.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ6 Vertex SDK AutoML Text Classification.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1337,7 +1337,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ6 Vertex SDK AutoML Text Classification.ipynb",
|
||||
"name": "sdk-automl-text-classification-batch-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -33,18 +33,18 @@
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-automl-text-classification-batch-prediction.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1428,7 +1428,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb",
|
||||
"name": "sdk-automl-text-entity-extraction-batch-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+1
-1
@@ -1413,7 +1413,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb",
|
||||
"name": "sdk-automl-text-sentiment-analysis-batch-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+1
-1
@@ -1297,7 +1297,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ14 Vertex SDK AutoML Video Classification.ipynb",
|
||||
"name": "sdk-automl-video-classification-batch-prediction.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-custom-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-custom-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-image-classification-custom-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1899,7 +1899,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb",
|
||||
"name": "sdk-custom-image-classification-custom-container.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-prebuilt-container.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-image-classification-prebuilt-container.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-image-classification-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -2005,7 +2005,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb",
|
||||
"name": "sdk-custom-image-classification-prebuilt-container.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -33,18 +33,18 @@
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-scikit-learn-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-scikit-learn-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-scikit-learn-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1577,7 +1577,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb",
|
||||
"name": "sdk-custom-scikit-learn-prebuilt-container.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -33,18 +33,18 @@
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1459,7 +1459,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb",
|
||||
"name": "sdk-custom-xgboost-prebuilt-container.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
+4
-4
@@ -34,18 +34,18 @@
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ11 Vertex SDK Hyperparameter Tuning.ipynb\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-hyperparameter-tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ11 Vertex SDK Hyperparameter Tuning.ipynb\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-hyperparameter-tuning.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ11 Vertex SDK Hyperparameter Tuning.ipynb\" target='_blank'>\n",
|
||||
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-hyperparameter-tuning.ipynb\" target='_blank'>\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
@@ -1414,7 +1414,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "UJ11 Vertex SDK Hyperparameter Tuning.ipynb",
|
||||
"name": "sdk-hyperparameter-tuning.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
@@ -50,7 +50,7 @@ Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex AI `Dataset`.
|
||||
- Train a Automl Tabular Classification model on the `Dataset` resource.
|
||||
- Train a Automl Text Classification model on the `Dataset` resource.
|
||||
- Import the trained `AutoML model resource` into the pipeline.
|
||||
- Run a `Batch Prediction` job.
|
||||
- Evaulate the AutoML model using the `Classification Evaluation Component`.
|
||||
@@ -152,3 +152,5 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Model evaluation in Vertex AI](https://cloud.google.com/vertex-ai/docs/evaluation/introduction).
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ The steps performed include:
|
||||
Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).
|
||||
|
||||
|
||||
[Challenger vs Blessed methodology for model deployment into production](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/pipelines/challenger_vs_blessed_deployment_method.ipynb)
|
||||
[Challenger vs Blessed methodology for model deployment into production](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/challenger_vs_blessed_deployment_method.ipynb)
|
||||
|
||||
```
|
||||
Learn how to construct a Vertex AI pipeline, which trains a new challenger version of a model, evaluates the model and compares the evaluation to the existing blessed model in production, to determine whether the challenger model becomes the blessed model for replacement in production.
|
||||
@@ -45,6 +45,10 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
Learn more about [Model evaluation in Vertex AI](https://cloud.google.com/vertex-ai/docs/evaluation/introduction).
|
||||
|
||||
|
||||
[Pipeline control structures using the KFP SDK](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/control_flow_kfp.ipynb)
|
||||
|
||||
@@ -84,7 +88,7 @@ The steps performed include:
|
||||
Learn more about [Custom training components](https://cloud.google.com/vertex-ai/docs/training/create-training-pipeline).
|
||||
|
||||
|
||||
[Training and batch prediction with BigQuery source and destinantion for a custom tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/custom_tabular_train_batch_pred_bq_pipeline.ipynb)
|
||||
[Training and batch prediction with BigQuery source and destination for a custom tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/custom_tabular_train_batch_pred_bq_pipeline.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you train a scikit-learn tabular classification model and create batch prediction job for it through a Vertex AI pipeline using `google_cloud_pipeline_components`.
|
||||
@@ -110,6 +114,43 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Batch Prediction components](https://cloud.google.com/vertex-ai/docs/pipelines/batchprediction-component).
|
||||
|
||||
|
||||
[Get started with Vertex AI Hyperparameter Tuning pipeline components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/get_started_with_hpt_pipeline_components.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use prebuilt `Google Cloud Pipeline Components` for `Vertex AI Hyperparameter Tuning`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Construct a pipeline for:
|
||||
- Hyperparameter tune/train a custom model.
|
||||
- Retrieve the tuned hyperparameter values and metrics to optimize.
|
||||
- If the metrics exceed a specified threshold.
|
||||
- Get the location of the model artifacts for the best tuned model.
|
||||
- Upload the model artifacts to a `Vertex AI Model` resource.
|
||||
- Execute a Vertex AI pipeline.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
|
||||
[Get started with machine management for Vertex AI Pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/get_started_with_machine_management.ipynb)
|
||||
|
||||
```
|
||||
Learn how to convert a self-contained custom training component into a `Vertex AI CustomJob`, whereby:
|
||||
|
||||
The steps performed in this tutorial include:
|
||||
|
||||
- Create a custom component with a self-contained training job.
|
||||
- Execute pipeline using component-level settings for machine resources
|
||||
- Convert the self-contained training component into a `Vertex AI CustomJob`.
|
||||
- Execute pipeline using customjob-level settings for machine resources
|
||||
|
||||
```
|
||||
|
||||
|
||||
[AutoML image classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_automl_images.ipynb)
|
||||
|
||||
```
|
||||
@@ -298,6 +339,32 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
|
||||
[Multicontender vs Champion methodology for model deployment into production](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/multicontender_vs_champion_deployment_method.ipynb)
|
||||
|
||||
```
|
||||
Learn how to construct a Vertex AI pipeline, which evaluates new production data from a deployed model against other versions of the model, to determine if a contender model becomes the champion model for replacement in production.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Import a pretrained (champion) model to the `Vertex AI Model Registry`.
|
||||
- Import synthetic model training evaluation metrics to the corresponding (champion) model.
|
||||
- Create a `Vertex AI Endpoint` resource
|
||||
- Deploy the champion model to the `Endpoint` resource.
|
||||
- Import additional (contender) versions of the deployed model.
|
||||
- Import synthetic model training evaluation metrics to the corresponding (contender) models.
|
||||
- Create a Vertex AI Pipeline
|
||||
- Get the champion model.
|
||||
- (Fake) Fine-tune champion model with production data
|
||||
- Import synthetic train+production evaluation metrics for the champion model.
|
||||
- Get the contender models.
|
||||
- (Fake) Fine-tune contender model with production data
|
||||
- Import synthetic train+production evaluation metrics for the contenders modesl.
|
||||
- Compare the evaluations of the contenders to the champion and set the new champion as the default.
|
||||
- Deploy the new champion model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Pipelines introduction for KFP](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/pipelines_intro_kfp.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -265,7 +265,7 @@
|
||||
"source": [
|
||||
"# Install Python package dependencies.\n",
|
||||
"print(\"Installing libraries\")\n",
|
||||
"! pip3 install {USER_FLAG} --quiet google-cloud-pipeline-components kfp\n",
|
||||
"! pip3 install {USER_FLAG} --quiet 'google-cloud-pipeline-components==1.0.20' 'kfp<2'\n",
|
||||
"! pip3 install {USER_FLAG} --quiet --upgrade google-cloud-aiplatform google-cloud-bigquery"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -14,3 +14,40 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Batch Prediction](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions).
|
||||
|
||||
|
||||
[Serving PyTorch image models with prebuilt containers on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/pytorch_image_classification_with_prebuilt_serving_containers.ipynb)
|
||||
|
||||
```
|
||||
Learn how to package and deploy a PyTorch image classification model using a prebuilt Vertex AI container with TorchServe for serving online and batch predictions.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download a pretrained image model from PyTorch
|
||||
- Create a custom model handler
|
||||
- Package model artifacts in a model archive file
|
||||
- Upload model for deployment
|
||||
- Deploy model for prediction
|
||||
- Make online predictions
|
||||
- Make batch predictions
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Pre-built containers for prediction](https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers).
|
||||
|
||||
|
||||
[Train and deploy PyTorch models with prebuilt containers on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/pytorch_train_deploy_models_with_prebuilt_containers.ipynb)
|
||||
|
||||
```
|
||||
Learn how to build, train and deploy a PyTorch image classification model using prebuilt containers for custom training and prediction.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Package training application into a Python source distribution
|
||||
- Configure and run training job in a prebuilt container
|
||||
- Package model artifacts in a model archive file
|
||||
- Upload model for deployment
|
||||
- Deploy model using a prebuilt container for prediction
|
||||
- Make online predictions
|
||||
|
||||
```
|
||||
|
||||
|
||||
@@ -15,6 +15,26 @@ The steps performed are:
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
|
||||
[Get started with TabNet builtin algorithm for training tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/get_started_with_tabnet.ipynb)
|
||||
|
||||
```
|
||||
Learn how to run `Vertex AI TabNet` built algorithm for training custom tabular models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Get the training data.
|
||||
- Configure training parameters for the `Vertex AI TabNet` container.
|
||||
- Train the model using `Vertex AI Training` using CSV data.
|
||||
- Upload the model as a `Vertex AI Model` resource.
|
||||
- Deploy the `Vertex AI Model` resource to a `Vertex AI Endpoint` resource.
|
||||
- Make a prediction with the deployed model.
|
||||
- Hyperparameter tuning the `Vertex AI TabNet` model.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
|
||||
[Vertex AI TabNet](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/tabnet_vertex_tutorial.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -14,6 +14,8 @@ The steps performed are:
|
||||
|
||||
Learn more about [Google Cloud Pipeline Components](https://cloud.google.com/vertex-ai/docs/pipelines/components-introduction).
|
||||
|
||||
Learn more about [Prophet for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/forecasting-prophet).
|
||||
|
||||
|
||||
[TabNet Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/tabnet_on_vertex_pipelines.ipynb)
|
||||
|
||||
|
||||
@@ -10,3 +10,19 @@ Learn how to use `Vertex AI Vizier` to optimize a multi-objective study.
|
||||
|
||||
Learn more about [Vertex AI Vizier](https://cloud.google.com/vertex-ai/docs/vizier/overview).
|
||||
|
||||
|
||||
[Get started with Vertex AI Vizier](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/vizier/get_started_vertex_vizier.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Vizier` for when training with `Vertex AI`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Hyperparameter tuning with Random algorithm.
|
||||
- Hyperparameter tuning with Vizier (Bayesian) algorithm.
|
||||
- Suggesting trials and updating results for Vizier study
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Vizier](https://cloud.google.com/vertex-ai/docs/vizier/overview).
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/managed/visualize-data-bigquery).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
|
||||
@@ -127,7 +127,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml#machine_learning_directly_in).
|
||||
|
||||
|
||||
[Inventory prediction on ecommerce data using Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/inventory-prediction/inventory_prediction.ipynb)
|
||||
@@ -194,7 +194,7 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml#machine_learning_directly_in).
|
||||
|
||||
|
||||
[Sentiment Analysis using AutoML Natural Language and Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/sentiment_analysis/Sentiment_Analysis.ipynb)
|
||||
@@ -218,7 +218,7 @@ The steps performed are:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/tutorials/text-classification-automl/training).
|
||||
Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text).
|
||||
|
||||
|
||||
[Digest and analyze data from BigQuery with Dataproc](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/spark/spark_bigquery.ipynb)
|
||||
|
||||
Reference in New Issue
Block a user