feat: Automl text model batch predict

* feat: add notebook for custom tabular batch predict

* feat: add notebook for custom tabular batch predict

* feat: add example for BQ input

* feat: add example for BQ input

* feat: add example for BQ input

* feat: add example for BQ input

community-content/CODEOWNERS

@@ -1,5 +1,6 @@
 *  @vertex-ai-samples-contributors  @GoogleCloudPlatform/cloudml-samples-owners
 /tf_agents_bandits_movie_recommendation_with_kfp_and_vertex_sdk	@yinghsienwu
+/pytorch_pre_built_images_deployment @googleapis/vertex-prediction-team
 /pytorch_text_classification_using_vertex_sdk_and_gcloud  @RajeshThallam
 /pytorch_text_classification_using_vertex_sdk_and_gcloud		@RajeshThallam @ultrons
 /sklearn_text_classification_from_script_using_vertex_sdk       @maxhardt

community-content/pytorch_pre_built_images_deployment/README.md

@@ -0,0 +1,30 @@
+# PyTorch Deployment on Google Cloud: Text Classification
+
+**This is an Experimental release**, covered by the Pre-GA Offerings Terms of your Google Cloud Platform [Terms of Service](https://cloud.google.com/terms).
+
+Experiments are focused on validating a prototype and are not guaranteed to be released. They are not intended for production use or covered by any SLA, support obligation, or deprecation policy and might be subject to backward-incompatible changes.
+
+**Kindly drop us a note before you run any scale tests.**
+
+**Do not hesitate to contact vertexai-prediction-preview-feedback@google.com if you have any questions or run into any issues.**
+
+The projects need to be added to the allowlist in order to deploy PyTorch models using Vertex AI Prediction pre-built PyTorch images. If you are interested in the feature, please send an email to vertexai-prediction-preview-feedback@google.com to provide your project numbers OR project ids.
+
+## Overview
+
+In the PyTorch on Google Cloud series of blog posts, we aim to share how to deploy PyTorch models at scale on [Vertex AI](https://cloud.google.com/vertex-ai).
+
+This tutorial on text classification shows how to deploy a PyTorch based text classification model on [Vertex AI](https://cloud.google.com/vertex-ai/docs/start/client-libraries#python) using Vertex SDK and [`gcloud ai`](https://cloud.google.com/sdk/gcloud/reference/beta/ai).
+
+## Notebooks
+
+| <h4>Notebook</h4> | <h4>Description</h4>                       |
+| :-------- | :------- |
+| [pytorch-text-classification-vertex-ai-deploy.ipynb](./pytorch-text-classification-vertex-ai-deploy.ipynb) | Notebook to show deploying a PyTorch model on Vertex AI |
+
+## Folders
+
+
+| <h4>Folder Name</h4> | <h4>Description</h4>                       |
+| :-------- | :------- |
+| [`predictor`](./predictor) | Folder with custom prediction handler to deploy a PyTorch model to Vertex Prediction. In the [notebook](./pytorch-text-classification-vertex-ai-deploy.ipynb), this folder is used for deploying a PyTorch model on Vertex AI using Vertex Prediction pre-built PyTorch images |

community-content/pytorch_pre_built_images_deployment/predictor/custom_handler.py

@@ -0,0 +1,91 @@
+
+import os
+import json
+import logging
+
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+from ts.torch_handler.base_handler import BaseHandler
+
+logger = logging.getLogger(__name__)
+
+
+class TransformersClassifierHandler(BaseHandler):
+    """
+    The handler takes an input string and returns the classification text 
+    based on the serialized transformers checkpoint.
+    """
+    def __init__(self):
+        super(TransformersClassifierHandler, self).__init__()
+        self.initialized = False
+
+    def initialize(self, ctx):
+        """ Loads the model.pt file and initialized the model object.
+        Instantiates Tokenizer for preprocessor to use
+        Loads labels to name mapping file for post-processing inference response
+        """
+        self.manifest = ctx.manifest
+
+        properties = ctx.system_properties
+        model_dir = properties.get("model_dir")
+        self.device = torch.device("cuda:" + str(properties.get("gpu_id")) if torch.cuda.is_available() else "cpu")
+
+        # Read model serialize/pt file
+        serialized_file = self.manifest["model"]["serializedFile"]
+        model_pt_path = os.path.join(model_dir, serialized_file)
+        if not os.path.isfile(model_pt_path):
+            raise RuntimeError("Missing the model.pt or pytorch_model.bin file")
+        
+        # Load model
+        self.model = AutoModelForSequenceClassification.from_pretrained(model_dir)
+        self.model.to(self.device)
+        self.model.eval()
+        logger.debug('Transformer model from path {0} loaded successfully'.format(model_dir))
+        
+        # Ensure to use the same tokenizer used during training
+        self.tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
+
+        # Read the mapping file, index to object name
+        mapping_file_path = os.path.join(model_dir, "index_to_name.json")
+
+        if os.path.isfile(mapping_file_path):
+            with open(mapping_file_path) as f:
+                self.mapping = json.load(f)
+        else:
+            logger.warning('Missing the index_to_name.json file. Inference output will default.')
+            self.mapping = {"0": "Negative",  "1": "Positive"}
+
+        self.initialized = True
+
+    def preprocess(self, data):
+        """ Preprocessing input request by tokenizing
+            Extend with your own preprocessing steps as needed
+        """
+        text = data[0].get("data")
+        if text is None:
+            text = data[0].get("body")
+        sentences = text.decode('utf-8')
+        logger.info("Received text: '%s'", sentences)
+
+        # Tokenize the texts
+        tokenizer_args = ((sentences,))
+        inputs = self.tokenizer(*tokenizer_args,
+                                padding='max_length',
+                                max_length=128,
+                                truncation=True,
+                                return_tensors = "pt")
+        return inputs
+
+    def inference(self, inputs):
+        """ Predict the class of a text using a trained transformer model.
+        """
+        prediction = self.model(inputs['input_ids'].to(self.device))[0].argmax().item()
+
+        if self.mapping:
+            prediction = self.mapping[str(prediction)]
+
+        logger.info("Model predicted: '%s'", prediction)
+        return [prediction]
+
+    def postprocess(self, inference_output):
+        return inference_output

community-content/pytorch_pre_built_images_deployment/predictor/index_to_name.json

@@ -0,0 +1,5 @@
+
+{
+    "0": "Negative", 
+    "1": "Positive"
+}

notebooks/community/ml_ops/stage6/get_started_with_custom_image_model_batch.ipynb

@@ -82,9 +82,11 @@
         "The steps performed include:\n",
         "\n",
         "- Download a pretrained image classification model from TensorFlow Hub.\n",
+        "- Upload the TensorFlow Hub model as a `Vertex AI Model` resource.\n",
+        "- Make batch prediction with raw (uncompressed) image data to the `Model` resource, in JSONL format.\n",
         "- Create a serving function to receive compressed image data, and output decomopressed preprocessed data for the model input.\n",
         "- Upload the TensorFlow Hub model and serving function as a `Vertex AI Model` resource.\n",
-        "- Make batch prediction to the `Model` resource, in JSONL format.\n",
+        "- Make batch prediction with compressed image data to the `Model` resource, in File-List format.\n",
         "\n",
         "There is one key difference between using batch prediction and using online prediction:\n",
         "\n",
@@ -697,7 +699,7 @@
       "source": [
         "### Upload the TensorFlow Hub model to a `Vertex AI Model` resource\n",
         "\n",
-        "Finally, you upload the model artifacts from the TFHub model and serving function into a `Vertex AI Model` resource using the method `upload()`, with the following parameters:\n",
+        "Finally, you upload the model artifacts from the TFHub model into a `Vertex AI Model` resource using the method `upload()`, with the following parameters:\n",
         "\n",
         "- `display_name`: A human readable name for the `Model` resource.\n",
         "- `artifact_uri`: The Cloud Storage location of the model package.\n",
@@ -1035,6 +1037,377 @@
         "            break"
       ]
     },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "12ed21da6f1f"
+      },
+      "source": [
+        "#### Delete the batch prediction job\n",
+        "\n",
+        "You can delete your `Vertex AI Batch Prediction` job with the `delete()` method."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "286a90d9b6e7"
+      },
+      "outputs": [],
+      "source": [
+        "batch_prediction_job.delete()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "6cdd298348a1"
+      },
+      "source": [
+        "#### Delete the model\n",
+        "\n",
+        "You can delete your `Vertex AI Model` resource with the `delete()` method."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4c9484c41c39"
+      },
+      "outputs": [],
+      "source": [
+        "model.delete()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "68f6562b12cb"
+      },
+      "source": [
+        "## Image models with serving functions\n",
+        "\n",
+        "Previously, your model server took input as a 3-dimensional array. Image models typically take a compressed image and use a serving function fused to the model to decompress the compressed image into a 3-dimensional array, and other preprocesing -- such as normalizing the pixel values.\n",
+        "\n",
+        "Next, you upload your custom image model as a `Vertex AI Model` resource with a serving function. During upload, you define a serving function to convert data to the format your model expects. If you send encoded data to Vertex AI, your serving function ensures that the data is decoded on the model server before it is passed as input to your model.\n",
+        "\n",
+        "### How does the serving function work\n",
+        "\n",
+        "When you send a request to an online prediction server, the request is received by a HTTP server. The HTTP server extracts the prediction request from the HTTP request content body. The extracted prediction request is forwarded to the serving function. For Google pre-built prediction containers, the request content is passed to the serving function as a `tf.string`.\n",
+        "\n",
+        "The serving function consists of two parts:\n",
+        "\n",
+        "- `preprocessing function`:\n",
+        "  - Converts the input (`tf.string`) to the input shape and data type of the underlying model (dynamic graph).\n",
+        "  - Performs the same preprocessing of the data that was done during training the underlying model -- e.g., normalizing, scaling, etc.\n",
+        "- `post-processing function`:\n",
+        "  - Converts the model output to format expected by the receiving application -- e.q., compresses the output.\n",
+        "  - Packages the output for the the receiving application -- e.g., add headings, make JSON object, etc.\n",
+        "\n",
+        "Both the preprocessing and post-processing functions are converted to static graphs which are fused to the model. The output from the underlying model is passed to the post-processing function. The post-processing function passes the converted/packaged output back to the HTTP server. The HTTP server returns the output as the HTTP response content.\n",
+        "\n",
+        "One consideration you need to consider when building serving functions for TF.Keras models is that they run as static graphs. That means, you cannot use TF graph operations that require a dynamic graph. If you do, you will get an error during the compile of the serving function which will indicate that you are using an EagerTensor which is not supported."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "serving_function_image:post"
+      },
+      "source": [
+        "### Serving function for image data\n",
+        "\n",
+        "#### Preprocessing\n",
+        "\n",
+        "To pass images to the prediction service, you encode the compressed (e.g., JPEG) image bytes into base 64 -- which makes the content safe from modification while transmitting binary data over the network. Since this deployed model expects input data as raw (uncompressed) bytes, you need to ensure that the base 64 encoded data gets converted back to raw bytes, and then preprocessed to match the model input requirements, before it is passed as input to the deployed model.\n",
+        "\n",
+        "To resolve this, you define a serving function (`serving_fn`) and attach it to the model as a preprocessing step. Add a `@tf.function` decorator so the serving function is fused to the underlying model (instead of upstream on a CPU).\n",
+        "\n",
+        "When you send a prediction or explanation request, the content of the request is base 64 decoded into a Tensorflow string (`tf.string`), which is passed to the serving function (`serving_fn`). The serving function preprocesses the `tf.string` into raw (uncompressed) numpy bytes (`preprocess_fn`) to match the input requirements of the model:\n",
+        "\n",
+        "- `io.decode_jpeg`- Decompresses the JPG image which is returned as a Tensorflow tensor with three channels (RGB).\n",
+        "- `image.convert_image_dtype` - Changes integer pixel values to float 32, and rescales pixel data between 0 and 1.\n",
+        "- `image.resize` - Resizes the image to match the input shape for the model.\n",
+        "\n",
+        "At this point, the data can be passed to the model (`m_call`), via a concrete function. The serving function is a static graph, while the model is a dynamic graph. The concrete function performs the tasks of marshalling the input data from the serving function to the model, and marshalling the prediction result from the model back to the serving function."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "serving_function_image"
+      },
+      "outputs": [],
+      "source": [
+        "CONCRETE_INPUT = \"numpy_inputs\"\n",
+        "\n",
+        "\n",
+        "def _preprocess(bytes_input):\n",
+        "    decoded = tf.io.decode_jpeg(bytes_input, channels=3)\n",
+        "    decoded = tf.image.convert_image_dtype(decoded, tf.float32)\n",
+        "    resized = tf.image.resize(decoded, size=(224, 224))\n",
+        "    return resized\n",
+        "\n",
+        "\n",
+        "@tf.function(input_signature=[tf.TensorSpec([None], tf.string)])\n",
+        "def preprocess_fn(bytes_inputs):\n",
+        "    decoded_images = tf.map_fn(\n",
+        "        _preprocess, bytes_inputs, dtype=tf.float32, back_prop=False\n",
+        "    )\n",
+        "    return {\n",
+        "        CONCRETE_INPUT: decoded_images\n",
+        "    }  # User needs to make sure the key matches model's input\n",
+        "\n",
+        "\n",
+        "@tf.function(input_signature=[tf.TensorSpec([None], tf.string)])\n",
+        "def serving_fn(bytes_inputs):\n",
+        "    images = preprocess_fn(bytes_inputs)\n",
+        "    prob = m_call(**images)\n",
+        "    return prob\n",
+        "\n",
+        "\n",
+        "m_call = tf.function(tfhub_model.call).get_concrete_function(\n",
+        "    [tf.TensorSpec(shape=[None, 224, 224, 3], dtype=tf.float32, name=CONCRETE_INPUT)]\n",
+        ")\n",
+        "\n",
+        "tf.saved_model.save(tfhub_model, MODEL_DIR, signatures={\"serving_default\": serving_fn})"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "serving_function_signature:image"
+      },
+      "source": [
+        "## Get the serving function signature\n",
+        "\n",
+        "You can get the signatures of your model's input and output layers by reloading the model into memory, and querying it for the signatures corresponding to each layer.\n",
+        "\n",
+        "For your purpose, you need the signature of the serving function. Why? Well, when we send our data for prediction as a HTTP request packet, the image data is base64 encoded, and our TF.Keras model takes numpy input. Your serving function will do the conversion from base64 to a numpy array.\n",
+        "\n",
+        "When making a prediction request, you need to route the request to the serving function instead of the model, so you need to know the input layer name of the serving function -- which you will use later when you make a prediction request."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "serving_function_signature:image"
+      },
+      "outputs": [],
+      "source": [
+        "loaded = tf.saved_model.load(MODEL_DIR)\n",
+        "\n",
+        "serving_input = list(\n",
+        "    loaded.signatures[\"serving_default\"].structured_input_signature[1].keys()\n",
+        ")[0]\n",
+        "print(\"Serving function input:\", serving_input)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e8ce91147c93"
+      },
+      "source": [
+        "### Upload the TensorFlow Hub model to a `Vertex AI Model` resource\n",
+        "\n",
+        "Finally, you upload the model artifacts from the TFHub model and serving function into a `Vertex AI Model` resource using the method `upload()`, with the following parameters:\n",
+        "\n",
+        "- `display_name`: A human readable name for the `Model` resource.\n",
+        "- `artifact_uri`: The Cloud Storage location of the model package.\n",
+        "- `serving_container_image_uri`: The serving container image.\n",
+        "\n",
+        "Uploading a model into a Vertex AI Model resource returns a long running operation, since it may take a few moments. \n",
+        "\n",
+        "*Note:* When you upload the model artifacts to a `Vertex AI Model` resource, you specify the corresponding deployment container image."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ad61e1429512"
+      },
+      "outputs": [],
+      "source": [
+        "model = aip.Model.upload(\n",
+        "    display_name=\"resnet_\" + UUID,\n",
+        "    artifact_uri=MODEL_DIR,\n",
+        "    serving_container_image_uri=DEPLOY_IMAGE,\n",
+        ")\n",
+        "\n",
+        "print(model)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "make_prediction"
+      },
+      "source": [
+        "### Make a batch prediction request\n",
+        "\n",
+        "Previously, you formatted the batch prediction instances using JSONL as 3-dimensional arrays. This time, you format the batch prediction instances as a file-list, where each line in the file is a Cloud Storage location of a compressed image."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "b1e29665076f"
+      },
+      "source": [
+        "#### Prepare data for batch prediction\n",
+        "\n",
+        "\n",
+        "Next, you format the same batch prediction request instances as a File-List format."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "becaaf02edde"
+      },
+      "outputs": [],
+      "source": [
+        "import tensorflow.python.ops.numpy_ops.np_config as np_config\n",
+        "\n",
+        "np_config.enable_numpy_behavior()\n",
+        "\n",
+        "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()\n",
+        "\n",
+        "n = 1\n",
+        "for image in x_test[:10]:\n",
+        "    c_image = tf.io.encode_jpeg(image)\n",
+        "    with tf.io.gfile.GFile(BUCKET_URI + f\"/images/image{n}.jpg\", \"wb\") as f:\n",
+        "        f.write(c_image.numpy())\n",
+        "    n += 1\n",
+        "\n",
+        "BATCH_PREDICTION_INSTANCES_FILE = \"batch_prediction_instances.txt\"\n",
+        "\n",
+        "BATCH_PREDICTION_GCS_SOURCE = (\n",
+        "    BUCKET_URI + \"/batch_prediction_instances/\" + BATCH_PREDICTION_INSTANCES_FILE\n",
+        ")\n",
+        "\n",
+        "with tf.io.gfile.GFile(BATCH_PREDICTION_GCS_SOURCE, \"w\") as f:\n",
+        "    for n in range(1, 11):\n",
+        "        f.write(BUCKET_URI + f\"/images/image{n}.jpg\\n\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "send_prediction_request:image"
+      },
+      "source": [
+        "### Send the prediction request\n",
+        "\n",
+        "BLAH\n",
+        "\n",
+        "To make a batch prediction request, call the model object's `batch_predict` method with the following parameters: \n",
+        "- `instances_format`: The format of the batch prediction request file: \"jsonl\", \"csv\", \"bigquery\", \"tf-record\", \"tf-record-gzip\" or \"file-list\"\n",
+        "- `prediction_format`: The format of the batch prediction response file: \"jsonl\", \"csv\", \"bigquery\", \"tf-record\", \"tf-record-gzip\" or \"file-list\"\n",
+        "- `job_display_name`: The human readable name for the prediction job.\n",
+        " - `gcs_source`: A list of one or more Cloud Storage paths to your batch prediction requests.\n",
+        "- `gcs_destination_prefix`: The Cloud Storage path that the service will write the predictions to.\n",
+        "- `model_parameters`: Additional filtering parameters for serving prediction results.\n",
+        "- `machine_type`: The type of machine to use for training.\n",
+        "- `accelerator_type`: The hardware accelerator type.\n",
+        "- `accelerator_count`: The number of accelerators to attach to a worker replica.\n",
+        "- `starting_replica_count`: The number of compute instances to initially provision.\n",
+        "- `max_replica_count`: The maximum number of compute instances to scale to. In this tutorial, only one instance is provisioned.\n",
+        "\n",
+        "### Compute instance scaling\n",
+        "\n",
+        "You can specify a single instance (or node) to process your batch prediction request. This tutorial uses a single node, so the variables `MIN_NODES` and `MAX_NODES` are both set to `1`.\n",
+        "\n",
+        "If you want to use multiple nodes to process your batch prediction request, set `MAX_NODES` to the maximum number of nodes you want to use. Vertex AI autoscales the number of nodes used to serve your predictions, up to the maximum number you set. Refer to the [pricing page](https://cloud.google.com/vertex-ai/pricing#prediction-prices) to understand the costs of autoscaling with multiple nodes.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1cf1076178fc"
+      },
+      "outputs": [],
+      "source": [
+        "MIN_NODES = 1\n",
+        "MAX_NODES = 1\n",
+        "\n",
+        "# The name of the job\n",
+        "BATCH_PREDICTION_JOB_NAME = \"cifar10_batch-\" + UUID\n",
+        "\n",
+        "# Folder in the bucket to write results to\n",
+        "DESTINATION_FOLDER = \"batch_prediction_results2\"\n",
+        "\n",
+        "# The Cloud Storage bucket to upload results to\n",
+        "BATCH_PREDICTION_GCS_DEST_PREFIX = BUCKET_URI + \"/\" + DESTINATION_FOLDER\n",
+        "\n",
+        "# Make SDK batch_predict method call\n",
+        "batch_prediction_job = model.batch_predict(\n",
+        "    instances_format=\"file-list\",\n",
+        "    predictions_format=\"jsonl\",\n",
+        "    job_display_name=BATCH_PREDICTION_JOB_NAME,\n",
+        "    gcs_source=BATCH_PREDICTION_GCS_SOURCE,\n",
+        "    gcs_destination_prefix=BATCH_PREDICTION_GCS_DEST_PREFIX,\n",
+        "    model_parameters=None,\n",
+        "    machine_type=DEPLOY_COMPUTE,\n",
+        "    accelerator_type=DEPLOY_GPU,\n",
+        "    accelerator_count=DEPLOY_NGPU,\n",
+        "    starting_replica_count=MIN_NODES,\n",
+        "    max_replica_count=MAX_NODES,\n",
+        "    sync=True,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "get_batch_prediction:mbsdk,custom,icn"
+      },
+      "source": [
+        "### Get the predictions\n",
+        "\n",
+        "Next, get the results from the completed batch prediction job.\n",
+        "\n",
+        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:\n",
+        "\n",
+        "- `instance`: The prediction request.\n",
+        "- `prediction`: The prediction response."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "get_batch_prediction:mbsdk,custom,icn"
+      },
+      "outputs": [],
+      "source": [
+        "import json\n",
+        "\n",
+        "bp_iter_outputs = batch_prediction_job.iter_outputs()\n",
+        "\n",
+        "prediction_results = list()\n",
+        "for blob in bp_iter_outputs:\n",
+        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
+        "        prediction_results.append(blob.name)\n",
+        "\n",
+        "tags = list()\n",
+        "for prediction_result in prediction_results:\n",
+        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
+        "    with tf.io.gfile.GFile(name=gfile_name, mode=\"r\") as gfile:\n",
+        "        for line in gfile.readlines():\n",
+        "            line = json.loads(line)\n",
+        "            print(line)\n",
+        "            break"
+      ]
+    },
     {
       "cell_type": "markdown",
       "metadata": {

notebooks/community/ml_ops/stage6/get_started_with_tf_serving.ipynb

@@ -702,7 +702,7 @@
         "        + f\"/{PRIVATE_REPO}\"\n",
         "        + \"/tf_serving:gpu\"\n",
         "    )\n",
-        "    TF_IMAGE = \"tensorflow/serving:latest-gpu\"\n",
+        "    TF_IMAGE = \"tensorflow/serving:2.5.4-gpu\"\n",
         "else:\n",
         "    DEPLOY_IMAGE = (\n",
         "        f\"{REGION}-docker.pkg.dev/\"\n",
@@ -710,15 +710,15 @@
         "        + f\"/{PRIVATE_REPO}\"\n",
         "        + \"/tf_serving:cpu\"\n",
         "    )\n",
-        "    TF_IMAGE = \"tensorflow/serving:latest\"\n",
+        "    TF_IMAGE = \"tensorflow/serving:2.5.4\"\n",
         "\n",
         "if not IS_COLAB:\n",
         "    if DEPLOY_GPU:\n",
-        "        ! sudo docker pull tensorflow/serving:latest-gpu\n",
+        "        ! sudo docker pull tensorflow/serving:2.5.4-gpu\n",
         "    else:\n",
-        "        ! sudo docker pull tensorflow/serving:latest\n",
+        "        ! sudo docker pull tensorflow/serving:2.5.4\n",
         "\n",
-        "    ! docker tag tensorflow/serving $DEPLOY_IMAGE\n",
+        "    ! docker tag $TF_IMAGE $DEPLOY_IMAGE\n",
         "    ! docker push $DEPLOY_IMAGE\n",
         "else:\n",
         "    # install docker daemon\n",
@@ -1434,7 +1434,7 @@
       },
       "outputs": [],
       "source": [
-        "delete_bucket = False\n",
+        "delete_bucket = True\n",
         "delete_model = True\n",
         "delete_endpoint = True\n",
         "delete_batch_job = True\n",

notebooks/official/automl/sdk_automl_image_object_detection_batch.ipynb

@@ -8,7 +8,7 @@
       },
       "outputs": [],
       "source": [
-        "# Copyright 2022 Google LLC\n",
+        "# Copyright 2021 Google LLC\n",
         "#\n",
         "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
         "# you may not use this file except in compliance with the License.\n",
@@ -66,17 +66,6 @@
         "This tutorial demonstrates how to use the Vertex AI SDK to create image object detection models and do batch prediction using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model."
       ]
     },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "dataset:salads,iod"
-      },
-      "source": [
-        "### Dataset\n",
-        "\n",
-        "The dataset used for this tutorial is the Salads category of the [OpenImages dataset](https://www.tensorflow.org/datasets/catalog/open_images_v4) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the bounding box locations and the corresponding type of salad items in an image from a class of five items: salad, seafood, tomato, baked goods, or cheese."
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -101,6 +90,17 @@
         "* Batch Prediction Service: Does a queued (batch) prediction for the entire set of instances in the background and stores the results in a Cloud Storage bucket when ready."
       ]
     },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "dataset:salads,iod"
+      },
+      "source": [
+        "### Dataset\n",
+        "\n",
+        "The dataset used for this tutorial is the Salads category of the [OpenImages dataset](https://www.tensorflow.org/datasets/catalog/open_images_v4) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the bounding box locations and the corresponding type of salad items in an image from a class of five items: salad, seafood, tomato, baked goods, or cheese."
+      ]
+    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -201,7 +201,7 @@
       },
       "outputs": [],
       "source": [
-        "! pip3 install -U google-cloud-storage $USER_FLAG"
+        "! pip3 install -U --upgrade tensorflow google-cloud-storage $USER_FLAG"
       ]
     },
     {
@@ -213,17 +213,6 @@
         "Install the latest version of *tensorflow* library."
       ]
     },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "install_tensorflow"
-      },
-      "outputs": [],
-      "source": [
-        "! pip3 install --upgrade tensorflow $USER_FLAG"
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -383,9 +372,9 @@
         "id": "timestamp"
       },
       "source": [
-        "#### Timestamp\n",
+        "#### UUID\n",
         "\n",
-        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial."
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
       ]
     },
     {
@@ -396,9 +385,16 @@
       },
       "outputs": [],
       "source": [
-        "from datetime import datetime\n",
+        "import random\n",
+        "import string\n",
         "\n",
-        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
+        "\n",
+        "# Generate a uuid of a specifed length(default=8)\n",
+        "def generate_uuid(length: int = 8) -> str:\n",
+        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
+        "\n",
+        "\n",
+        "UUID = generate_uuid()"
       ]
     },
     {
@@ -409,7 +405,7 @@
       "source": [
         "### Authenticate your Google Cloud account\n",
         "\n",
-        "**If you are using Google Cloud Notebooks**, your environment is already authenticated. Skip this step.\n",
+        "**If you are using Google Cloud Notebooks**, your environment is already authenticated.\n",
         "\n",
         "**If you are using Colab**, run the cell below and follow the instructions when prompted to authenticate your account via oAuth.\n",
         "\n",
@@ -494,7 +490,7 @@
       "outputs": [],
       "source": [
         "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
-        "    BUCKET_URI = \"gs://\" + PROJECT_ID + \"aip-\" + TIMESTAMP"
+        "    BUCKET_URI = \"gs://\" + PROJECT_ID + \"aip-\" + UUID"
       ]
     },
     {
@@ -669,7 +665,7 @@
       "outputs": [],
       "source": [
         "dataset = aiplatform.ImageDataset.create(\n",
-        "    display_name=\"Salads\" + \"_\" + TIMESTAMP,\n",
+        "    display_name=\"Salads\" + \"_\" + UUID,\n",
         "    gcs_source=[IMPORT_FILE],\n",
         "    import_schema_uri=aiplatform.schema.dataset.ioformat.image.bounding_box,\n",
         ")\n",
@@ -717,7 +713,7 @@
       "outputs": [],
       "source": [
         "job = aiplatform.AutoMLImageTrainingJob(\n",
-        "    display_name=\"salads_\" + TIMESTAMP,\n",
+        "    display_name=\"salads_\" + UUID,\n",
         "    prediction_type=\"object_detection\",\n",
         "    multi_label=False,\n",
         "    model_type=\"CLOUD\",\n",
@@ -760,7 +756,7 @@
       "source": [
         "model = job.run(\n",
         "    dataset=dataset,\n",
-        "    model_display_name=\"salads_\" + TIMESTAMP,\n",
+        "    model_display_name=\"salads_\" + UUID,\n",
         "    training_fraction_split=0.8,\n",
         "    validation_fraction_split=0.1,\n",
         "    test_fraction_split=0.1,\n",
@@ -790,7 +786,7 @@
       "outputs": [],
       "source": [
         "# Get model resource ID\n",
-        "models = aiplatform.Model.list(filter=\"display_name=salads_\" + TIMESTAMP)\n",
+        "models = aiplatform.Model.list(filter=\"display_name=salads_\" + UUID)\n",
         "\n",
         "# Get a reference to the Model Service client\n",
         "client_options = {\"api_endpoint\": f\"{REGION}-aiplatform.googleapis.com\"}\n",
@@ -961,7 +957,7 @@
       "outputs": [],
       "source": [
         "batch_predict_job = model.batch_predict(\n",
-        "    job_display_name=\"salads_\" + TIMESTAMP,\n",
+        "    job_display_name=\"salads_\" + UUID,\n",
         "    gcs_source=gcs_input_uri,\n",
         "    gcs_destination_prefix=BUCKET_URI,\n",
         "    machine_type=\"n1-standard-4\",\n",

notebooks/official/custom/custom-tabular-bq-managed-dataset.ipynb

@@ -44,7 +44,7 @@
         "    </a>\n",
         "  </td>\n",
         "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/custom-tabular-bq-managed-dataset.ipynb\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/custom/custom-tabular-bq-managed-dataset.ipynb\">\n",
         "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
         "      Open in Vertex AI Workbench\n",
         "    </a>\n",