fix: detecting copyright cell

fix: filename rename (#943 )
* fix: filename rename * fix: lint issues
2026-09-26 14:42:04 +00:00 · 2022-09-13 18:08:32 +00:00 · 2022-09-13 10:01:22 -07:00 · 2022-09-13 09:28:16 -07:00 · 2022-09-12 18:24:53 -04:00 · 2022-09-12 17:28:09 -04:00
37 changed files with 12677 additions and 400 deletions
@@ -2,4 +2,5 @@ cpr_model_server.py
 entrypoint.py
 state_dict.pth
 config.json
-**/__pycache__
+**/__pycache__
+!testdata/**
@@ -2,7 +2,7 @@

 ## About CPR

-CPR ([custom prediction routines](https://github.com/googleapis/python-aiplatform/blob/custom-prediction-routine/google/cloud/aiplatform/prediction/README.md)) is a framework designed by Google Cloud developers to make it easier to combine machine learning models with custom preprocessing and postprocessing logic in a real-time serving application. 
+CPR ([custom prediction routines](https://github.com/googleapis/python-aiplatform/blob/main/google/cloud/aiplatform/prediction/README.md)) is a framework designed by Google Cloud developers to make it easier to combine machine learning models with custom preprocessing and postprocessing logic in a real-time serving application. 

 ## Using this example

@@ -34,6 +34,23 @@ Finally, install the Python modules required to build and run the model server:
 pip install -r requirements.txt
 ```

+### Auth
+
+This example uses Google Cloud Storage for hosting model artifacts and Artifact Registry to store the container image. 
+You'll need to authorize yourself before you can interact with these.
+
+First, log in to GCP with application default credentials:
+```sh
+gcloud auth application-default login
+```
+
+Next, if you haven't done so already, set up the [gcloud credential helper](https://cloud.google.com/artifact-registry/docs/docker/authentication)
+for the Artifact Registry region where you intend to host the image.  
+```
+gcloud auth configure-docker <region>-docker.pkg.dev
+```
+
+
 ### Predictor

 The `TimmPredictor` class in `timm_serving/predictor.py` implements most of the important logic for the server.
@@ -60,9 +60,9 @@ class CPRConfig(object):
    image: str = "timm_predictor:latest"
    artifact_local_dir: str = ""
    region: str = "us-central1"
-    project_id: str = "samthrasher-experimental"
+    project_id: str = "<your project ID here>"
    repository: str = "cpr-images"
-    artifact_gcs_dir: str = "gs://samthrasher-cpr-example/timm-vit224/"
+    artifact_gcs_dir: str = "gs://<your bucket ID here>/timm-vit224/"
    model_name: str = ""
    endpoint_name: str = ""
    machine_type: str = "n1-standard-2"
@@ -5,4 +5,4 @@ timm==0.5.4
 smart_open==6.0.0

 google-cloud-storage>=1.26.0,<2.0.0dev
-google-cloud-aiplatform[prediction] @ git+https://github.com/googleapis/python-aiplatform.git@custom-prediction-routine
+google-cloud-aiplatform[prediction]>=1.16.0
@@ -70,7 +70,10 @@ class PredictorUnitTests(absltest.TestCase):
    def setUp(self):
        super().setUp()
        self.config = CPRConfig()
-        self.config.load()
+        try:
+            self.config.load()
+        except FileNotFoundError:
+            logging.info("No saved config file found, using default values.")
        self.predictor = predictor.TimmPredictor()

    def test_load_from_saved_state_dict_ok(self):
@@ -170,7 +173,10 @@ class ServerEndToEndTests(absltest.TestCase):
    def setUp(self):
        super().setUp()
        self.config = CPRConfig()
-        self.config.load()
+        try:
+            self.config.load()
+        except FileNotFoundError:
+            logging.info("No saved config file found, using default values.")
        self.local_model = cpr.LocalModel(
            serving_container_spec=aiplatform.gapic.ModelContainerSpec(
                image_uri=self.config.image
@@ -0,0 +1 @@
+blah
@@ -17,6 +17,7 @@
 /explainable_ai/SDK_Custom_Container_XAI.ipynb @brianchunkang
 /matching_engine/sdk_matching_engine_for_indexing.ipynb @ivanmkc
 /matching_engine/matching_engine_for_indexing.ipynb @yinghsienwu
+/matching_engine/stream_update_for_matching_engine.ipynb @peterping666
 /sdk/pytorch_lightning_custom_container_training.ipynb @brianchunkang
 /tensorboard @yfang1
 /feature_store @nayaknishant @morgandu
@@ -212,7 +212,7 @@
        "\n",
        "3. [Enable the Vertex AI APIs and Compute Engine APIs.](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component)\n",
        "\n",
-        "4. [Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Google Cloud Notebooks.\n",
+        "4. [Google Cloud SDK](https://cloud.google.com/sdk) is already installed in Vertex AI Workbench Notebooks.\n",
        "\n",
        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -374,15 +374,8 @@
        "### Authenticate your Google Cloud account\n",
        "\n",
        "**If you are using Vertex AI Workbench Notebooks**, your environment is already\n",
-        "authenticated. Skip this step."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "32e1cd21a5d5"
-      },
-      "source": [
+        "authenticated. \n",
+        "\n",
        "**If you are using Colab**, run the cell below and follow the instructions\n",
        "when prompted to authenticate your account via oAuth.\n",
        "\n",
@@ -8,7 +8,7 @@
      },
      "outputs": [],
      "source": [
-        "# Copyright 2021 Google LLC\n",
+        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
@@ -33,19 +33,20 @@
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/community/ml_ops/get_started_with_automl_image_model_batch.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage6/get_started_with_automl_image_model_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/community/ml_ops/get_started_with_automl_image_model_batch.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage6/get_started_with_automl_image_model_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage6/get_started_with_automl_tabular_model_batch.ipynb\">\n",
-        "      Open in Google Cloud Notebooks\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage6/get_started_with_automl_image_model_batch.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
@@ -61,7 +62,7 @@
        "## Overview\n",
        "\n",
        "\n",
-        "This tutorial demonstrates how to use the Vertex SDK to create image classification models and do batch prediction using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model."
+        "This tutorial demonstrates how to use the Vertex AI SDK to create image classification models and do batch prediction using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model."
      ]
    },
    {
@@ -244,7 +245,7 @@
        "\n",
        "3. [Enable the following APIs: Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component,storage-component.googleapis.com)\n",
        "\n",
-        "4. If you are running this notebook locally, you will need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
+        "4. If you are running this notebook locally, you need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
        "\n",
        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -252,6 +253,17 @@
        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
      ]
    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "project_id"
+      },
+      "source": [
+        "#### Set your project ID\n",
+        "\n",
+        "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
+      ]
+    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -711,7 +723,7 @@
        "\n",
        "The `run` method when completed returns the `Model` resource.\n",
        "\n",
-        "The execution of the training pipeline will take upto 20 minutes."
+        "The execution of the training pipeline may take upto 20 minutes."
      ]
    },
    {
@@ -743,7 +755,7 @@
        "\n",
        "Batch prediction provides the ability to do offline batch processing of large amounts of prediction requests. Resources are only provisioned during the batch process and then deprovisioned when the batch request is completed. The results are stored in Cloud Storage, in contrast to online prediction where the results are returned as a HTTP response packet.\n",
        "\n",
-        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server will convert to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
+        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server converts to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
        "\n",
        "### Input format for batch prediction jobs\n",
        "\n",
@@ -752,7 +764,7 @@
        "\n",
        "- JSONL\n",
        "\n",
-        "The batch server accepts the following input formats for AutoML image models:\n",
+        "The batch server accepts the following output formats for AutoML image models:\n",
        "\n",
        "- JSONL\n",
        "\n",
@@ -776,7 +788,7 @@
        "\n",
        "**CSV**\n",
        "\n",
-        "The csv header in the first line will always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
+        "The csv header in the first line is always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
        "\n",
        "    col1,col2,col3\n",
        "    1,3,\"cat1\"\n",
@@ -954,7 +966,7 @@
        "- `prediction_format`: The format of the batch prediction response file: \"jsonl\", \"csv\", \"bigquery\", \"tf-record\", \"tf-record-gzip\" or \"file-list\"\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
-        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
+        "- `sync`: If set to True, the call blocks while waiting for the asynchronous batch job to complete."
      ]
    },
    {
@@ -8,7 +8,7 @@
      },
      "outputs": [],
      "source": [
-        "# Copyright 2021 Google LLC\n",
+        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
@@ -33,12 +33,12 @@
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/community/ml_ops/get_started_with_automl_tabular_model_batch.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage6/get_started_with_automl_tabular_model_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/community/ml_ops/get_started_with_automl_tabular_model_batch.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage6/get_started_with_automl_tabular_model_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
@@ -106,7 +106,7 @@
      "source": [
        "### Dataset\n",
        "\n",
-        "The dataset used for this tutorial is the [Bank Marketing](https://pantheon.corp.google.com/storage/browser/_details/cloud-ml-tables-data/bank-marketing.csv) . This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket."
+        "The dataset used for this tutorial is the [Bank Marketing](https://pantheon.corp.google.com/storage/browser/_details/cloud-ml-tables-data/bank-marketing.csv) . This dataset does not require any feature engineering. The version of the dataset you use in this tutorial is stored in a public Cloud Storage bucket."
      ]
    },
    {
@@ -119,14 +119,11 @@
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
-        "* Vertex AI\n",
-        "* Cloud Storage\n",
+        "- Vertex AI\n",
+        "- Cloud Storage\n",
+        "- BigQuery\n",
        "\n",
-        "Learn about [Vertex AI\n",
-        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
-        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
-        "Calculator](https://cloud.google.com/products/calculator/)\n",
-        "to generate a cost estimate based on your projected usage."
+        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage pricing](https://cloud.google.com/storage/pricing) and [BigQuery pricing](https://cloud.google.com/bigquery/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
      ]
    },
    {
@@ -250,7 +247,7 @@
        "\n",
        "3. [Enable the following APIs: Vertex AI APIs, Compute Engine APIs, and Cloud Storage.](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component,storage-component.googleapis.com)\n",
        "\n",
-        "4. If you are running this notebook locally, you will need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
+        "4. If you are running this notebook locally, you need to install the [Cloud SDK]((https://cloud.google.com/sdk)).\n",
        "\n",
        "5. Enter your project ID in the cell below. Then run the  cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -258,6 +255,17 @@
        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$`."
      ]
    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "project_id"
+      },
+      "source": [
+        "#### Set your project ID\n",
+        "\n",
+        "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
+      ]
+    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -743,7 +751,7 @@
        "\n",
        "The `run` method when completed returns the `Model` resource.\n",
        "\n",
-        "The execution of the training pipeline will take upto 8 hours."
+        "The execution of the training pipeline may take upto 8 hours."
      ]
    },
    {
@@ -776,7 +784,7 @@
        "\n",
        "Batch prediction provides the ability to do offline batch processing of large amounts of prediction requests. Resources are only provisioned during the batch process and then deprovisioned when the batch request is completed. The results are stored in Cloud Storage, in contrast to online prediction where the results are returned as a HTTP response packet.\n",
        "\n",
-        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server will convert to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
+        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server converts to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
        "\n",
        "### Input format for batch prediction jobs\n",
        "\n",
@@ -786,7 +794,9 @@
        "- CSV\n",
        "- Big Query table\n",
        "\n",
-        "The batch server accepts the following input formats for AutoML tabular models:\n",
+        "### Output format for batch prediction jobs\n",
+        "\n",
+        "The batch server accepts the following output formats for AutoML tabular models:\n",
        "\n",
        "- JSONL\n",
        "- CSV\n",
@@ -812,7 +822,7 @@
        "\n",
        "**CSV**\n",
        "\n",
-        "The csv header in the first line will always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
+        "The csv header in the first line is always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
        "\n",
        "    col1,col2,col3\n",
        "    1,3,\"cat1\"\n",
@@ -926,7 +936,7 @@
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `instances_format`: The format for the input instances, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `predictions_format`: The format for the output predictions, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
-        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
+        "- `sync`: If set to True, the call blocks while waiting for the asynchronous batch job to complete."
      ]
    },
    {
@@ -1107,7 +1117,7 @@
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `instances_format`: The format for the input instances, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
        "- `predictions_format`: The format for the output predictions, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
-        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete."
+        "- `sync`: If set to True, the call blocks while waiting for the asynchronous batch job to complete."
      ]
    },
    {
@@ -741,13 +741,16 @@
        "\n",
        "### Input format for batch prediction jobs\n",
        "\n",
-        "The batch server accepts the following input formats:\n",
+        "The batch server accepts the following input formats for custom image models:\n",
        "\n",
        "- JSONL\n",
-        "- CSV\n",
-        "- TFRecords\n",
        "- File-List\n",
-        "- BigQuery table\n",
+        "\n",
+        "### Output format for batch prediction jobs\n",
+        "\n",
+        "The batch server accepts the following output formats for custom image models:\n",
+        "\n",
+        "- JSONL\n",
        "\n",
        "### Pivot format\n",
        "\n",
@@ -1306,7 +1309,6 @@
      "source": [
        "### Send the prediction request\n",
        "\n",
-        "BLAH\n",
        "\n",
        "To make a batch prediction request, call the model object's `batch_predict` method with the following parameters: \n",
        "- `instances_format`: The format of the batch prediction request file: \"jsonl\", \"csv\", \"bigquery\", \"tf-record\", \"tf-record-gzip\" or \"file-list\"\n",
@@ -110,7 +110,7 @@
        "- identity - unique player identitity numbers\n",
        "- demographic features - information about the player, such as the geographic region in which a player is located\n",
        "- behavioral features - counts of the number of times a  player has triggered certain game events, such as reaching a new level\n",
-        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player will churn, i.e. stop being an active player.\n",
+        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player may churn, i.e. stop being an active player.\n",
        "\n",
        "**CSV batch input example**\n",
        "\n",
@@ -574,7 +574,7 @@
        "\n",
        "Learn more about [hardware accelerator support for your region](https://cloud.google.com/vertex-ai/docs/general/locations#accelerators).\n",
        "\n",
-        "*Note*: TF releases before 2.3 for GPU support will fail to load the custom model in this tutorial. It is a known issue and fixed in TF 2.3. This is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
+        "*Note*: TF releases before 2.3 for GPU support fails to load the custom model in this tutorial. It is a known issue and fixed in TF 2.3. This is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
      ]
    },
    {
@@ -638,7 +638,7 @@
        "\n",
        "Next, set the machine type to use for prediction.\n",
        "\n",
-        "- Set the variable `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you will use for for prediction.\n",
+        "- Set the variable `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you use for for prediction.\n",
        " - `machine type`\n",
        "     - `n1-standard`: 3.75GB of memory per vCPU.\n",
        "     - `n1-highmem`: 6.5GB of memory per vCPU\n",
@@ -712,18 +712,23 @@
        "\n",
        "Batch prediction provides the ability to do offline batch processing of large amounts of prediction requests. Resources are only provisioned during the batch process and then deprovisioned when the batch request is completed. The results are stored in Cloud Storage, in contrast to online prediction where the results are returned as a HTTP response packet.\n",
        "\n",
-        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server will convert to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
+        "The input format for your batch job is dependent on the format supported by your model server. Foremost, the web server in your model server must support a JSONL format, which the web server converts to a format support either directly by the model input intertace or a serving function interface. For batch prediction, this JSONL format is referred to as the `pivot` format.\n",
        "\n",
        "### Input format for batch prediction jobs\n",
        "\n",
-        "The batch server accepts the following input formats:\n",
+        "The batch server accepts the following input formats for custom tabular models:\n",
        "\n",
        "- JSONL\n",
        "- CSV\n",
-        "- TFRecords\n",
-        "- File-List\n",
        "- BigQuery table\n",
        "\n",
+        "### Output format for batch prediction jobs\n",
+        "\n",
+        "The batch server accepts the following output formats for custom tabular models:\n",
+        "\n",
+        "- JSONL\n",
+        "- BigQuery table (when input is BigQuery table)\n",
+        "\n",
        "### Pivot format\n",
        "\n",
        "The batch server converts the input format to the `pivot` (JSONL) format as follows:\n",
@@ -744,7 +749,7 @@
        "\n",
        "**CSV**\n",
        "\n",
-        "The csv header in the first line will always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
+        "The csv header in the first line is always be ignored. String fields are required to be double quoted explicitly, otherwise the row is discarded and parsing error messages are outputted to error files. Non-quoted values are always transferred as floats.\n",
        "\n",
        "    col1,col2,col3\n",
        "    1,3,\"cat1\"\n",
@@ -905,7 +910,7 @@
        "- `prediction_format`: The format of the batch prediction response file: \"jsonl\", \"csv\", \"bigquery\", \"tf-record\", \"tf-record-gzip\" or \"file-list\"\n",
        "- `job_display_name`: The human readable name for the prediction job.\n",
        " - `gcs_source`: A list of one or more Cloud Storage paths to your batch prediction requests.\n",
-        "- `gcs_destination_prefix`: The Cloud Storage path that the service will write the predictions to.\n",
+        "- `gcs_destination_prefix`: The Cloud Storage path that the service writes the predictions to.\n",
        "- `model_parameters`: Additional filtering parameters for serving prediction results.\n",
        "- `machine_type`: The type of machine to use for training.\n",
        "- `accelerator_type`: The hardware accelerator type.\n",
@@ -1098,12 +1103,12 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "0d078e5e8953"
+        "id": "776346c90074"
      },
      "outputs": [],
      "source": [
-        "DATA_DIR = \"gs://cloud-samples-data/ai-platform/iris/iris_data.csv\"\n",
-        "! gsutil cat $data_dir | head -n 10 > test.csv\n",
+        "data_file = \"gs://cloud-samples-data/ai-platform/iris/iris_data.csv\"\n",
+        "! gsutil cat $data_file | head -n 10 > test.csv\n",
        "\n",
        "! cat test.csv\n",
        "\n",
@@ -1209,6 +1214,264 @@
        "            break"
      ]
    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "9ce65fff362b"
+      },
+      "source": [
+        "#### Delete the batch prediction job\n",
+        "\n",
+        "You can delete your batch prediction job using the `delete()` method."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5943dfde5123"
+      },
+      "outputs": [],
+      "source": [
+        "batch_prediction_job.delete()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "d61289b87070"
+      },
+      "source": [
+        "## Batch prediction with BigQuery input format\n",
+        "\n",
+        "Next, you do the same batch job, except the input format is a BigQuery table. When the input format is a BigQuery table, the output format has to be a BigQuery table as well. To use BigQuery as the input format, your model must take its input as a list (array) of values. "
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "csv_to_bq"
+      },
+      "source": [
+        "### Create a BigQuery dataset from CSV files\n",
+        "\n",
+        "You can create a BigQuery dataset from CSV files using the BigQuery `create_dataset()` and `load_table_from_uri()` methods, as follows:\n",
+        "\n",
+        "- `create_dataset()`: Creates an empty BigQuery dataset, with the following parameters:\n",
+        " - `dataset_ref`: The `DatasetReference` created from the dataset_id -- e.g., samples.\n",
+        "- `load_table_from_uri()`: Loads one or more CSV files into a table within the corresponding dataset, with the following parameters:\n",
+        " - `url`: A set of one or more CVS files in Cloud Storage storage.\n",
+        " - `table`: The `TableReference` for the table.\n",
+        " - `job_config`: Specifications on how to load the CSV data.\n",
+        "\n",
+        "Learn more about [Importing CSV data into BigQuery](https://www.tensorflow.org/io/tutorials/bigquery#import_census_data_into_bigquery)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "csv_to_bq"
+      },
+      "outputs": [],
+      "source": [
+        "LOCATION = \"us\"\n",
+        "\n",
+        "CSV_SCHEMA = [\n",
+        "    bigquery.SchemaField(\"sepal_length\", \"FLOAT\"),\n",
+        "    bigquery.SchemaField(\"sepal_width\", \"FLOAT\"),\n",
+        "    bigquery.SchemaField(\"petal_length\", \"FLOAT\"),\n",
+        "    bigquery.SchemaField(\"petal_width\", \"FLOAT\"),\n",
+        "]\n",
+        "\n",
+        "DATASET_ID = \"batch\"\n",
+        "TABLE_ID = \"slice1\"\n",
+        "\n",
+        "\n",
+        "def create_bigquery_dataset(dataset_id):\n",
+        "    dataset = bigquery.Dataset(\n",
+        "        bigquery.dataset.DatasetReference(PROJECT_ID, dataset_id)\n",
+        "    )\n",
+        "    dataset.location = \"us\"\n",
+        "\n",
+        "    try:\n",
+        "        dataset = bqclient.create_dataset(dataset)  # API request\n",
+        "        return True\n",
+        "    except Exception as err:\n",
+        "        print(err)\n",
+        "        if err.code != 409:  # http_client.CONFLICT\n",
+        "            raise\n",
+        "    return False\n",
+        "\n",
+        "\n",
+        "def load_data_into_bigquery(url, dataset_id, table_id):\n",
+        "    create_bigquery_dataset(dataset_id)\n",
+        "    dataset = bqclient.dataset(dataset_id)\n",
+        "    table = dataset.table(table_id)\n",
+        "\n",
+        "    job_config = bigquery.LoadJobConfig()\n",
+        "    job_config.write_disposition = bigquery.WriteDisposition.WRITE_TRUNCATE\n",
+        "    job_config.source_format = bigquery.SourceFormat.CSV\n",
+        "    job_config.schema = CSV_SCHEMA\n",
+        "    job_config.skip_leading_rows = 1  # heading\n",
+        "\n",
+        "    load_job = bqclient.load_table_from_uri(url, table, job_config=job_config)\n",
+        "    print(\"Starting job {}\".format(load_job.job_id))\n",
+        "\n",
+        "    load_job.result()  # Waits for table load to complete.\n",
+        "    print(\"Job finished.\")\n",
+        "\n",
+        "    destination_table = bqclient.get_table(table)\n",
+        "    print(\"Loaded {} rows.\".format(destination_table.num_rows))\n",
+        "\n",
+        "    return destination_table\n",
+        "\n",
+        "\n",
+        "bq_table = load_data_into_bigquery(data_file, DATASET_ID, TABLE_ID)\n",
+        "print(bq_table)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "send_prediction_request:image"
+      },
+      "source": [
+        "### Send the batch prediction request\n",
+        "\n",
+        "Again, you make a batch prediction request with the `batch_predict()` method, but with the following changes in parameters:\n",
+        "\n",
+        "- `instances_format`: Set to 'bigquery'\n",
+        "- `predictions_format`: Set to 'bigquery'\n",
+        "- `bigquery_source`: Used instead of `gcs_source`.\n",
+        "- `bigquery_destination_prefix`: Used instead of `gcs_destination_prefix`"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1cf1076178fc"
+      },
+      "outputs": [],
+      "source": [
+        "MIN_NODES = 1\n",
+        "MAX_NODES = 1\n",
+        "\n",
+        "# The name of the job\n",
+        "BATCH_PREDICTION_JOB_NAME = \"churn_batch-\" + UUID\n",
+        "\n",
+        "# Folder in the bucket to write results to\n",
+        "DESTINATION_FOLDER = \"batch_prediction_results_bq\"\n",
+        "\n",
+        "# The Cloud Storage bucket to upload results to\n",
+        "BATCH_PREDICTION_GCS_DEST_PREFIX = BUCKET_URI + \"/\" + DESTINATION_FOLDER\n",
+        "\n",
+        "BQ_TABLE_SOURCE = f\"bq://{PROJECT_ID}.{bq_table.dataset_id}.{bq_table.table_id}\"\n",
+        "\n",
+        "# Make SDK batch_predict method call\n",
+        "batch_prediction_job = model.batch_predict(\n",
+        "    instances_format=\"bigquery\",\n",
+        "    predictions_format=\"bigquery\",\n",
+        "    job_display_name=BATCH_PREDICTION_JOB_NAME,\n",
+        "    bigquery_source=BQ_TABLE_SOURCE,\n",
+        "    bigquery_destination_prefix=f\"bq://{PROJECT_ID}.batch\",\n",
+        "    model_parameters=None,\n",
+        "    machine_type=DEPLOY_COMPUTE,\n",
+        "    accelerator_type=DEPLOY_GPU,\n",
+        "    accelerator_count=DEPLOY_NGPU,\n",
+        "    starting_replica_count=MIN_NODES,\n",
+        "    max_replica_count=MAX_NODES,\n",
+        "    sync=True,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "get_batch_prediction:mbsdk,custom,icn"
+      },
+      "source": [
+        "### Get the predictions\n",
+        "\n",
+        "Next, get the results from the completed batch prediction job.\n",
+        "\n",
+        "The results are written to a BigQuery table at the BigQuery dataset path you specified as the destination. The batch server creates the table, where the table location is specified by:\n",
+        "\n",
+        "`batch_prediction_job.output_info.bigquery_output_dataset`: The project and dataset components.\n",
+        "`batch_prediction_job.output_info.bigquery_output_table`: The table component.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cd515ed616c8"
+      },
+      "outputs": [],
+      "source": [
+        "BQ_RESULTS_TABLE = (\n",
+        "    batch_prediction_job.output_info.bigquery_output_dataset\n",
+        "    + \".\"\n",
+        "    + batch_prediction_job.output_info.bigquery_output_table\n",
+        ")\n",
+        "\n",
+        "print(BQ_RESULTS_TABLE)\n",
+        "\n",
+        "table = bigquery.TableReference.from_string(BQ_RESULTS_TABLE[5:])\n",
+        "\n",
+        "rows = bqclient.list_rows(table, max_results=10)\n",
+        "\n",
+        "for row in rows:\n",
+        "    print(row)\n",
+        "    for key, value in row.items():\n",
+        "        pass"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "c7cda79d0e42"
+      },
+      "source": [
+        "#### Delete the batch prediction job\n",
+        "\n",
+        "You can delete your batch prediction job using the `delete()` method."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "de4b5c638c2a"
+      },
+      "outputs": [],
+      "source": [
+        "batch_prediction_job.delete()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e5d4b9c51e86"
+      },
+      "source": [
+        "#### Delete the model\n",
+        "\n",
+        "You can delete your model using the `delete()` method."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "00fa6a7b4f24"
+      },
+      "outputs": [],
+      "source": [
+        "model.delete()"
+      ]
+    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -1232,16 +1495,6 @@
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
-        "delete_model = True\n",
-        "delete_batch_job = True\n",
-        "\n",
-        "if delete_model:\n",
-        "    try:\n",
-        "        model.delete()\n",
-        "    except Exception as e:\n",
-        "        print(e)\n",
-        "if delete_batch_job:\n",
-        "    batch_prediction_job.delete()\n",
        "\n",
        "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil rm -rf {BUCKET_URI}"
@@ -133,8 +133,6 @@
        "id": "8yVpQt-JHKPF"
      },
      "source": [
-        "## Before you begin\n",
-        "\n",
        "### Set up your local development environment\n",
        "\n",
        "**If you are using Colab or Vertex AI Workbench notebooks**, your environment already meets\n",
@@ -247,6 +245,8 @@
        "id": "BF1j6f9HApxa"
      },
      "source": [
+        "## Before you begin\n",
+        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
@@ -1521,7 +1521,7 @@
  "metadata": {
    "colab": {
      "collapsed_sections": [],
-      "name": "model_monitoring_automl.ipynb",
+      "name": "get_started_with_model_monitoring_automl.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
@@ -35,20 +35,20 @@
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/ai-platform/notebooks/deploy-notebook?name=Model%20Monitoring&download_url=https%3A%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fmodel_monitoring%2Fmodel_monitoring_custom.ipynb\">\n",
-        "       <img src=\"https://www.gstatic.com/cloud/images/navigation/vertex-ai.svg\" alt=\"Google Cloud Notebooks\">Open in Workbench AI Notebook\n",
-        "    </a>\n",
-        "  </td> \n",
-        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/model_monitoring_custom.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage7/model_monitoring_custom.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/model_monitoring_custom.ipynb\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage7/model_monitoring_custom.ipynb\">\n",
        "        <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
+        "  </td> <td>\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage7/model_monitoring_custom.ipynb\">\n",
+        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
+        "      Open in Vertex AI Workbench\n",
+        "    </a>\n",
        "  </td>\n",
        "</table>"
      ]
@@ -110,7 +110,7 @@
        "- identity - unique player identitity numbers\n",
        "- demographic features - information about the player, such as the geographic region in which a player is located\n",
        "- behavioral features - counts of the number of times a  player has triggered certain game events, such as reaching a new level\n",
-        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player will churn, i.e. stop being an active player."
+        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player may churn, i.e. stop being an active player."
      ]
    },
    {
@@ -140,8 +140,6 @@
        "id": "8yVpQt-JHKPF"
      },
      "source": [
-        "## Before you begin\n",
-        "\n",
        "### Set up your local development environment\n",
        "\n",
        "**If you are using Colab or Vertex AI Workbench notebooks**, your environment already meets\n",
@@ -254,6 +252,8 @@
        "id": "BF1j6f9HApxa"
      },
      "source": [
+        "## Before you begin\n",
+        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
@@ -264,7 +264,7 @@
        "\n",
        "1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n",
        "\n",
-        "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
+        "1. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
        "\n",
        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -780,7 +780,7 @@
        "\n",
        "When model monitoring is enabled, the sampled incoming prediction requests are logged into a BigQuery table. The input feature values contained in the logged requests are then analyzed for skew or drift on an specified interval basis. You set a sampling rate to monitor a subset of the production inputs to a model, and the monitoring interval.\n",
        "\n",
-        "The model monitoring service needs to know how to parse the feature values, which is referred to as the input schema. For AutoML tabular models, the input schema is automatically generated. For custom tabular models, the service will attempt to automatically derive the input schema from the first 1000 prediction requests. Alternatively, one can upload the input schema.\n",
+        "The model monitoring service needs to know how to parse the feature values, which is referred to as the input schema. For AutoML tabular models, the input schema is automatically generated. For custom tabular models, the service attempts to automatically derive the input schema from the first 1000 prediction requests. Alternatively, one can upload the input schema.\n",
        "\n",
        "For skew detection, the monitoring service requires a baseline for the statistical distribution of values in the training data. For AutoML tabular models this is automatically derived. For custom tabular models, you upload the training data to the service, and have the service automatically derive the distribution.\n",
        "\n",
@@ -1643,7 +1643,7 @@
  "metadata": {
    "colab": {
      "collapsed_sections": [],
-      "name": "model_monitoring_custom.ipynb",
+      "name": "get_started_with_model_monitoring_custom.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
@@ -110,7 +110,7 @@
        "- identity - unique player identitity numbers\n",
        "- demographic features - information about the player, such as the geographic region in which a player is located\n",
        "- behavioral features - counts of the number of times a  player has triggered certain game events, such as reaching a new level\n",
-        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player will churn, i.e. stop being an active player."
+        "- churn propensity - this is the label or target feature, it provides an estimated probability that this player may churn, i.e. stop being an active player."
      ]
    },
    {
@@ -140,8 +140,6 @@
        "id": "8yVpQt-JHKPF"
      },
      "source": [
-        "## Before you begin\n",
-        "\n",
        "### Set up your local development environment\n",
        "\n",
        "**If you are using Colab or Vertex AI Workbench notebooks**, your environment already meets\n",
@@ -254,6 +252,8 @@
        "id": "BF1j6f9HApxa"
      },
      "source": [
+        "## Before you begin\n",
+        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
@@ -264,7 +264,7 @@
        "\n",
        "1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n",
        "\n",
-        "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
+        "1. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
        "\n",
        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -944,7 +944,7 @@
        "\n",
        "When model monitoring is enabled, the sampled incoming prediction requests are logged into a BigQuery table. The input feature values contained in the logged requests are then analyzed for skew or drift on an specified interval basis. You set a sampling rate to monitor a subset of the production inputs to a model, and the monitoring interval.\n",
        "\n",
-        "The model monitoring service needs to know how to parse the feature values, which is referred to as the input schema. For AutoML tabular models, the input schema is automatically generated. For custom tabular models, the service will attempt to automatically derive the input schema from the first 1000 prediction requests. Alternatively, one can upload the input schema.\n",
+        "The model monitoring service needs to know how to parse the feature values, which is referred to as the input schema. For AutoML tabular models, the input schema is automatically generated. For custom tabular models, the service attempts to automatically derive the input schema from the first 1000 prediction requests. Alternatively, one can upload the input schema.\n",
        "\n",
        "For skew detection, the monitoring service requires a baseline for the statistical distribution of values in the training data. For AutoML tabular models this is automatically derived. For custom tabular models, you upload the training data to the service, and have the service automatically derive the distribution.\n",
        "\n",
@@ -1859,7 +1859,7 @@
  "metadata": {
    "colab": {
      "collapsed_sections": [],
-      "name": "model_monitoring_custom_tf_serving.ipynb",
+      "name": "get_started_with_model_monitoring_custom_tf_serving.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
@@ -140,8 +140,6 @@
        "id": "8yVpQt-JHKPF"
      },
      "source": [
-        "## Before you begin\n",
-        "\n",
        "### Set up your local development environment\n",
        "\n",
        "**If you are using Colab or Vertex AI Workbench notebooks**, your environment already meets\n",
@@ -254,6 +252,8 @@
        "id": "BF1j6f9HApxa"
      },
      "source": [
+        "## Before you begin\n",
+        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
@@ -1697,7 +1697,7 @@
  "metadata": {
    "colab": {
      "collapsed_sections": [],
-      "name": "model_monitoring_setup.ipynb",
+      "name": "get_started_with_model_monitoring_setup.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
@@ -56,7 +56,7 @@ def parse_notebook(path):
        # cell 1 is copyright
        nth = 0
        cell, nth = get_cell(path, cells, nth)
-        if not cell['source'][0].startswith('# Copyright'):
+        if not 'Copyright' in cell['source'][0]:
            report_error(path, 0, "missing copyright cell")
            
        # check for notices
@@ -16,6 +16,7 @@
 /model_monitoring @andrewferlitsch
 /tensorboard @zbl94

+/bigquery_ml/bqml-online-prediction.ipynb @polong-lin
 /model_monitoring/model_monitoring.ipynb @mco-gh
 /ml_metadata/sdk-metric-parameter-tracking-for-custom-jobs.ipynb @jialuzh
 /ml_metadata/sdk-metric-parameter-tracking-for-locally-trained-models.ipynb @jialuzh
@@ -28,6 +29,9 @@
 /automl/automl_forecasting_bqml_arima_plus_comparison.ipynb @TheMichaelHu
 /automl/automl_tabular_on_vertex_pipelines.ipynb @helinwang
 /custom/custom_training_tensorboard_profiler.ipynb @itseric
-/workbench/spark/spark_sample_notebook.ipynb @bmiro
+/workbench/spark/spark_sample_notebook.ipynb @bradmiro
+/workbench/spark/spark_ml.ipynb @bradmiro
 /model-registry/bqml-vertexai-model-registry.ipynb @soheilazangeneh
 /workbench/exploratory_data_analysis/explore_data_in_bigquery_with_workbench.ipynb @alokpattani
+/model_evaluation/automl_tabular_classification_model_evaluation.ipynb @soheilazangeneh
+/model_evaluation/automl_tabular_regression_model_evaluation.ipynb @soheilazangeneh
@@ -32,18 +32,18 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/bigquery_ml/bqml-online-prediction.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/official/bigquery_ml/bqml-online-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/bigquery_ml/bqml-online-prediction.ipynb\">\n",
+        "    <a href=\"https://github/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/official/bigquery_ml/bqml-online-prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/community/bigquery_ml/bqml-online-prediction.ipynb\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/blob/master/notebooks/official/bigquery_ml/bqml-online-prediction.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -63,7 +63,7 @@
        "\n",
        "### Dataset\n",
        "\n",
-        "The dataset, [available publicly on BigQuery](https://console.cloud.google.com/bigquery?project=bigquery-public-data&d=ga4_obfuscated_sample_ecommerce&p=bigquery-public-data&page=dataset), comes from obfuscated [Google Analytics 4 data](https://support.google.com/analytics/answer/10937659) from the [Google Merchandise Store](https://shop.googlemerchandisestore.com/).\n",
+        "The dataset, <a href=\"https://console.cloud.google.com/bigquery?project=bigquery-public-data&d=ga4_obfuscated_sample_ecommerce&p=bigquery-public-data&page=dataset\" target=\"_blank\">available publicly on BigQuery</a>, comes from obfuscated <a href=\"https://support.google.com/analytics/answer/10937659\" target=\"_blank\">Google Analytics 4 data</a> from the <a href=\"https://shop.googlemerchandisestore.com/\" target=\"_blank\">Google Merchandise Store</a>).\n",
        "\n",
        "### Objective\n",
        "\n",
@@ -95,9 +95,9 @@
        "* Vertex AI\n",
        "\n",
        "\n",
-        "Learn about [BigQuery Pricing](https://cloud.google.com/bigquery/pricing), [BigQuery ML pricing](https://cloud.google.com/bigquery-ml/pricing), [Vertex AI\n",
-        "pricing](https://cloud.google.com/vertex-ai/pricing), and use the [Pricing\n",
-        "Calculator](https://cloud.google.com/products/calculator/)\n",
+        "Learn about <a href=\"https://cloud.google.com/bigquery/pricing\" target=\"_blank\">BigQuery Pricing</a>, <a href=\"https://cloud.google.com/bigquery-ml/pricing\" target=\"_blank\">BigQuery ML pricing</a>, <a href=\"https://cloud.google.com/vertex-ai/pricing\" target=\"_blank\">Vertex AI\n",
+        "pricing</a>, and use the <a href=\"https://cloud.google.com/products/calculator/\" target=\"_blank\">Pricing\n",
+        "Calculator</a>\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
@@ -128,18 +128,18 @@
        "* virtualenv\n",
        "* Jupyter notebook running in a virtual environment with Python 3\n",
        "\n",
-        "The Google Cloud guide to [Setting up a Python development\n",
-        "environment](https://cloud.google.com/python/setup) and the [Jupyter\n",
-        "installation guide](https://jupyter.org/install) provide detailed instructions\n",
+        "The Google Cloud guide to <a href=\"https://cloud.google.com/python/setup\" target=\"_blank\">Setting up a Python development\n",
+        "environment</a> and the <a href=\"https://jupyter.org/install\" target=\"_blank\">Jupyter\n",
+        "installation guide</a> provide detailed instructions\n",
        "for meeting these requirements. The following steps provide a condensed set of\n",
        "instructions:\n",
        "\n",
-        "1. [Install and initialize the Cloud SDK.](https://cloud.google.com/sdk/docs/)\n",
+        "1. <a href=\"https://cloud.google.com/sdk/docs/\" target=\"_blank\">Install and initialize the Cloud SDK.</a>\n",
        "\n",
-        "1. [Install Python 3.](https://cloud.google.com/python/setup#installing_python)\n",
+        "1. <a href=\"https://cloud.google.com/python/setup#installing_python\" target=\"_blank\">Install Python 3.</a>\n",
        "\n",
-        "1. [Install\n",
-        "   virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv)\n",
+        "1. <a href=\"https://cloud.google.com/python/setup#installing_and_using_virtualenv\" target=\"_blank\">Install\n",
+        "   virtualenv</a>\n",
        "   and create a virtual environment that uses Python 3. Activate the virtual environment.\n",
        "\n",
        "1. To install Jupyter, run `pip3 install jupyter` on the\n",
@@ -234,13 +234,13 @@
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
-        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
+        "1. <a href=\"https://console.cloud.google.com/cloud-resource-manager\" target=\"_blank\">Select or create a Google Cloud project</a>. When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
-        "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
+        "1. <a href=\"https://cloud.google.com/billing/docs/how-to/modify-project\" target=\"_blank\">Make sure that billing is enabled for your project</a>.\n",
        "\n",
-        "1. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
+        "1. <a href=\"https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com\" target=\"_blank\">Enable the Vertex AI API</a>.\n",
        "\n",
-        "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
+        "1. If you are running this notebook locally, you will need to install the <a href=\"https://cloud.google.com/sdk\" target=\"_blank\">Cloud SDK</a>.\n",
        "\n",
        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
@@ -267,7 +267,7 @@
      },
      "outputs": [],
      "source": [
-        "PROJECT_ID = \"YOUR-PROJECT-ID\"\n",
+        "PROJECT_ID = \"[YOUR-PROJECT-ID]\"\n",
        "\n",
        "# Get your Google Cloud project ID from gcloud\n",
        "import os\n",
@@ -314,9 +314,9 @@
        "- Europe: `europe-west4`\n",
        "- Asia Pacific: `asia-east1`\n",
        "\n",
-        "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
+        "You might not be able to use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
        "\n",
-        "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
+        "Learn more about <a href=\"https://cloud.google.com/vertex-ai/docs/general/locations\" target=\"_blank\">Vertex AI regions</a>."
      ]
    },
    {
@@ -339,9 +339,9 @@
        "id": "06571eb4063b"
      },
      "source": [
-        "#### Timestamp\n",
+        "#### UUID\n",
        "\n",
-        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial."
+        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
@@ -352,9 +352,16 @@
      },
      "outputs": [],
      "source": [
-        "from datetime import datetime\n",
+        "import random\n",
+        "import string\n",
        "\n",
-        "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
+        "\n",
+        "# Generate a uuid of a specifed length(default=8)\n",
+        "def generate_uuid(length: int = 8) -> str:\n",
+        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
+        "\n",
+        "\n",
+        "UUID = generate_uuid()"
      ]
    },
    {
@@ -380,8 +387,7 @@
        "\n",
        "**Otherwise**, follow these steps:\n",
        "\n",
-        "1. In the Cloud Console, go to the [**Create service account key**\n",
-        "   page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n",
+        "1. In the Cloud Console, go to the <a href=\"https://console.cloud.google.com/apis/credentials/serviceaccountkey\" target=\"_blank\">**Create service account key** page</a>.\n",
        "\n",
        "2. Click **Create service account**.\n",
        "\n",
@@ -486,7 +492,10 @@
      },
      "outputs": [],
      "source": [
+        "from typing import Union\n",
+        "\n",
        "import google.cloud.aiplatform as vertex_ai\n",
+        "import pandas as pd\n",
        "from google.cloud import bigquery"
      ]
    },
@@ -550,24 +559,17 @@
      "outputs": [],
      "source": [
        "# Wrapper to use BigQuery client to run query/job, return job ID or result as DF\n",
-        "def bq_query(sql):\n",
+        "def run_bq_query(sql: str) -> Union[str, pd.DataFrame]:\n",
        "    \"\"\"\n",
        "    Input: SQL query, as a string, to execute in BigQuery\n",
        "    Returns the query results as a pandas DataFrame, or error, if any\n",
        "    \"\"\"\n",
-        "    # Import Exceptions library to help with dataset error catching\n",
-        "    from google.cloud.exceptions import BadRequest\n",
        "\n",
        "    # Try dry run before executing query to catch any errors\n",
-        "    try:\n",
-        "        job_config = bigquery.QueryJobConfig(dry_run=True, use_query_cache=False)\n",
-        "\n",
-        "        bq_client.query(sql, job_config=job_config)\n",
-        "\n",
-        "    except BadRequest as err:\n",
-        "        print(err)\n",
-        "        return\n",
+        "    job_config = bigquery.QueryJobConfig(dry_run=True, use_query_cache=False)\n",
+        "    bq_client.query(sql, job_config=job_config)\n",
        "\n",
+        "    # If dry run succeeds without errors, proceed to run query\n",
        "    job_config = bigquery.QueryJobConfig()\n",
        "    client_result = bq_client.query(sql, job_config=job_config)\n",
        "\n",
@@ -589,7 +591,7 @@
        "\n",
        "BigQuery ML (BQML) provides the capability to train ML tabular models, such as classification, regression, forecasting, and matrix factorization, in BigQuery using SQL syntax directly. BigQuery ML uses the scalable infrastructure of BigQuery ML so you don't need to set up additional infrastructure for training or batch serving.\n",
        "\n",
-        "Learn more about [BigQuery ML documentation](https://cloud.google.com/bigquery-ml/docs)."
+        "Learn more about <a href=\"https://cloud.google.com/bigquery-ml/docs\" target=\"_blank\">BigQuery ML documentation</a>."
      ]
    },
    {
@@ -600,9 +602,13 @@
      },
      "outputs": [],
      "source": [
-        "BQ_DATASET_NAME = \"ga4_churnprediction\"\n",
+        "BQ_DATASET_NAME = f\"ga4_churnprediction_{UUID}\"\n",
        "\n",
-        "bq_query(f\"\"\"CREATE SCHEMA IF NOT EXISTS {BQ_DATASET_NAME}\"\"\")"
+        "sql_create_dataset = f\"\"\"CREATE SCHEMA IF NOT EXISTS {BQ_DATASET_NAME}\"\"\"\n",
+        "\n",
+        "print(sql_create_dataset)\n",
+        "\n",
+        "run_bq_query(sql_create_dataset)"
      ]
    },
    {
@@ -620,7 +626,7 @@
        "id": "49dd00d5fbe5"
      },
      "source": [
-        "Inpect data that has been pre-processed from [Google Analytics 4 data from the Google Merchandise Store](https://support.google.com/analytics/answer/10937659) so that it can be used for classification. For more information on how this data was prepared, read [this blog post](https://cloud.google.com/blog/topics/developers-practitioners/churn-prediction-game-developers-using-google-analytics-4-ga4-and-bigquery-ml).\n",
+        "Inpect data that has been pre-processed from <a href=\"https://support.google.com/analytics/answer/10937659\" target=\"_blank\">Google Analytics 4 data from the Google Merchandise Store</a> so that it can be used for classification. For more information on how this data was prepared, read <a href=\"https://cloud.google.com/blog/topics/developers-practitioners/churn-prediction-game-developers-using-google-analytics-4-ga4-and-bigquery-ml\" target=\"_blank\">this blog post</a>.\n",
        "\n",
        "As seen below, each row represents a single user, and the columns represent their demographic features, their aggregated behavioral features in the first 24 hours of visiting the Google Merchandise Store, and the label (whether the user churned or returned any time after the first 24 hours)."
      ]
@@ -641,7 +647,7 @@
        "LIMIT\n",
        "    100\n",
        "\"\"\"\n",
-        "bq_query(sql_inspect)"
+        "run_bq_query(sql_inspect)"
      ]
    },
    {
@@ -662,9 +668,9 @@
        "The query below trains a logistic regression model using BigQuery ML. BigQuery resources are used to train the model.\n",
        "\n",
        "In the `OPTIONS` parameter:\n",
-        "* with `model_registry=\"vertex_ai\"`, the BigQuery ML model will automatically be [registered to Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/model-registry-bqml), which enables you to view all of your registered models and its versions on Google Cloud in one place.\n",
+        "* with `model_registry=\"vertex_ai\"`, the BigQuery ML model will automatically be <a href=\"https://cloud.google.com/vertex-ai/docs/model-registry/model-registry-bqml\" target=\"_blank\">registered to Vertex AI Model Registry</a>, which enables you to view all of your registered models and its versions on Google Cloud in one place.\n",
        "\n",
-        "* `vertex_ai_model_version_aliases allows you to set aliases to help you keep track of your model version ([documentation](https://cloud.google.com/vertex-ai/docs/model-registry/model-alias))."
+        "* `vertex_ai_model_version_aliases allows you to set aliases to help you keep track of your model version (<a href=\"https://cloud.google.com/vertex-ai/docs/model-registry/model-alias\" target=\"_blank\">documentation</a>)."
      ]
    },
    {
@@ -677,7 +683,7 @@
      "source": [
        "# this cell may take ~1 min to run\n",
        "\n",
-        "BQML_MODEL_NAME = \"bqmlmodelchurn\"\n",
+        "BQML_MODEL_NAME = f\"bqml_model_churn_{UUID}\"\n",
        "\n",
        "sql_train_model_bqml = f\"\"\"\n",
        "CREATE OR REPLACE MODEL {BQ_DATASET_NAME}.{BQML_MODEL_NAME}    \n",
@@ -696,7 +702,7 @@
        "\n",
        "print(sql_train_model_bqml)\n",
        "\n",
-        "bq_query(sql_train_model_bqml)"
+        "run_bq_query(sql_train_model_bqml)"
      ]
    },
    {
@@ -714,7 +720,7 @@
        "id": "2aaaae772f67"
      },
      "source": [
-        "With the model created, you can now evaluate the logistic regression model. Behind the scenes, BigQuery ML automatically [split the data](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create#data_split_method), which makes it easier to quickly train and evaluate models."
+        "With the model created, you can now evaluate the logistic regression model. Behind the scenes, BigQuery ML automatically <a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create#data_split_method\" target=\"_blank\">split the data</a>, which makes it easier to quickly train and evaluate models."
      ]
    },
    {
@@ -734,7 +740,7 @@
        "\n",
        "print(sql_evaluate_model)\n",
        "\n",
-        "bq_query(sql_evaluate_model)"
+        "run_bq_query(sql_evaluate_model)"
      ]
    },
    {
@@ -745,7 +751,7 @@
      "source": [
        "These metrics help you understand the performance of the model. \n",
        "\n",
-        "There are various metrics for logistic regression and other model types (full list of metrics can be found in the [documentation](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-evaluate#mlevaluate_output))."
+        "There are various metrics for logistic regression and other model types (full list of metrics can be found in the <a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-evaluate#mlevaluate_output\" target=\"_blank\">documentation</a>)."
      ]
    },
    {
@@ -765,7 +771,7 @@
      "source": [
        "Make a batch prediction in BigQuery ML on the original training data to check the probability of churn for each of the users, as seen in the `probability` column, with the predicted label under the `predicted_churn` column.\n",
        "\n",
-        "[ML.EXPLAIN_PREDICT](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-explain-predict) has built-in [Explainable AI](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-xai-overview). This allows you to see the top contributing features to each prediction and interpret how it was computed."
+        "<a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-explain-predict\" target=\"_blank\">ML.EXPLAIN_PREDICT</a> has built-in <a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-xai-overview\" target=\"_blank\">Explainable AI</a>. This allows you to see the top contributing features to each prediction and interpret how it was computed."
      ]
    },
    {
@@ -787,7 +793,7 @@
        "\n",
        "print(sql_explain_predict)\n",
        "\n",
-        "bq_query(sql_explain_predict)"
+        "run_bq_query(sql_explain_predict)"
      ]
    },
    {
@@ -796,7 +802,7 @@
        "id": "fa1f96c0f452"
      },
      "source": [
-        "Since the `top_feature_attributions` is a nested column, you can unnest the array ([documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/arrays)) into separate rows for each of the features. In other words, since ML.EXPLAIN_PREDICT provides the top 5 most important features, using `UNNEST` results in 5 rows per prediction:"
+        "Since the `top_feature_attributions` is a nested column, you can unnest the array (<a href=\"https://cloud.google.com/bigquery/docs/reference/standard-sql/arrays\" target=\"_blank\">documentation</a>) into separate rows for each of the features. In other words, since ML.EXPLAIN_PREDICT provides the top 5 most important features, using `UNNEST` results in 5 rows per prediction:"
      ]
    },
    {
@@ -827,7 +833,7 @@
        "\n",
        "print(sql_explain_predict)\n",
        "\n",
-        "bq_query(sql_explain_predict)"
+        "run_bq_query(sql_explain_predict)"
      ]
    },
    {
@@ -847,7 +853,7 @@
      "source": [
        "When the model was trained in BigQuery ML, the line `model_registry=\"vertex_ai\"` registered the model to Vertex AI Model Registry automatically upon completion.\n",
        "\n",
-        "You can view the model on the [Vertex AI Model Registry page](https://console.cloud.google.com/vertex-ai/models), or use the code below to check that it was successfully registered:"
+        "You can view the model on the <a href=\"https://console.cloud.google.com/vertex-ai/models\" target=\"_blank\">Vertex AI Model Registry page</a>, or use the code below to check that it was successfully registered:"
      ]
    },
    {
@@ -858,12 +864,7 @@
      },
      "outputs": [],
      "source": [
-        "print(f\"BQML_MODEL_NAME = {BQML_MODEL_NAME}\")\n",
-        "\n",
-        "models = vertex_ai.Model.list(\n",
-        "    filter=f\"display_name={BQML_MODEL_NAME}\", order_by=\"update_time\"\n",
-        ")\n",
-        "model = models[0]\n",
+        "model = vertex_ai.Model(model_name=BQML_MODEL_NAME)\n",
        "\n",
        "print(model.gca_resource)"
      ]
@@ -883,7 +884,7 @@
        "id": "b6120dcc1ff6"
      },
      "source": [
-        "While BigQuery ML supports batch prediction with [ML.PREDICT](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict) and [ML.EXPLAIN_PREDICT](https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-explain-predict), BigQuery ML is not suitable for real-time predictions where you need low latency predictions with potentially high frequency of requests.\n",
+        "While BigQuery ML supports batch prediction with <a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict\" target=\"_blank\">ML.PREDICT</a> and <a href=\"https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-explain-predict\" target=\"_blank\">ML.EXPLAIN_PREDICT</a>, BigQuery ML is not suitable for real-time predictions where you need low latency predictions with potentially high frequency of requests.\n",
        "\n",
        "In other words, deploying the BigQuery ML model to an endpoint enables you to do online predictions."
      ]
@@ -906,30 +907,6 @@
        "To deploy your model to an endpoint, you will first need to create an endpoint before you deploy the model to it."
      ]
    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "3ce73125dff6"
-      },
-      "outputs": [],
-      "source": [
-        "def create_endpoint(\n",
-        "    project: str,\n",
-        "    display_name: str,\n",
-        "    location: str,\n",
-        "):\n",
-        "    endpoint = vertex_ai.Endpoint.create(\n",
-        "        display_name=display_name,\n",
-        "        project=project,\n",
-        "        location=location,\n",
-        "    )\n",
-        "\n",
-        "    print(endpoint.display_name)\n",
-        "    print(endpoint.resource_name)\n",
-        "    return endpoint"
-      ]
-    },
    {
      "cell_type": "code",
      "execution_count": null,
@@ -938,17 +915,16 @@
      },
      "outputs": [],
      "source": [
-        "endpoint_name = f\"{BQML_MODEL_NAME}-{TIMESTAMP}\"\n",
+        "ENDPOINT_NAME = f\"{BQML_MODEL_NAME}-endpoint\"\n",
        "\n",
-        "print(\n",
-        "    f\"\"\"\n",
-        "PROJECT_ID: {PROJECT_ID},\n",
-        "endpoint_name: {endpoint_name}\n",
-        "REGION: {REGION}\n",
-        "\"\"\"\n",
+        "endpoint = vertex_ai.Endpoint.create(\n",
+        "    display_name=ENDPOINT_NAME,\n",
+        "    project=PROJECT_ID,\n",
+        "    location=REGION,\n",
        ")\n",
        "\n",
-        "create_endpoint(PROJECT_ID, endpoint_name, REGION)"
+        "print(endpoint.display_name)\n",
+        "print(endpoint.resource_name)"
      ]
    },
    {
@@ -966,31 +942,7 @@
        "id": "951ed1693f6b"
      },
      "source": [
-        "List the endpoints to make sure it has successfully been created. You can also view your endpoints on the [Vertex AI Endpoints page](https://console.cloud.google.com/vertex-ai/endpoints?project=polong-contentdev)."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "0a9bad8d9ad4"
-      },
-      "outputs": [],
-      "source": [
-        "endpoint = vertex_ai.Endpoint.list(\n",
-        "    # filter=f'display_name={endpoint_name}', # optional: filter by specific endpoint name\n",
-        "    order_by=\"update_time\"\n",
-        ")\n",
-        "endpoint[-1]"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "2431a4d28d97"
-      },
-      "source": [
-        "Retrieve the endpoint id so you can use it in the next step."
+        "List the endpoints to make sure it has successfully been created. (You can also view your endpoints on the <a href=\"https://console.cloud.google.com/vertex-ai/endpoints\" target=\"_blank\">Vertex AI Endpoints page</a>)."
      ]
    },
    {
@@ -1001,7 +953,7 @@
      },
      "outputs": [],
      "source": [
-        "endpoint[-1].to_dict()"
+        "endpoint.list()"
      ]
    },
    {
@@ -1019,74 +971,19 @@
        "id": "6a90be5b77a2"
      },
      "source": [
-        "With the model, you can now deploy it to an endpoint. "
+        "With the new endpoint, you can now deploy your model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "af323ea42c5b"
-      },
-      "outputs": [],
-      "source": [
-        "from typing import Dict, Optional, Sequence, Tuple\n",
-        "\n",
-        "\n",
-        "def deploy_model_with_automatic_resources_sample(\n",
-        "    project,\n",
-        "    location,\n",
-        "    model_name: str,\n",
-        "    endpoint: Optional[vertex_ai.Endpoint] = None,\n",
-        "    deployed_model_display_name: Optional[str] = None,\n",
-        "    traffic_percentage: Optional[int] = 0,\n",
-        "    traffic_split: Optional[Dict[str, int]] = None,\n",
-        "    min_replica_count: int = 1,\n",
-        "    max_replica_count: int = 1,\n",
-        "    metadata: Optional[Sequence[Tuple[str, str]]] = (),\n",
-        "    sync: bool = True,\n",
-        "):\n",
-        "    \"\"\"\n",
-        "    model_name: A fully-qualified model resource name or model ID.\n",
-        "          Example: \"projects/123/locations/us-central1/models/456\" or\n",
-        "          \"456\" when project and location are initialized or passed.\n",
-        "    \"\"\"\n",
-        "\n",
-        "    model = vertex_ai.Model(model_name=model_name)\n",
-        "\n",
-        "    model.deploy(\n",
-        "        endpoint=endpoint,\n",
-        "        deployed_model_display_name=deployed_model_display_name,\n",
-        "        traffic_percentage=traffic_percentage,\n",
-        "        traffic_split=traffic_split,\n",
-        "        min_replica_count=min_replica_count,\n",
-        "        max_replica_count=max_replica_count,\n",
-        "        metadata=metadata,\n",
-        "        sync=sync,\n",
-        "    )\n",
-        "\n",
-        "    model.wait()\n",
-        "\n",
-        "    print(model.display_name)\n",
-        "    print(model.resource_name)\n",
-        "    return"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "9e6763369af4"
+        "id": "c70ecc568ee5"
      },
      "outputs": [],
      "source": [
        "# deploying the model to the endpoint may take 10-15 minutes\n",
-        "deploy_model_with_automatic_resources_sample(\n",
-        "    project=PROJECT_ID,\n",
-        "    location=REGION,\n",
-        "    model_name=BQML_MODEL_NAME,\n",
-        "    endpoint=endpoint[-1],\n",
-        ")"
+        "model.deploy(endpoint=endpoint)"
      ]
    },
    {
@@ -1095,7 +992,7 @@
        "id": "c303d779477b"
      },
      "source": [
-        "You can also check on the status of your model by visiting the [Vertex AI Endpoints page](https://console.cloud.google.com/vertex-ai/endpoints)."
+        "You can also check on the status of your model by visiting the <a href=\"https://console.cloud.google.com/vertex-ai/endpoints\" target=\"_blank\">Vertex AI Endpoints page</a>."
      ]
    },
    {
@@ -1168,35 +1065,12 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "2c6093ce9f8a"
+        "id": "b4839f31d2f8"
      },
      "outputs": [],
      "source": [
-        "def endpoint_predict_sample(\n",
-        "    project: str, location: str, instances: list, endpoint: str\n",
-        "):\n",
-        "    endpoint = vertex_ai.Endpoint(endpoint)\n",
-        "\n",
-        "    prediction = endpoint.predict(instances=instances)\n",
-        "    return prediction"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "0c41fd6eeb6f"
-      },
-      "outputs": [],
-      "source": [
-        "prediction_response = endpoint_predict_sample(\n",
-        "    project=PROJECT_ID,\n",
-        "    location=REGION,\n",
-        "    instances=df_sample_requests_list,\n",
-        "    endpoint=endpoint[-1].name,\n",
-        ")\n",
-        "\n",
-        "prediction_response"
+        "prediction = endpoint.predict(df_sample_requests_list)\n",
+        "print(prediction)"
      ]
    },
    {
@@ -1216,7 +1090,7 @@
      },
      "outputs": [],
      "source": [
-        "prediction_response.predictions"
+        "prediction.predictions"
      ]
    },
    {
@@ -1227,8 +1101,8 @@
      "source": [
        "## Cleaning up\n",
        "\n",
-        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
-        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
+        "To clean up all Google Cloud resources used in this project, you can <a href=\"https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects\" target=\"_blank\">delete the Google Cloud\n",
+        "project</a> you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:"
      ]
@@ -1241,18 +1115,12 @@
      },
      "outputs": [],
      "source": [
-        "# MODEL_ID = model.name\n",
+        "# Undeploy model from endpoint and delete endpoint\n",
+        "endpoint.undeploy_all()\n",
+        "endpoint.delete()\n",
        "\n",
-        "ENDPOINT_ID = int(endpoint[-1].name)\n",
-        "\n",
-        "# Undeploy model from endpoint\n",
-        "endpoint[-1].undeploy_all()\n",
-        "\n",
-        "# Delete endpoint resource\n",
-        "! gcloud ai endpoints delete $ENDPOINT_ID --quiet --region $REGION\n",
-        "\n",
-        "# Delete BigQuery ML model\n",
-        "! bq rm -f --model $PROJECT_ID\\:$BQ_DATASET_NAME\\.$BQML_MODEL_NAME"
+        "# Delete BigQuery dataset, including the BigQuery ML model\n",
+        "! bq rm -r -f $PROJECT_ID:$BQ_DATASET_NAME"
      ]
    }
  ],
@@ -263,7 +263,7 @@
        "\n",
        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
        "\n",
-        "3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
+        "3. [Enable the following APIs: Vertex AI API, Cloud Resource Manager API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,cloudresourcemanager.googleapis.com).\n",
        "\n",
        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
        "\n",
@@ -32,18 +32,18 @@
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery-ml/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model-registry/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery-ml/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
+        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model-registry/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery-ml/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
+        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model-registry/bqml-vertexai-model-registry.ipynb\" target=\"_blank\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -677,7 +677,7 @@
      "source": [
        "### Find the model in the Vertex Model Registry\n",
        "\n",
-        "You can use the `Vertex AI Model list()` method with a filter query to find the automatically registered model."
+        "You can use the `Vertex AI Model()` method with `model_name` parameter to find the automatically registered model."
      ]
    },
    {
@@ -223,12 +223,22 @@
        "if IS_WORKBENCH_NOTEBOOK:\n",
        "    USER_FLAG = \"--user\"\n",
        "\n",
+        "# Don't bother installing tensorflow or explainable_ai_sdk on Colab\n",
+        "extra_pkgs = \"tensorflow explainable_ai_sdk\"\n",
+        "if \"google.colab\" in sys.modules:\n",
+        "  extra_pkgs = \"\"\n",
+        "\n",
        "# Install required packages.\n",
-        "! pip3 install {USER_FLAG} --quiet --upgrade google-cloud-aiplatform\n",
-        "! pip3 install {USER_FLAG} --quiet --upgrade tensorflow\n",
-        "! pip3 install {USER_FLAG} --quiet --upgrade explainable_ai_sdk\n",
-        "! pip3 install {USER_FLAG} --quiet --upgrade google-api-python-client google-auth-oauthlib google-auth-httplib2 oauth2client requests\n",
-        "! pip3 install {USER_FLAG} --quiet --upgrade google-cloud-storage==1.32.0"
+        "! pip3 install {USER_FLAG} \\\n",
+        "    google-cloud-aiplatform \\\n",
+        "    explainable_ai_sdk \\\n",
+        "    $extra_pkgs \\\n",
+        "    google-api-python-client \\\n",
+        "    google-auth-oauthlib \\\n",
+        "    google-auth-httplib2 \\\n",
+        "    oauth2client \\\n",
+        "    requests \\\n",
+        "    google-cloud-storage==1.32.0"
      ]
    },
    {
@@ -562,7 +572,7 @@
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
-        "id": "init_aip:mbsdk,region"
+        "id": "wGa5T9eRR8Mz"
      },
      "outputs": [],
      "source": [
@@ -81,7 +81,6 @@
        "The steps performed include:\n",
        "\n",
        "- Define and compile a `Vertex AI` pipeline.\n",
-        "- Schedule a recurring pipeline run.\n",
        "- Specify which service account to use for a pipeline run."
      ]
    },
@@ -97,13 +96,9 @@
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
-        "* Cloud Functions\n",
-        "* Cloud Scheduler\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
        "[Cloud Storage pricing](https://cloud.google.com/storage/pricing),\n",
-        "[Cloud Functions pricing](ttps://cloud.google.com/functions/pricing), and\n",
-        "[Clould Scheduler pricing]((https://cloud.google.com/scheduler/pricing)),\n",
        "and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
@@ -926,69 +921,6 @@
        "job.delete()"
      ]
    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "schedule_pipeline_run"
-      },
-      "source": [
-        "## Recurring pipeline runs: create a scheduled pipeline job\n",
-        "\n",
-        "This section shows how to create a **scheduled pipeline job**.  You do this using the pipeline you already defined.\n",
-        "\n",
-        "Under the hood, the scheduled jobs are supported by the Cloud Scheduler and a Cloud Functions function.  Check first that the APIs for both of these services are enabled.\n",
-        "You will need to first enable the [enable the Cloud Scheduler API](http://console.cloud.google.com/apis/library/cloudscheduler.googleapis.com) and the [Cloud Functions and Cloud Build APIs](https://console.cloud.google.com/flows/enableapi?apiid=cloudfunctions,cloudbuild.googleapis.com) if you have not already done so.\n",
-        "Note:you need to [create an App Engine app for your project](https://cloud.google.com/scheduler/docs/quickstart) if one does not already exist.\n",
-        "\n",
-        "\n",
-        "See the [Cloud Scheduler](https://cloud.google.com/scheduler/docs/configuring/cron-job-schedules) documentation for more on the cron syntax.\n",
-        "\n",
-        "Create a scheduled pipeline job, passing as an argument the job specification file that you compiled above.\n",
-        "\n",
-        "*Note:* You can pass a `parameter_values` dict that specifies the pipeline input parameters you want to use."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "Ty5hDoNX2Ou8"
-      },
-      "outputs": [],
-      "source": [
-        "if not os.getenv(\"IS_TESTING\"):\n",
-        "    from kfp.v2.google.client import AIPlatformClient  # noqa: F811\n",
-        "\n",
-        "    api_client = AIPlatformClient(project_id=PROJECT_ID, region=REGION)\n",
-        "\n",
-        "    # adjust time zone and cron schedule as necessary\n",
-        "    response = api_client.create_schedule_from_job_spec(\n",
-        "        job_spec_path=\"intro_pipeline.json\",\n",
-        "        schedule=\"2 * * * *\",\n",
-        "        time_zone=\"America/Los_Angeles\",  # change this as necessary\n",
-        "        parameter_values={\"text\": \"Hello world!\"},\n",
-        "        # pipeline_root=PIPELINE_ROOT  # this argument is necessary if you did not specify PIPELINE_ROOT as part of the pipeline definition.\n",
-        "    )"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "J8AP1viy2Ou8"
-      },
-      "source": [
-        "Once the scheduled job is created, you can see it listed in the [Cloud Scheduler](https://console.cloud.google.com/cloudscheduler/) panel in the Console.\n",
-        "\n",
-        "<a href=\"https://storage.googleapis.com/amy-jo/images/kf-pls/pipelines_scheduler.png\" target=\"_blank\"><img src=\"https://storage.googleapis.com/amy-jo/images/kf-pls/pipelines_scheduler.png\" width=\"95%\"/></a>\n",
-        "\n",
-        "You can test the setup from the Cloud Scheduler panel by clicking 'RUN NOW'.\n",
-        "\n",
-        "> **Note**: The implementation is using a Cloud Functions function, which you can see listed in the [Cloud Functions](https://console.cloud.google.com/functions/list) panel in the console as `templated_http_request-v1`.\n",
-        "Don't delete this function, as it will prevent the Cloud Scheduler jobs from actually kicking off the pipeline run.  If you do delete it, create a new scheduled job in order to recreate the function.\n",
-        "\n",
-        "When you're done experimenting, you probably want to **PAUSE** your scheduled job from the Cloud Scheduler panel, so that the recurrent jobs do not keep running."
-      ]
-    },
    {
      "cell_type": "markdown",
      "metadata": {
@@ -1222,7 +1154,7 @@
      "outputs": [],
      "source": [
        "delete_pipeline = True\n",
-        "delete_bucket = True\n",
+        "delete_bucket = False\n",
        "\n",
        "try:\n",
        "    if delete_pipeline and \"DISPLAY_NAME\" in globals():\n",
Author	SHA1	Message	Date
Andrew Ferlitsch	986b76c6de	fix: detecting copyright cell	2022-09-13 18:08:32 +00:00
Andrew FerlitschandGitHub	3f06f48282	fix: filename rename (#943 ) * fix: filename rename * fix: lint issues	2022-09-13 10:01:22 -07:00
Soheila Zangeneh GitHub krishr2d2 sudarshan-SpringML Andrew Ferlitsch	33abd1e427	Move model evaluation notebooks from community to official (#940 ) * Add automl regression model eval first draft * Remove extra file * Pring evaluation results * adds the automl-tabular-classification notebook in model_evaluation folder * removes unnecessary imports * adjusts the imports inside the pipeline * adjusts the imports * elaborates imports inside pipeline * modified regression notebook * renamed pipeline displayname to resolve error * Add automl regression model eval first draft * Remove extra file * Pring evaluation results * modified some text * added suggested updates from review: remove dataflow params, add/change textual descriptions, add UUID * removes the output from the notebooks * removes the extra matplotlib import * ran linter test * addressed soheila's comments * ran linter * addresses the review comments * ran linter test * removes the artifacts comment * ran linter test * reviewed comments * ran linter * addresses review comments: textual updates, removes unnecessary parameters * ran linter test * addressed comments * ran linter * removed unwanted variables * ran linter * addresses the tech-writer's comments + updates the pipeline image with data-sampler task * ran linter test * Update text * Move model eval folder to official * Update CODEOWNERS * Run linter * Removed problem_type parameter * Run linter * addresses Andrew's review comments: textual updates and removes additional gcpc installation * ran linter test * comments addressed * ran linter * removed trailing comma on last parameter of trainingjob.run * ran linter Co-authored-by: krishr2d2 <krishna.movva@springml.com> Co-authored-by: sudarshan-SpringML <sudarshan.c@springml.com> Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>	2022-09-13 09:28:16 -07:00
Peter PingandGitHub	1d9bfe9934	Stream Update v2 (#946 ) * Stream Update v2 * correct codeowner name	2022-09-12 18:24:53 -04:00
Brad Miro GitHub Hyunuk Lim aman-ebay	fb9defa985	Merge to sparkml branch (#881 ) (#882 ) * Merge to sparkml branch (#881) * feat: initial commit * feat: WIP * feat: still WIP, need to work on EDA * fix: change name, still WIP * fix: initial draft * install geopandas in the notebook * fix: add codeowners * fix: install pyarrow * fix: add condition for testing env * fix: indentation * fix: add dependencies for gpd * fix: install seaborn * fix: isort and codeowner * fix: description * fix: add debriefing the result * fix: decrease sample size for testing * fix: code review wip * fix: code review * fix: change dataset to 2017 * fix: code review * fix: not using sql * fix: lint * fix: delete outputs * fix: code review * fix: typo * fix: make sample pandas df if not testing * Update spark_ml.ipynb (#884) (Tech writer edit) Editing for syntax and clarification. * small text updates * lint fixes * constraining plotting to non-test environments * lint fixes * put plotting back into tests * address review feedback * added comment to rerun cell if URLError thrown Co-authored-by: Hyunuk Lim <hyunuklim@google.com> Co-authored-by: aman-ebay <amancuso@google.com>	2022-09-12 17:28:09 -04:00
Soheila ZangenehandGitHub	824fb689e4	Bqml vertex model registry (#945 ) * Add bqml-vertexai-model-registry notebook	2022-09-12 16:44:46 -04:00
Andrew FerlitschandGitHub	c48dd8662b	Issue 235883443 (#944 ) * fix: remove obsoleted case * fix: remove obsoleted case	2022-09-12 15:49:52 -04:00
gericdongandGitHub	f251721d23	Enable Cloud Resource Manager API (#939 ) * Enable Cloud Resource Manager API * Reformatted file	2022-09-09 14:05:36 -07:00
Andrew FerlitschandGitHub	e949eb128f	fix: autoreview of mlops notebooks (#936 ) * fix: tune for autoreview * fix: tune for autoreview	2022-09-09 12:02:50 -04:00
Andrew Ferlitsch GitHub gericdong	df48e74f59	feat: Batch prediction for custom text model (#934 ) * feat: notebook for custom text model batch prediction * feat: notebook for custom text model batch prediction Co-authored-by: gericdong <itseric@google.com>	2022-09-09 09:26:13 -04:00
MarcandGitHub	ce9e6ecf62	add back pip install of xai sdk (#935 )	2022-09-09 01:02:56 +01:00
Soheila Zangeneh GitHub krishr2d2 sudarshan-SpringML Andrew Ferlitsch	9ab5f4274a	Add automl regression and classification with model evaluation (#911 ) * Add automl regression model eval first draft * Remove extra file * Pring evaluation results * adds the automl-tabular-classification notebook in model_evaluation folder * removes unnecessary imports * adjusts the imports inside the pipeline * adjusts the imports * elaborates imports inside pipeline * modified regression notebook * renamed pipeline displayname to resolve error * Add automl regression model eval first draft * Remove extra file * Pring evaluation results * modified some text * added suggested updates from review: remove dataflow params, add/change textual descriptions, add UUID * removes the output from the notebooks * removes the extra matplotlib import * ran linter test * addressed soheila's comments * ran linter * addresses the review comments * ran linter test * removes the artifacts comment * ran linter test * reviewed comments * ran linter * addresses review comments: textual updates, removes unnecessary parameters * ran linter test * addressed comments * ran linter * removed unwanted variables * ran linter * addresses the tech-writer's comments + updates the pipeline image with data-sampler task * ran linter test Co-authored-by: krishr2d2 <krishna.movva@springml.com> Co-authored-by: sudarshan-SpringML <sudarshan.c@springml.com> Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>	2022-09-08 14:42:04 -07:00
Andrew FerlitschandGitHub	29e584a422	feat: add batch automl video notebook (#933 ) * feat: batch for automl video * feat: batch for automl video * feat: batch for automl video	2022-09-08 11:34:29 -07:00
MarcandGitHub	beabb87cff	fix import problem described in b/245553683 (#932 ) * fix aiplatform import problem * lint fix * build fix, missing tensforflow * lint fixes	2022-09-08 16:47:20 +01:00
Polong Lin GitHub Ivan Cheung	e3f6717ff6	community -> official for bqml-online-prediction.ipynb (#788 ) * move bqml-vertex notebook from community to official * add to CODEOWNERS official * fix errors for execution-test * fix project_id line * fix linting * fixes re: comments from sarahcdugan * fix links at top of notebook from community/ to official/ * added UUID to model name * fix linting * fix error in TIMESTAMP --> UUID * fixing linting double space * fixes re: ivanmkc comments * fixed notebook after linting issues * linting via cloud shell * simplified run_bq_query function * linting Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com>	2022-09-07 18:20:02 -04:00
samthrasher GitHub Andrew Ferlitsch	fd30c4014a	Minor fixes for CPR Pytorch sample (#744 ) * Minor fixes for CPR Pytorch sample: Add missing test data, add auth info to readme, scrub private project and bucket names from config, tolerate missing config.json in unit tests. * Minor fixes for CPR Pytorch sample: Add missing test data, add auth info to readme, scrub private project and bucket names from config, tolerate missing config.json in unit tests. * Fix merge conflicts * fix typo * Point CPR links to main branch of SDK repo. Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>	2022-09-07 13:02:55 -07:00
Andrew FerlitschandGitHub	4be8b0a59a	fix: finetuning of batch notebooks (#930 ) * fix: fine-tuning * fix: fine-tuning	2022-09-07 10:44:44 -07:00
Andrew FerlitschandGitHub	aa09d46265	feat: batch prediction for AutoML text models (#929 ) * feat: Automl text model batch predict * feat: Automl text model batch predict * feat: Automl text model batch predict	2022-09-07 13:14:16 -04:00
Andrew FerlitschandGitHub	5667967131	feat: add BQ input example (#926 ) * feat: add notebook for custom tabular batch predict * feat: add notebook for custom tabular batch predict * feat: add example for BQ input * feat: add example for BQ input * feat: add example for BQ input * feat: add example for BQ input	2022-09-07 08:38:36 -07:00