fix: restore requirements.txt

migration: automl (#1455 )
migration: MM notebook (#1445 )
2026-09-28 16:11:56 +00:00 · 2023-01-13 17:41:45 +00:00 · 2023-01-12 09:29:25 -08:00 · 2023-01-12 09:28:43 -08:00 · 2023-01-11 22:38:26 -08:00 · 2023-01-11 18:16:34 -08:00
28 changed files with 19590 additions and 915 deletions
@@ -11,4 +11,3 @@ google-cloud-storage
 google-cloud-build
 ratemate
 GitPython
-google-api-core==2.10
@@ -1028,6 +1028,9 @@
        "deployment_resource_pool.dedicated_resources.min_replica_count = MIN_NODES\n",
        "deployment_resource_pool.dedicated_resources.max_replica_count = MAX_NODES\n",
        "deployment_resource_pool.dedicated_resources.machine_spec.machine_type = DEPLOY_COMPUTE\n",
+        "if DEPLOY_NGPU:\n",
+        "    deployment_resource_pool.dedicated_resources.machine_spec.accelerator_type = DEPLOY_GPU\n",
+        "    deployment_resource_pool.dedicated_resources.machine_spec.accelerator_count = DEPLOY_NGPU\n",
        "\n",
        "request = aip_beta.CreateDeploymentResourcePoolRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{REGION}\",\n",
@@ -112,7 +112,7 @@ def benchmark(

    results = []
    for qps in qps_list:
-        num_requests = max(qps * duration_sec, 10)
+        num_requests = int(max(qps * duration_sec, 10))
        requests_for_qps = list(
            itertools.islice(itertools.cycle(requests), num_requests)
        )
@@ -12,6 +12,10 @@
        --errors-codes: A list of error codes to report errors. Otherwise, all errors are reported.
        --errors-csv: Report errors in CSV format
        
+        # options for automatic fixing
+        --fix: Automatic fix
+        --fix-codes: A list of fix codes to fix. Otherwise, all fix codes are enabled.
+        
        # index generatation
        --repo: Generate index in markdown format
        --web: Generate index in HTML format
@@ -19,6 +23,7 @@
        --desc: Add description to index
        --steps: Add steps to index
        --uses: Add "resources" used to index
+        --linkback: Add linkback to index
        
    Format of CSV file for notebooks to review:
    
@@ -63,7 +63,7 @@
        "\n",
        "This notebook is aimed at data analysts and data scientists who have data in BigQuery, want to train a model using BigQuery ML, register the model to Vertex AI Model Registry, and deploy it to an endpoint for real-time prediction. \n",
        "\n",
-        "Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction)."
+        "Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -29,7 +29,7 @@
        "id": "title:generic,gcp"
      },
      "source": [
-        "# E2E ML on GCP: MLOps stage 1 : data management: get started with BigQuery datasets\n",
+        "# Get started with BigQuery datasets\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
@@ -61,7 +61,7 @@
        "## Overview\n",
        "\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with BigQuery datasets.\n",
+        "This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with BigQuery datasets.\n",
        "\n",
        "Learn more about [BigQuery Datasets](https://cloud.google.com/bigquery/docs/datasets-intro)."
      ]
@@ -29,7 +29,7 @@
        "id": "title:generic,gcp"
      },
      "source": [
-        "# E2E ML on GCP: MLOps stage 1 : formalization: get started with Vertex AI Data Labeling\n",
+        "# Get started with Vertex AI Data Labeling\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
@@ -62,7 +62,7 @@
        "## Overview\n",
        "\n",
        "\n",
-        "This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 1 : data management: get started with Vertex AI Data Labeling service.\n",
+        "This tutorial demonstrates how to use Vertex AI in production. This tutorial covers data management: get started with Vertex AI Data Labeling service.\n",
        "\n",
        "Learn more about [Vertex AI Data Labeling](https://cloud.google.com/vertex-ai/docs/datasets/data-labeling-job)."
      ]
@@ -33,19 +33,19 @@
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
-        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/structured_data/rapid_prototyping_bqml_automl.ipynb\">\n",
+        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/rapid_prototyping_bqml_automl.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>  \n",
        "\n",
        "  <td>\n",
-        "<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/structured_data/rapid_prototyping_bqml_automl.ipynb\" target='_blank'>\n",
+        "<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/rapid_prototyping_bqml_automl.ipynb\" target='_blank'>\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
-        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/structured_data/rapid_prototyping_bqml_automl.ipynb\" target='_blank'>\n",
+        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/pipelines/rapid_prototyping_bqml_automl.ipynb\" target='_blank'>\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
@@ -66,7 +66,7 @@
        "\n",
        "<img src=\"https://storage.googleapis.com/rafacarv-public-bucket-do-not-delete/abalone/automl_and_bqml.png\" />\n",
        "\n",
-        "Learn more about [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction)."
+        "Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component) and [BigQuery ML components](https://cloud.google.com/vertex-ai/docs/pipelines/bigqueryml-component)."
      ]
    },
    {
@@ -1,21 +0,0 @@
-
-[BQML and AutoML - Experimenting with Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/structured_data/rapid_prototyping_bqml_automl.ipynb)
-
-```
-Learn how to use `Vertex AI Predictions` for rapid prototyping a model.
-
-The steps performed include:
-
- Creating a BigQuery and Vertex AI training dataset.
- Training a BigQuery ML and AutoML model.
- Extracting evaluation metrics from the BigQueryML and AutoML models.
- Selecting the best trained model.
- Deploying the best trained model.
- Testing the deployed model infrastructure.
-
-```
-
-&nbsp;&nbsp;&nbsp;Learn more about [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users).
-
-&nbsp;&nbsp;&nbsp;Learn more about [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/introduction).
-
@@ -65,7 +65,7 @@
        "\n",
        "The goal of the tutorial is to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm.\n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -67,7 +67,7 @@
        "\n",
        "TabNet uses a machine learning technique called sequential attention to select which model features to reason from at each step in the model. This mechanism makes it possible to explain how the model arrives at its predictions and helps it learn more accurate models. Thanks to this design, TabNet not only outperforms other neural networks and decision trees but also provides interpretable feature attributions. Releasing TabNet as a First Party Trainer in Vertex AI means you'll be able to easily take advantage of TabNet's architecture and explainability and use it to train models on your own data. \n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook showcases how to run the TabNet algorithm using Vertex AI Tabular Workflows.\n",
        "\n",
-        "Learn more about [Vertex AI TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
+        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook showcases how to run the Wide & Deep algorithm using Vertex AI Tabular Workflows.\n",
        "\n",
-        "Learn more about [Vertex AI Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep) and [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction)."
+        "Learn more about [Tabular Workflow for Wide & Deep](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep)."
      ]
    },
    {
@@ -63,7 +63,7 @@
        "\n",
        "This notebook is written for data analysts and data scientists who have data in BigQuery and want to perform exploratory data analysis to gather insights from that data in an interactive environment.\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery](https://cloud.google.com/bigquery)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -92,7 +92,7 @@
        "\n",
        "This tutorial shows you how to train, evaluate a propensity model in BigQuery ML to predict user retention on a mobile game, based on app measurement data from Google Analytics 4.\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
@@ -87,7 +87,7 @@
        "\n",
        "*Note: This notebook file was developed to run in a [Vertex AI Workbench managed notebooks](https://console.cloud.google.com/vertex-ai/workbench/list/managed) instance using the Python (Local) kernel. Some components of this notebook may not work in other notebook environments.*\n",
        "\n",
-        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/managing-models-vertex)."
+        "Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction) and Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml)."
      ]
    },
    {
Author	SHA1	Message	Date
Andrew Ferlitsch	ddb061b9ac	fix: restore requirements.txt	2023-01-13 17:41:45 +00:00
Andrew FerlitschandGitHub	f42a184171	migration: automl (#1455 )	2023-01-12 09:29:25 -08:00
Andrew FerlitschandGitHub	03f0647b76	migration: MM notebook (#1445 ) * migration: MM notebook * migration: fix USER_EMAIL	2023-01-12 09:28:43 -08:00
Andrew FerlitschandGitHub	1f39732ae9	migration: distributed training (#1460 )	2023-01-11 22:38:26 -08:00
Andrew FerlitschandGitHub	7dd0b31b58	migration: experiments (#1461 )	2023-01-11 18:16:34 -08:00
Andrew FerlitschandGitHub	ff843173cf	Autoindex official 2 (#1459 ) * fix: update linkbacks to vertex pages * fix: update linkbacks to vertex pages * fix: update linkbacks to vertex pages	2023-01-11 16:53:28 -08:00
Andrew FerlitschandGitHub	da19b116e9	fix: update linkbacks to vertex pages (#1458 )	2023-01-11 16:30:10 -08:00
Andrew FerlitschandGitHub	d8b365dfd4	fix: update the linkback (#1457 )	2023-01-11 15:02:14 -08:00
Andrew FerlitschandGitHub	1247c80fed	migration: bqml (#1456 )	2023-01-11 14:53:13 -08:00
Aleksey Vlasenko GitHub Aleksey Vlasenko	2cf2bf1080	Adding sample T5x sample for optimized TensorFlow runtime (#1453 ) * adding T5x sample * update for benchmark params * update for benchmark params * updated model GCS buckets for optimized TF runtime T5x sample * added GPU accelerators for deployment pool in Vertex shared VM sample * final updates for T5x sample * addressed PR feedback Co-authored-by: Aleksey Vlasenko <alekseyv@google.com>	2023-01-11 13:32:53 -08:00
Andrew FerlitschandGitHub	5e9e8139c1	Update get_started_with_model_monitoring_xgboost.ipynb	2023-01-11 12:10:46 -08:00
Andrew FerlitschandGitHub	0728a0036f	Update get_started_with_model_monitoring_setup.ipynb	2023-01-11 12:10:11 -08:00
Andrew FerlitschandGitHub	aa52d21643	Update get_started_with_model_monitoring_custom_tf_serving.ipynb	2023-01-11 12:09:18 -08:00
Andrew FerlitschandGitHub	103888d75e	Update get_started_with_model_monitoring_custom.ipynb	2023-01-11 12:08:33 -08:00
Andrew FerlitschandGitHub	13d0d5d9b0	Update get_started_bq_datasets.ipynb	2023-01-11 12:06:47 -08:00
Andrew FerlitschandGitHub	79c6669686	Update get_started_with_data_labeling.ipynb	2023-01-11 12:06:20 -08:00
Andrew FerlitschandGitHub	dceb0c4c1c	Update get_started_bq_datasets.ipynb	2023-01-11 12:04:31 -08:00
Andrew FerlitschandGitHub	0c9cdca713	migration: MM notebook (#1449 ) * migration: MM notebook * migration: MM notebook	2023-01-10 20:58:19 -08:00
Andrew FerlitschandGitHub	6124092681	migration: MM notebook (#1447 ) * migration: MM notebook * migration: MM notebook	2023-01-10 18:39:45 -08:00
Andrew FerlitschandGitHub	629e739327	migration: MM notebook (#1446 ) * migration: MM notebook * migration: MM notebook	2023-01-10 17:51:00 -08:00
Andrew FerlitschandGitHub	b68cbd8255	migration: move to pipelines folder (#1452 )	2023-01-10 16:44:17 -08:00
Andrew FerlitschandGitHub	7fec30c12f	migration: MM notebook (#1448 )	2023-01-10 16:31:14 -08:00
Andrew FerlitschandGitHub	a754843c39	migration: MM notebook (#1444 ) * migration: MM notebook * migration: MM notebook * migration: MM notebook	2023-01-10 15:33:26 -08:00
reznitskii GitHub Max Reznitskii	436db4a35b	Fixed documentation links (#1450 ) Co-authored-by: Max Reznitskii <reznitskii@google.com>	2023-01-10 15:22:58 -08:00