From 7eb4eea074e49c229e5e2e8640ee2097fc026b86 Mon Sep 17 00:00:00 2001 From: Andrew Ferlitsch Date: Tue, 7 Mar 2023 20:06:00 +0000 Subject: [PATCH] update: march update of index --- notebooks/official/README.md | 438 ------------------ notebooks/official/automl/README.md | 6 +- notebooks/official/custom/README.md | 8 +- notebooks/official/experiments/README.md | 2 +- notebooks/official/explainable_ai/README.md | 24 +- notebooks/official/matching_engine/README.md | 52 +-- notebooks/official/migration/README.md | 36 +- notebooks/official/ml_metadata/README.md | 2 +- notebooks/official/model_evaluation/README.md | 32 +- notebooks/official/model_monitoring/README.md | 20 +- notebooks/official/model_registry/README.md | 18 + notebooks/official/pipelines/README.md | 57 +-- notebooks/official/sdk/README.md | 4 +- notebooks/official/tabnet/README.md | 4 +- .../official/tabular_workflows/README.md | 16 + notebooks/official/tensorboard/README.md | 36 +- notebooks/official/training/README.md | 26 +- notebooks/official/workbench/README.md | 4 +- 18 files changed, 244 insertions(+), 541 deletions(-) diff --git a/notebooks/official/README.md b/notebooks/official/README.md index 1572043fc..6b260b6ab 100644 --- a/notebooks/official/README.md +++ b/notebooks/official/README.md @@ -4,441 +4,3 @@ The official notebooks are a collection of curated and non-curated notebooks aut The official notebooks are organized by Google Cloud Vertex AI services. -## Manifest of Curated Notebooks - -### AutoML Text data - - -[Create, train, and deploy an AutoML text classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl-text-classification.ipynb) - -Learn how to use `AutoML` to train a text classification model. - -The steps performed include: - -* Create a `Vertex AI Dataset`. -* Train an `AutoML` text classification `Model` resource. -* Obtain the evaluation metrics for the `Model` resource. -* Create an `Endpoint` resource. -* Deploy the `Model` resource to the `Endpoint` resource. -* Make an online prediction -* Make a batch prediction - -### AutoML Tabular data - - -[AutoML tabular forecasting model for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_tabular_forecasting_batch.ipynb) - -Learn how to create an `AutoML` tabular forecasting model from a Python script, and then do a batch prediction using the Vertex AI SDK. - -The steps performed include: - -- Create a `Vertex AI Dataset` resource. -- Train an `AutoML` tabular forecasting `Model` resource. -- Obtain the evaluation metrics for the `Model` resource. -- Make a batch prediction. - -### BigQuery ML Vertex AI Model Registry Batch prediction - - -[Deploy BiqQuery ML Model on Vertex AI Model Registry and make predictions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model-registry/bqml-vertexai-model-registry.ipynb) - -Learn how to use `Vertex AI Model Registry` with `BigQuery ML` and make batch predictions: - -The steps performed include: - -- Train a model with `BigQuery ML` -- Upload the model to `Vertex AI Model Registry` -- Create a `Vertex AI Endpoint` resource -- Deploy the `Model` resource to the `Endpoint` resource -- Make `prediction` requests to the model endpoint -- Run `batch prediction` job on the `Model` resource - - -### BigQuery ML Vertex AI Model Registry Online prediction - - -[Online prediction with BigQuery ML](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery_ml/bqml-online-prediction.ipynb) - -Learn how to train and deploy a churn prediction model for real-time inference, with the data in BigQuery and model trained using BigQuery ML, registered to Vertex AI Model Registry, and deployed to an endpoint on Vertex AI for online predictions. - -The steps performed include: - -- Using Python & SQL to query the public data in BigQuery -- Preparing the data for modeling -- Training a classification model using BigQuery ML and registering it to Vertex AI Model Registry -- Inspecting the model on Vertex AI Model Registry -- Deploying the model to an endpoint on Vertex AI -- Making sample online predictions to the model endpoint - - -### Custom Training - - -[Custom training and batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/sdk-custom-image-classification-batch.ipynb) - -Learn to use `Vertex AI Training` to create a custom trained model and use `Vertex AI Batch Prediction` to do a batch prediction on the trained model. - -The steps performed include: - -- Create a `Vertex AI` custom job for training a TensorFlow model. -- Upload the trained model artifacts as a `Model` resource. -- Make a batch prediction. - -[Custom training and online prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/sdk-custom-image-classification-online.ipynb) - -Learn to use `Vertex AI Training` to create a custom-trained model from a Python script in a Docker container, and learn to use `Vertex AI Prediction` to do a prediction on the deployed model by sending data. - -The steps performed include: - -- Create a `Vertex AI` custom job for training a TensorFlow model. -- Upload the trained model artifacts to a `Model` resource. -- Create a serving `Endpoint` resource. -- Deploy the `Model` resource to a serving `Endpoint` resource. -- Make a prediction. -- Undeploy the `Model` resource. - -### Tabular Data - - -[Compare Vertex AI Forecasting and BigQuery ML ARIMA_PLUS](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb) - -Learn how to create an BQML ARIMA_PLUS model using a training [Vertex AI Pipeline](https://cloud. - -The steps performed are: - -- Train the BQML ARIMA_PLUS model. -- View BQML model evaluation. -- Make a batch prediction with the BQML model. -- Create a Vertex AI `Dataset` resource. -- Train the Vertex AI Forecasting model. -- View the Model evaluation. -- Make a batch prediction with the Model. - - -### AutoML Tabular Data - - -[AutoML Tabular Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_tabular_on_vertex_pipelines.ipynb) - -Learn how to create two regression models using [Vertex Pipelines](https://cloud. - -The steps performed are: - -- Create a training pipeline that reduces the search space from the default to save time. -- Create a training pipeline that reuses the architecture search results from the previous pipeline to save time. - -### Vertex AI Experiments - - -[Compare pipeline runs with Vertex AI Experiments](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/comparing_pipeline_runs.ipynb) - -Learn how to use `Vertex AI Experiments` to log a pipeline job and compare different pipeline jobs. - - - -[Build Vertex AI Experiment lineage for custom training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/build_model_experimentation_lineage_with_prebuild_code.ipynb) - -Learn how to integrate preprocessing code in a Vertex AI experiments. - - - -[Track parameters and metrics for locally trained models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/comparing_local_trained_models.ipynb) - -Learn how to use Vertex AI Experiments to compare and evaluate model experiments. - -The steps performed include: - -- log the model parameters -- log the loss and metrics on every epoch to TensorBoard -- log the evaluation metrics - - -### Vertex AI Feature Store - - -[Online and Batch predictions using Vertex AI Feature Store](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/sdk-feature-store.ipynb) - -Learn how to use `Vertex AI Feature Store` to import feature data, and to access the feature data for both online serving and offline tasks, such as training. - -The steps performed include: - -- Create featurestore, entity type, and feature resources. -- Import feature data into `Vertex AI Feature Store` resource. -- Serve online prediction requests using the imported features. -- Access imported features in offline jobs, such as training jobs. - -### Matching Engine - - -[Create Vertex AI Matching Engine index](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_for_indexing.ipynb) - -Learn how to create Approximate Nearest Neighbor (ANN) Index, query against indexes, and validate the performance of the index. - -The steps performed include: - -* Create ANN Index and Brute Force Index -* Create an IndexEndpoint with VPC Network -* Deploy ANN Index and Brute Force Index -* Perform online query -* Compute recall - - -### Model Monitoring - - -[Vertex AI Model Monitoring with Explainable AI Feature Attributions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/model_monitoring.ipynb) - -Learn to use the `Vertex AI Model Monitoring` service to detect drift and anomalies in prediction requests from a deployed `Vertex AI Model` resource. - -The steps performed include: - -- Upload a pre-trained model as a `Vertex AI Model` resource. -- Create an `Vertex AI Endpoint` resource. -- Deploy the `Model` resource to the `Endpoint` resource. -- Configure the `Endpoint` resource for model monitoring. -- Initialize the baseline distribution for model monitoring. -- Generate synthetic prediction requests. -- Understand how to interpret the statistics, visualizations, other data reported by the model monitoring feature. - -### Vertex AI Pipelines - - -[Lightweight Python function-based components, and component I/O](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/lightweight_functions_component_io_kfp.ipynb) - -Learn to use the KFP SDK to build lightweight Python function-based components, and then you learn to use `Vertex AI Pipelines` to execute the pipeline. - -The steps performed include: - -- Build Python function-based KFP components. -- Construct a KFP pipeline. -- Pass *Artifacts* and *parameters* between components, both by path reference and by value. -- Use the `kfp.dsl.importer` method. -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - -### Vertex AI Pipelines Image data - - -[AutoML image classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_automl_images.ipynb) - -Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` image classification model. - -The steps performed include: - -- Create a KFP pipeline: - - Create a `Dataset` resource. - - Train an AutoML image classification `Model` resource. - - Create an `Endpoint` resource. - - Deploys the `Model` resource to the `Endpoint` resource. -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - - - -### Vertex AI Pipelines Tabular data - - -[AutoML Tabular pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/automl_tabular_classification_beans.ipynb) - -Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` tabular classification model. - -The steps performed include: - -- Create a KFP pipeline: - - Create a `Dataset` resource. - - Train an AutoML tabular classification `Model` resource. - - Create an `Endpoint` resource. - - Deploys the `Model` resource to the `Endpoint` resource. -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - - - -[AutoML tabular regression pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_automl_tabular.ipynb) - -Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` tabular regression model. - -The steps performed include: - -- Create a KFP pipeline: - - Create a `Dataset` resource. - - Train an AutoML tabular regression `Model` resource. - - Create an `Endpoint` resource. - - Deploys the `Model` resource to the `Endpoint` resource. -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - - - -### Vertex AI Pipelines Text data - - -[AutoML text classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_automl_text.ipynb) - -Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` text classification model. - -The steps performed include: - -- Create a KFP pipeline: - - Create a `Dataset` resource. - - Train an AutoML text classification `Model` resource. - - Create an `Endpoint` resource. - - Deploys the `Model` resource to the `Endpoint` resource. -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - - - -### Vertex AI Pipelines - - -[Custom training with pre-built Google Cloud Pipeline Components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/custom_model_training_and_batch_prediction.ipynb) - -Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build a custom model. - -The steps performed include: - -- Create a KFP pipeline: - - Train a custom model. - - Upload the trained model as a `Model` resource. - - Create an `Endpoint` resource. - - Deploy the `Model` resource to the `Endpoint` resource. - - Make a batch prediction request. - - - -[Pipeline control structures using the KFP SDK](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/control_flow_kfp.ipynb) - -Learn how to use the KFP SDK to build pipelines that use loops and conditionals, including nested examples. - -The steps performed include: - -- Create a KFP pipeline: - - Use control flow components -- Compile the KFP pipeline. -- Execute the KFP pipeline using `Vertex AI Pipelines` - -[Metrics visualization and run comparison using the KFP SDK](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/metrics_viz_run_compare_kfp.ipynb) - -Learn how to use the KFP SDK to build pipelines that generate evaluation metrics. - -The steps performed include: - -- Create KFP components: - - Generate ROC curve and confusion matrix visualizations for classification results - - Write metrics -- Create KFP pipelines. -- Execute KFP pipelines -- Compare metrics across pipeline runs - -[Pipelines introduction for KFP](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/pipelines_intro_kfp.ipynb) - -Learn how to use the KFP SDK to build pipelines that generate evaluation metrics. - -The steps performed include: - -- Define and compile a `Vertex AI` pipeline. -- Specify which service account to use for a pipeline run. - -### Vertex AI Vizier - - -[Optimizing multiple objectives with Vertex AI Vizier](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/vizier/gapic-vizier-multi-objective-optimization.ipynb) - -Learn how to use `Vertex AI Vizier` to optimize a multi-objective study. - - - -### Vertex Explainable AI Tabular data - - -[AutoML training tabular binary classification model for batch explanation](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_automl_tabular_binary_classification_batch_explain.ipynb) - -Learn to use `AutoML` to create a tabular binary classification model from a Python script, and then learn to use `Vertex AI Batch Prediction` to make predictions with explanations. - -The steps performed include: - -- Create a `Vertex Dataset` resource. -- Train an `AutoML` tabular binary classification model. -- View the model evaluation metrics for the trained model. -- Make a batch prediction request with explainability. - - -* Prediction Service: Does an on-demand prediction for the entire set of instances (i.e., one or more data items) and returns the results in real-time. - -* Batch Prediction Service: Does a queued (batch) prediction for the entire set of instances in the background and stores the results in a Cloud Storage bucket when ready. - -[AutoML training tabular classification model for online explanation](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_automl_tabular_classification_online_explain.ipynb) - -Learn how to use `AutoML` to create a tabular binary classification model from a Python script, and then learn to use `Vertex AI Online Prediction` to make online predictions with explanations. - -The steps performed include: - -- Create a `Vertex Dataset` resource. -- Train an `AutoML` tabular binary classification model. -- View the model evaluation metrics for the trained model. -- Create a serving `Endpoint` resource. -- Deploy the `Model` resource to a serving `Endpoint` resource. -- Make an online prediction request with explainability. -- Undeploy the `Model` resource. - -### Vertex Explainable AI Image data - - -[Custom training image classification model for batch prediction with explainabilty](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_image_classification_batch_explain.ipynb) - -Learn to use `Vertex AI Training and Explainable AI` to create a custom image classification model with explanations, and then you learn to use `Vertex AI Batch Prediction` to make a batch prediction request with explanations. - -The steps performed include: - -- Create a `Vertex AI` custom job for training a TensorFlow model. -- View the model evaluation for the trained model. -- Set explanation parameters for when the model is deployed. -- Upload the trained model artifacts and explanation parameters as a `Model` resource. -- Make a batch prediction with explanations. - -[Custom training image classification model for online prediction with explainabilty](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_image_classification_online_explain.ipynb) - -Learn how to use `Vertex AI Training and Explainable AI` to create a custom image classification model with explanations, and then you learn to use `Vertex AI Prediction` to make an online prediction request with explanations. - -The steps performed include: - -- Create a `Vertex AI` custom job for training a TensorFlow model. -- View the model evaluation for the trained model. -- Set explanation parameters for when the model is deployed. -- Upload the trained model artifacts and explanations as a `Model` resource. -- Create a serving `Endpoint` resource. -- Deploy the `Model` resource to a serving `Endpoint` resource. -- Make a prediction with explanation. -- Undeploy the `Model` resource. - -### Vertex Explainable AI Tabular data - - -[Custom training tabular regression model for batch prediction with explainabilty](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_tabular_regression_batch_explain.ipynb) - -Learn how to use `Vertex AI Training and Explainable AI` to create a custom image classification model with explanations, and then you learn to use `Vertex AI Batch Prediction` to make a batch prediction request with explanations. - -The steps performed include: - -- Create a `Vertex AI` custom job for training a TensorFlow model. -- View the model evaluation for the trained model. -- Set explanation parameters for when the model is deployed. -- Upload the trained model artifacts and explanations as a `Model` resource. -- Make a batch prediction with explanations. - -### Vertex ML Metadata - - -[Track parameters and metrics for custom training jobs](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/ml_metadata/sdk-metric-parameter-tracking-for-custom-jobs.ipynb) - -Learn how to use Vertex AI SDK for Python to: - -The steps performed include: -- Track training parameters and prediction metrics for a custom training job. -- Extract and perform analysis for all parameters and metrics within an Experiment. - - - - - diff --git a/notebooks/official/automl/README.md b/notebooks/official/automl/README.md index 547a2db34..21c7da5a9 100644 --- a/notebooks/official/automl/README.md +++ b/notebooks/official/automl/README.md @@ -40,7 +40,7 @@ The steps performed include: [Compare Vertex AI Forecasting and BigQuery ML ARIMA_PLUS](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_forecasting_bqml_arima_plus_comparison.ipynb) ``` -Learn how to create an BigQuery ML ARIMA_PLUS model using a training [Vertex AI Pipeline](https://cloud. +Learn how to create an BigQuery ML ARIMA_PLUS model using a training Vertex AI Pipeline from Google Cloud Pipeline Components , and then do a batch prediction using the corresponding prediction pipeline. The steps performed are: @@ -60,7 +60,7 @@ The steps performed are: [AutoML Tabular Workflow pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_tabular_on_vertex_pipelines.ipynb) ``` -Learn how to create two regression models using [Vertex AI Pipelines](https://cloud. +Learn how to create two regression models using Vertex AI Pipelines downloaded from Google Cloud Pipeline Components . The steps performed are: @@ -88,7 +88,7 @@ The steps performed include: ``` -   Learn more about [Vertex AI for AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users). +   Learn more about [AutoML training](https://cloud.google.com/vertex-ai/docs/training-overview). [AutoML training hierarchical forecasting for batch prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_forecasting_hierarchical_batch.ipynb) diff --git a/notebooks/official/custom/README.md b/notebooks/official/custom/README.md index 04bbb8013..9d94a4dd1 100644 --- a/notebooks/official/custom/README.md +++ b/notebooks/official/custom/README.md @@ -14,7 +14,7 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).    Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). @@ -37,7 +37,7 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).    Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). @@ -90,7 +90,7 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).    Learn more about [Vertex AI Batch Prediction](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions). @@ -111,7 +111,7 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).    Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). diff --git a/notebooks/official/experiments/README.md b/notebooks/official/experiments/README.md index 4bc3ee21a..afed347f8 100644 --- a/notebooks/official/experiments/README.md +++ b/notebooks/official/experiments/README.md @@ -86,5 +86,5 @@ The steps performed include:    Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). diff --git a/notebooks/official/explainable_ai/README.md b/notebooks/official/explainable_ai/README.md index 89503c633..b542b19c5 100644 --- a/notebooks/official/explainable_ai/README.md +++ b/notebooks/official/explainable_ai/README.md @@ -13,7 +13,7 @@ The steps performed include: ``` -   Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/tabular-data/overview). +   Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).    Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview). @@ -35,7 +35,7 @@ The steps performed include: ``` -   Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/tabular-data/overview). +   Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview).    Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview). @@ -149,3 +149,23 @@ The steps performed include:    Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). + +[Explaining image classification with Vertex Explainable AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/xai_image_classification_feature_attributions.ipynb) + +``` +Learn how to configure feature-based explanations on a pre-trained image classification model and make online and batch predictions with explanations. + +The steps performed include: + +- Download pretrained model from TensorFlow Hub +- Upload model for deployment +- Deploy model for online prediction +- Make online prediction with explanations +- Make batch predictions with explanations + +``` + +   Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview). + +   Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions). + diff --git a/notebooks/official/matching_engine/README.md b/notebooks/official/matching_engine/README.md index 493b05d44..289d0629d 100644 --- a/notebooks/official/matching_engine/README.md +++ b/notebooks/official/matching_engine/README.md @@ -1,17 +1,32 @@ -[Introduction to builtin Swivel embedding algorithm](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/intro-swivel.ipynb) +[Using Vertex AI Matching Engine for StackOverflow Questions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_stack_overflow_embeddings.ipynb) ``` -Learn how to train custom embeddings using Vertex AI Pipelines and deploy the model for serving. +Learn how to encode custom text embeddings, create an Approximate Nearest Neighbor index, and query against indexes. The steps performed include: -1. **Setup**: Importing the required libraries and setting your global variables. -2. **Configure parameters**: Setting the appropriate parameter values for the pipeline job. -3. **Train on Vertex AI Pipelines**: Create a Swivel job to Vertex Pipelines using pipeline template. -4. **Deploy on Vertex AI Prediction**: Importing and deploying the trained model to a callable endpoint. -5. **Predict**: Calling the deployed endpoint using online prediction. -6. **Cleaning up**: Deleting resources created by this tutorial. +* Create ANN index +* Create an index endpoint with VPC Network +* Deploy ANN index +* Perform online query + +``` + +   Learn more about [Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview). + + +[Using Vertex AI Matching Engine for Text-to-Image Embeddings](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_text_to_image_embeddings.ipynb) + +``` +Learn how to encode custom text embeddings, create an Approximate Nearest Neighbor index, and query against indexes. + +The steps performed include: + +* Create ANN index +* Create an index endpoint with VPC Network +* Deploy ANN index +* Perform online query ``` @@ -21,7 +36,7 @@ The steps performed include: [Create Vertex AI Matching Engine index](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_for_indexing.ipynb) ``` -Learn how to create Approximate Nearest Neighbor (ANN) Index, query against indexes, and validate the performance of the index. +Learn how to create Approximate Nearest Neighbor Index, query against indexes, and validate the performance of the index. The steps performed include: @@ -35,22 +50,3 @@ The steps performed include:    Learn more about [Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview). - -[Introduction to builtin Two-Towers embedding algorithm](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/two-tower-model-introduction.ipynb) - -``` -Learn how to run the Two-Tower model. - -The steps performed include: -1. **Setup**: Importing the required libraries and setting your global variables. -2. **Configure parameters**: Setting the appropriate parameter values for the training job. -3. **Train on Vertex AI Training**: Submitting a training job. -4. **Deploy on Vertex AI Prediction**: Importing and deploying the trained model to a callable endpoint. -5. **Predict**: Calling the deployed endpoint using online or batch prediction. -6. **Hyperparameter tuning**: Running a hyperparameter tuning job. -7. **Cleaning up**: Deleting resources created by this tutorial. - -``` - -   Learn more about [Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview). - diff --git a/notebooks/official/migration/README.md b/notebooks/official/migration/README.md index 49f2c6cbc..a79d0b45d 100644 --- a/notebooks/official/migration/README.md +++ b/notebooks/official/migration/README.md @@ -15,7 +15,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Image](https://cloud.google.com/vertex-ai/docs/tutorials/image-recognition-automl/training). +   Learn more about [Classification for image data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_images). [Custom Scikit-Learn model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb) @@ -35,7 +35,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Hyperparameter Tuning](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ11 Vertex SDK Hyperparameter Tuning.ipynb) @@ -51,10 +51,10 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). -[AutoML Video Classificaton](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ14 Vertex SDK AutoML Video Classification.ipynb) +[AutoML Video Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ14 Vertex SDK AutoML Video Classification.ipynb) ``` Learn to use `AutoML` to train a video model and use `Vertex AI Batch Prediction` to do batch predictions. @@ -68,7 +68,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/tutorials/video-classification-automl/training). +   Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos). [AutoML Video Object Tracking](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb) @@ -85,7 +85,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Video](https://cloud.google.com/video-intelligence/automl/object-tracking/docs/index-object-tracking). +   Learn more about [Object tracking for video data](https://cloud.google.com/vertex-ai/docs/training-overview#object_tracking_for_videos). [Custom Image Classification w/pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb) @@ -109,7 +109,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Custom Image Classification w/custom training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb) @@ -133,7 +133,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [AutoML Tabular Binary Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb) @@ -154,7 +154,7 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables). +   Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). [AutoML Image Object Detection](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ5 Vertex SDK AutoML Image Object Detection.ipynb) @@ -173,35 +173,35 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Image](https://cloud.google.com/vertex-ai/docs/tutorials/image-recognition-automl/training). +   Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images). [AutoML Text Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ6 Vertex SDK AutoML Text Classification.ipynb) ``` -The objective of this notebook is to build a AutoML Video Classification Model. +The objective of this notebook is to build a AutoML Text Classification Model. The steps performed include the following: * Set your task name, and GCS prefix -* Copy AutoML video demo train data for creating managed dataset +* Copy AutoML text demo train data for creating managed dataset * Create a dataset on Vertex AI. * Configure a training job * Launch a training job and create a model on Vertex AI -* Copy AutoML Video Demo Prediction Data for creating batch prediction job +* Copy AutoML Text Demo Prediction Data for creating batch prediction job * Perform batch prediction job on the model ```    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/classification/prepare-data). +   Learn more about [Classification for text data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_text). [AutoML Text Entity Extraction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb) ``` -The objective of this notebook is to build a AutoML Text Entity Extraction Model. +The objective of this notebook is to build a AutoML Text Entity Extraction model. The steps performed include the following: @@ -217,7 +217,7 @@ The steps performed include the following:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/entity-extraction/prepare-data). +   Learn more about [Entity extraction for text data](https://cloud.google.com/vertex-ai/docs/training-overview#entity_extraction_for_text). [AutoML Text Sentiment Analysis](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb) @@ -238,7 +238,7 @@ The steps performed include the following:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/sentiment-analysis/prepare-data). +   Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text). [Custom XGBoost model with pre-built training container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb) @@ -258,5 +258,5 @@ The steps performed include:    Learn more about [Migrate to Vertex AI](https://cloud.google.com/vertex-ai/docs/start/migrating-to-vertex-ai). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). diff --git a/notebooks/official/ml_metadata/README.md b/notebooks/official/ml_metadata/README.md index 868f10e29..22bd4c21a 100644 --- a/notebooks/official/ml_metadata/README.md +++ b/notebooks/official/ml_metadata/README.md @@ -12,7 +12,7 @@ The steps performed include:    Learn more about [Vertex ML Metadata](https://cloud.google.com/vertex-ai/docs/ml-metadata). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Track parameters and metrics for locally trained models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/ml_metadata/sdk-metric-parameter-tracking-for-locally-trained-models.ipynb) diff --git a/notebooks/official/model_evaluation/README.md b/notebooks/official/model_evaluation/README.md index e2e274216..55a53d84b 100644 --- a/notebooks/official/model_evaluation/README.md +++ b/notebooks/official/model_evaluation/README.md @@ -17,7 +17,7 @@ The steps performed include:    Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). -   Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables). +   Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). [Evaluating batch prediction results from AutoML Tabular regression model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_tabular_regression_model_evaluation.ipynb) @@ -32,14 +32,14 @@ The steps performed include: - Run the `AutoMLTabularTrainingJob` which returns a model - Import a pre-trained `AutoML model resource` into the pipeline - Run a `batch prediction` job in the pipeline -- Evaulate the AutoML model using the `regression evaluation component` +- Evaluate the AutoML model using the `regression evaluation component` - Import the Regression Metrics to the AutoML model resource ```    Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). -   Learn more about [AutoML Tabular](https://cloud.google.com/vertex-ai/docs/start/automl-users#tables). +   Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). [AutoML text classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_text_classification_model_evaluation.ipynb) @@ -74,14 +74,14 @@ The steps performed include: - Train a Automl Video Classification model on the `Vertex AI Dataset` resource. - Import the trained `AutoML Vertex AI Model resource` into the pipeline. - Run a batch prediction job inside the pipeline. -- Evaulate the AutoML model using the classification evaluation component. +- Evaluate the AutoML model using the classification evaluation component. - Import the classification metrics to the AutoML Vertex AI Model resource. ```    Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). -   Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/video-data/classification/prepare-data). +   Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos). [Evaluating BatchPrediction results from a Custom Tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/custom_tabular_classification_model_evaluation.ipynb) @@ -125,12 +125,30 @@ The steps performed include: - Upload the model as a Vertex AI Model resource. - Import a pre-trained `Vertex AI model resource` into the pipeline. - Run a `batch prediction` job in the pipeline. -- Evaulate the model using the `regression evaluation component`. +- Evaluate the model using the `regression evaluation component`. - Import the Regression Metrics to the Vertex AI model resource. ```    Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). + + +[Get started with importing a custom model evaluation to the Vertex AI Model Registry](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/get_started_with_custom_model_evaluation_import.ipynb) + +``` +Learn how to construct and upload a custom model evaluation, and upload the custom model evaluation to a Model resource entry in Vertex AI Model Registry. + +The steps performed include: + +- Import a pretrained (blessed) model to the Vertex AI Model Registry. +- Construct a custom model evaluation. +- Import the model evaluation metrics to the corresponding model in the Vertex AI Model Registry. +- List the model evaluation for the corresponding model in the Vertex AI Model Registry. +- Construct a second custom model evaluation. +- Import the second model evaluation metrics to the corresponding model in the Vertex AI Model Registry. +- List the second model evaluation for the corresponding model in the Vertex AI Model Registry. + +``` diff --git a/notebooks/official/model_monitoring/README.md b/notebooks/official/model_monitoring/README.md index 4731b91a3..defda9307 100644 --- a/notebooks/official/model_monitoring/README.md +++ b/notebooks/official/model_monitoring/README.md @@ -26,7 +26,6 @@ The steps performed include: - Deploy the `Model` resource to the `Endpoint` resource. - Configure the `Endpoint` resource for model monitoring. - Generate synthetic prediction requests for skew. -- Wait for email alert notification. - Generate synthetic prediction requests for drift. - Wait for email alert notification. @@ -52,6 +51,25 @@ The steps performed include:    Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring). +[Vertex AI Model Monitoring for online prediction in AutoML image models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_automl_image_online.ipynb) + +``` +Learn how to use `Vertex AI Model Monitoring` with `Vertex AI Online Prediction` with an AutoML image classification model to detect an out of distribution image. + +The steps performed include: + +1. Train an AutoML image classification model. +2. Create an endpoint. +3. Deploy the model to the endpoint, and configure for model monitoring. +4. Submit a online prediction containing both in and out of distribution images. +5. Use Model Monitoring to calculate anomaly score on each image. +6. Identify the images in the online prediction request that are out of distribution. + +``` + +   Learn more about [Vertex AI Model Monitoring](https://cloud.google.com/vertex-ai/docs/model-monitoring). + + [Vertex AI Model Monitoring for custom tabular models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_monitoring/get_started_with_model_monitoring_custom.ipynb) ``` diff --git a/notebooks/official/model_registry/README.md b/notebooks/official/model_registry/README.md index 256943dd2..0f7d58637 100644 --- a/notebooks/official/model_registry/README.md +++ b/notebooks/official/model_registry/README.md @@ -19,3 +19,21 @@ The steps performed include:    Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml). + +[Get started with Vertex AI Model Registry](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_registry/get_started_with_model_registry.ipynb) + +``` +Learn how to use `Vertex AI Model Registry` to create and register multiple versions of a model. + +The steps performed include: + +- Create and register a first version of a model to `Vertex AI Model Registry`. +- Create and register a second version of a model to `Vertex AI Model Registry`. +- Updating the model version which is the default (blessed). +- Deleting a model version. +- Retraining the next model version. + +``` + +   Learn more about [Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction). + diff --git a/notebooks/official/pipelines/README.md b/notebooks/official/pipelines/README.md index 4ccace970..8e8519fa5 100644 --- a/notebooks/official/pipelines/README.md +++ b/notebooks/official/pipelines/README.md @@ -20,6 +20,31 @@ The steps performed include:    Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component). +   Learn more about [Classification for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). + + +[Challenger vs Blessed methodology for model deployment into production](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/pipelines/challenger_vs_blessed_deployment_method.ipynb) + +``` +Learn how to construct a Vertex AI pipeline, which trains a new challenger version of a model, evaluates the model and compares the evaluation to the existing blessed model in production, to determine whether the challenger model becomes the blessed model for replacement in production. + +The steps performed include: + +- Import a pretrained (blessed) model to the `Vertex AI Model Registry`. +- Import synthetic model evaluation metrics to the corresponding (blessed) model. +- Create a `Vertex AI Endpoint` resource +- Deploy the blessed model to the `Endpoint` resource. +- Create a Vertex AI Pipeline + - Get the blessed model. + - Import another instance (challenger) of the pretrained model. + - Register the pretrained (challenger) model as a new version of the existing blessed model. + - Create a synthetic model evaluation. + - Import the synthetic model evaluation metrics to the corresponding challenger model. + - Compare the evaluations and set the blessed or challenger as the default. + - Deploy the new blessed model. + +``` + [Pipeline control structures using the KFP SDK](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/control_flow_kfp.ipynb) @@ -56,7 +81,7 @@ The steps performed include:    Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction). -   Learn more about [Vertex AI Training components](https://cloud.google.com/vertex-ai/docs/training/create-training-pipeline). +   Learn more about [Custom training components](https://cloud.google.com/vertex-ai/docs/training/create-training-pipeline). [Training and batch prediction with BigQuery source and destinantion for a custom tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/custom_tabular_train_batch_pred_bq_pipeline.ipynb) @@ -128,6 +153,8 @@ The steps performed include:    Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component). +   Learn more about [Regression for tabular data](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/overview). + [AutoML text classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_automl_text.ipynb) @@ -211,7 +238,7 @@ The steps performed include:    Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction). -   Learn more about [Vertex AI Training components](https://cloud.google.com/vertex-ai/docs/pipelines/customjob-component). +   Learn more about [Custom training components](https://cloud.google.com/vertex-ai/docs/pipelines/customjob-component). [Model upload, predict, and evaluate using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb) @@ -304,29 +331,5 @@ The steps performed include:    Learn more about [AutoML components](https://cloud.google.com/vertex-ai/docs/pipelines/vertex-automl-component). -[Train custom tabular ML models with many frameworks and import to Vertex AI using Vertex Pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/pipelines/Train_tabular_models_with_many_frameworks_and_import_to_Vertex_AI_using_Pipelines) +   Learn more about [BigQuery ML components](https://cloud.google.com/vertex-ai/docs/pipelines/bigqueryml-component). -Learn how to build a pipeline that does the following: - -* Ingest data -* Transform data -* Clean up data -* Split data into train/test subsets -* Configure model -* Train model using multiple ML frameworks -* Import model into Vertex Model Registry -* [Optional] Deploy model to Vertex Endpoints for serving - -Included pipelines: - -* Train ML model -* * Tabular classification -* * * TensorFlow -* * * PyTorch -* * * XGBoost -* * * Scikit-learn -* * Tabular regression -* * * TensorFlow -* * * PyTorch -* * * XGBoost -* * * Scikit-learn diff --git a/notebooks/official/sdk/README.md b/notebooks/official/sdk/README.md index c3e5f6d1c..3509650e8 100644 --- a/notebooks/official/sdk/README.md +++ b/notebooks/official/sdk/README.md @@ -16,7 +16,7 @@ The steps performed include the following: ``` -   Learn more about [AutoML Video](https://cloud.google.com/vertex-ai/docs/tutorials/video-classification-automl/training). +   Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos). [Custom training using Python package, managed text dataset, and TF Serving container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/SDK_Custom_Training_Python_Package_Managed_Text_Dataset_Tensorflow_Serving_Container.ipynb) @@ -38,5 +38,5 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). diff --git a/notebooks/official/tabnet/README.md b/notebooks/official/tabnet/README.md index cf964df13..996443762 100644 --- a/notebooks/official/tabnet/README.md +++ b/notebooks/official/tabnet/README.md @@ -1,11 +1,11 @@ -[Vertex AI Explainations with TabNet models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb) +[Vertex AI Explanations with TabNet models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb) ``` Learn how to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm. The steps performed are: -* Setup the the project. +* Setup the project. * Download the prediction data of pretrain model onf Syn2 data. * Visualize and understand the feature importance based on the masks output. * Clean up the resource created by this tutorial. diff --git a/notebooks/official/tabular_workflows/README.md b/notebooks/official/tabular_workflows/README.md index 18a017496..8d9b15e41 100644 --- a/notebooks/official/tabular_workflows/README.md +++ b/notebooks/official/tabular_workflows/README.md @@ -1,4 +1,20 @@ +[Train a Prophet Model using Vertex AI Tabular Workflows](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/prophet_on_vertex_pipelines.ipynb) + +``` +Learn how to create several Prophet models using a training Vertex AI Pipeline from Google Cloud Pipeline Components , and then do a batch prediction using the corresponding prediction pipeline. + +The steps performed are: + +1. Train the Prophet models. +1. View the evaluation metrics. +1. Make a batch prediction with the Prophet models. + +``` + +   Learn more about [Google Cloud Pipeline Components](https://cloud.google.com/vertex-ai/docs/pipelines/components-introduction). + + [TabNet Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/tabnet_on_vertex_pipelines.ipynb) ``` diff --git a/notebooks/official/tensorboard/README.md b/notebooks/official/tensorboard/README.md index b9508734f..efb18749e 100644 --- a/notebooks/official/tensorboard/README.md +++ b/notebooks/official/tensorboard/README.md @@ -15,7 +15,7 @@ The steps performed include:    Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Vertex AI TensorBoard custom training with prebuilt container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_custom_training_with_prebuilt_container.ipynb) @@ -34,7 +34,23 @@ The steps performed include:    Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). + + +[Vertex AI TensorBoard Hyperparameter Tuning with the HParams Dashboard](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_hyperparameter_tuning_with_hparams.ipynb) + +``` +This tutorial shows you how to log hyperparameter experiment results in TensorFlow and visualize the results in TensorBoard's Hparams dashboard. + +The steps performed include: + +* Adapt TensorFlow runs to log hyperparameters and metrics. +* Start runs and log them all under one parent directory. +* Visualize the results in TensorBoard's HParams dashboard. + +``` + +   Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview). [Profile model training performance using Vertex AI TensorBoard Profiler](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_profiler_custom_training.ipynb) @@ -54,6 +70,22 @@ The steps performed include:    Learn more about [Vertex AI TensorBoard Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler). +[Profile model training performance using Vertex AI TensorBoard Profiler in custom training with prebuilt container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_profiler_custom_training_with_prebuilt_container.ipynb) + +``` +Learn how to enable the TensorBoard Profiler in Vertex AI for custom training jobs with a prebuilt container. + +The steps performed include: + +- Prepare your custom training code and load your training code as a Python package to a prebuilt container +- Create and run a custom training job that enables the TensorBoard Profiler +- View the TensorBoard Profiler dashboard to debug your model training performance + +``` + +   Learn more about [Vertex AI TensorBoard Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler). + + [Vertex AI TensorBoard integration with Vertex AI Pipelines](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tensorboard/tensorboard_vertex_ai_pipelines_integration.ipynb) ``` diff --git a/notebooks/official/training/README.md b/notebooks/official/training/README.md index 492c8f740..fa99e810c 100644 --- a/notebooks/official/training/README.md +++ b/notebooks/official/training/README.md @@ -2,7 +2,7 @@ [Get started with Vertex AI Distributed Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/get_started_with_vertex_distributed_training.ipynb) ``` -Learn how to use `Vertex AI Distributed Training` for when training with `Vertex AI`. +Learn how to use `Vertex AI Distributed Training` when training with `Vertex AI`. The steps performed include: @@ -66,7 +66,27 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). + + +[Training, tuning and deploying a PyTorch text sentiment classification model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/pytorch-text-sentiment-classification-custom-train-deploy.ipynb) + +``` +Learn to build, train, tune and deploy a PyTorch model on Vertex AI. + +The steps performed include: + +- Create training package for the text classification model. +- Train the model with custom training on Vertex AI. +- Check the created model artifacts. +- Create a custom container for predictions. +- Deploy the trained model to a Vertex AI Endpoint using the custom container for predictions. +- Send online prediction requests to the deployed model and validate. +- Clean up the resources created in this notebook. + +``` + +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Create a distributed custom training job](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb) @@ -83,5 +103,5 @@ The steps performed include: ``` -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). diff --git a/notebooks/official/workbench/README.md b/notebooks/official/workbench/README.md index fea7e5b6b..168432a4e 100644 --- a/notebooks/official/workbench/README.md +++ b/notebooks/official/workbench/README.md @@ -55,7 +55,7 @@ The steps performed include: * Model with BigQuery and the ARIMA model * Evaluate the model * Evaluate the model results using BigQuery ML (on training data) -* Evalute the model results - MAE, MAPE, MSE, RMSE (on test data) +* Evaluate the model results - MAE, MAPE, MSE, RMSE (on test data) * Use the executor feature ``` @@ -107,7 +107,7 @@ The steps performed include:    Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction). -   Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training). +   Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training). [Churn prediction for game developers using Google Analytics 4 and BigQuery ML](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/gaming_churn_prediction/churn_prediction_for_game_developers.ipynb)