mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 22:51:56 +00:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1b8d94a685 |
@@ -49,8 +49,8 @@ The steps performed are:
|
||||
- Make a batch prediction with the BigQuery ML model.
|
||||
- Create a Vertex AI `Dataset` resource.
|
||||
- Train the Vertex AI Forecasting model.
|
||||
- View the Model evaluation.
|
||||
- Make a batch prediction with the Model.
|
||||
- View the Vertex AI Model Evaluation results.
|
||||
- Make a batch prediction with the Vertex AI Forecasting model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
@@ -2,21 +2,19 @@
|
||||
[Online prediction with BigQuery ML](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery_ml/bqml-online-prediction.ipynb)
|
||||
|
||||
```
|
||||
Learn how to train and deploy a churn prediction model for real-time inference, with the data in BigQuery and model trained using BigQuery ML, registered to Vertex AI Model Registry, and deployed to an endpoint on Vertex AI for online predictions.
|
||||
In this tutorial, you fetch the required data from a public BigQuery dataset and prepare it for training.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Using Python & SQL to query the public data in BigQuery
|
||||
- Preparing the data for modeling
|
||||
- Training a classification model using BigQuery ML and registering it to Vertex AI Model Registry
|
||||
- Inspecting the model on Vertex AI Model Registry
|
||||
- Deploying the model to an endpoint on Vertex AI
|
||||
- Making sample online predictions to the model endpoint
|
||||
- Query and fetch the data from the public BigQuery dataset.
|
||||
- Prepare the data for training.
|
||||
- Train a churn classification model using BigQuery ML.
|
||||
- Save the trained model to Vertex AI Model Registry.
|
||||
- Deploy the model to a Vertex AI Endpoint.
|
||||
- Make online prediction requests to the endpoint.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [BigQuery ML](https://cloud.google.com/vertex-ai/docs/beginner/bqml).
|
||||
|
||||
|
||||
[Get started with BigQuery ML Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/bigquery_ml/get_started_with_bqml_training.ipynb)
|
||||
|
||||
|
||||
@@ -93,7 +93,7 @@ The steps performed include:
|
||||
Learn more about [Vertex AI TensorBoard Profiler](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-profiler).
|
||||
|
||||
|
||||
[Get started with Vertex AI Training for XGBoost](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/get_started_vertex_training.ipynb)
|
||||
[Get started with Vertex AI Training for XGBoost](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/custom/get_started_vertex_training_xgboost.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Training` for training a XGBoost custom model.
|
||||
|
||||
@@ -74,6 +74,22 @@ The steps performed include:
|
||||
Learn more about [Vertex AI TensorBoard](https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-overview).
|
||||
|
||||
|
||||
[Custom training autologging - Local script](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_custom_training_autologging_local_script.ipynb)
|
||||
|
||||
```
|
||||
Learn how to autolog paramenters and metrics of an ML experiment running on Vertex AI training by leveraging the integration with Vertex AI Experiments.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Formalize model experiment in a script
|
||||
- Run model traning using local script on Vertex AI Training
|
||||
- Check out ML experiment parameters and metrics in Vertex AI Experiments
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Experiments](https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments).
|
||||
|
||||
|
||||
[Get started with Vertex AI Experiments](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_vertex_experiments.ipynb)
|
||||
|
||||
```
|
||||
@@ -107,7 +123,7 @@ The steps performed include:
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Autologging](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/autologging.ipynb)
|
||||
[Autologging](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/experiments/get_started_with_vertex_experiments_autologging.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Autologging`.
|
||||
|
||||
@@ -169,3 +169,20 @@ The steps performed include:
|
||||
|
||||
Learn more about [Vertex AI Prediction](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions).
|
||||
|
||||
|
||||
[Explaining text classification with Vertex Explainable AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/xai_text_classification_feature_attributions.ipynb)
|
||||
|
||||
```
|
||||
Learn how to configure feature-based explanations using **sampled Shapley method** on a TensorFlow text classification model for online predictions with explanations.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Build and train a TensorFlow text classification model
|
||||
- Upload model for deployment
|
||||
- Deploy model for online prediction
|
||||
- Make online prediction with explanations
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview).
|
||||
|
||||
|
||||
@@ -15,6 +15,38 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Feature Store](https://cloud.google.com/vertex-ai/docs/featurestore).
|
||||
|
||||
|
||||
[Online feature serving and fetching of BigQuery data with Vertex AI Feature Store](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/online_feature_serving_and_fetching_bigquery_data_with_feature_store.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create and use an online feature store instance to host and serve data in `BigQuery` with `Vertex AI Feature Store` in an end to end workflow of feature values serving and fetching user journey.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Provision an online feature store instance to host and serve data.
|
||||
- Register a `BigQuery` view with the online feature store instance and set up the sync job.
|
||||
- Use the online server to fetch feature values for online prediction.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Feature Store](https://cloud.google.com/vertex-ai/docs/featurestore/overview).
|
||||
|
||||
|
||||
[Online feature serving and vector retrieval of BigQuery data with Vertex AI Feature Store](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/online_feature_serving_and_vector_retrieval_bigquery_data_with_feature_store.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create and use an online feature store instance to host and serve data in `BigQuery` with `Vertex AI Feature Store` in an end to end workflow of features serving and vector retrieval user journey.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Provision an online feature store instance to host and serve data.
|
||||
- Create an online feature store instance to serve a `BigQuery` table.
|
||||
- Use the online server to search nearest neighbors.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Feature Store](https://cloud.google.com/vertex-ai/docs/featurestore/overview).
|
||||
|
||||
|
||||
[Using Vertex AI Feature Store with Pandas Dataframe](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/sdk-feature-store-pandas.ipynb)
|
||||
|
||||
```
|
||||
@@ -22,7 +54,8 @@ Learn how to use `Vertex AI Feature Store` with pandas Dataframe.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Ingest Feature values from Pandas DataFrame into Feature Store's Entity types.
|
||||
- Create Featurestore, entity types and features.
|
||||
- Ingest feature values from Pandas DataFrame into Feature Store's Entity types.
|
||||
- Read Entity feature values from Online Feature Store into Pandas DataFrame.
|
||||
- Batch serve feature values from your Feature Store into Pandas DataFrame.
|
||||
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
|
||||
[Vertex AI LLM Evaluation & Batch Inference](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/generative_ai/batch_eval_llm.ipynb)
|
||||
|
||||
```
|
||||
Learn to use Vertex AI to evaluate a large language model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create Vertex AI Pipeline job using a predefined template for bulk inference.
|
||||
- Execute the pipeline using Vertex AI Pipelines.
|
||||
- Produce prediction results against a model for a given dataset.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Overview of Generative AI support on Vertex AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview).
|
||||
|
||||
|
||||
[Vertex AI LLM Reinforcement Learning from Human Feedback](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/genertive_ai/rlhf_tune_llm.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you will use `Vertex AI RLHF` to tune and deploy a large language model model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Set the number of model tuning steps.
|
||||
- Create Vertex AI Pipeline job using a predefined template for tuning.
|
||||
- Execute the pipeline using `Vertex AI Pipelines`.
|
||||
- Perform online prediction with the tuned model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Semantic Search using Embeddings](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/text_embedding_api_semantic_search_with_scann.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, we demonstrate how to create an embedding generated from text and perform a semantic search.
|
||||
|
||||
The steps performed include:
|
||||
- Installation and imports
|
||||
- Create embedding dataset
|
||||
- Create an index
|
||||
- Query the index
|
||||
|
||||
```
|
||||
|
||||
Learn more about [text embedding](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings).
|
||||
|
||||
|
||||
[Text Embedding New API](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/generative_ai/text_embedding_new_api.ipynb)
|
||||
|
||||
```
|
||||
Learn how to call text embedding latest APIs on two
|
||||
new models, textembedding-gecko@latest and textembedding-gecko-multilingual@latest:
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Installation and imports
|
||||
- Generate embeddings
|
||||
|
||||
```
|
||||
|
||||
Learn more about [text embedding api](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings#api_changes_to_models_released_in_or_after_august_2023).
|
||||
|
||||
|
||||
[Vertex AI Tuning a PEFT model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/generative_ai/tune_peft.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI LLM` to tune and deploy a PEFT large language model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Get the Vertex AI LLM model.
|
||||
- Tune the model.
|
||||
- This will automatically create a Vertex AI endpoint and deploy the model to it.
|
||||
- Make a prediction using `Vertex AI LLM`.
|
||||
- Make a prediction using `Vertex AI Prediction`
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Using the Vertex AI SDK with Large Language Models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/generative_ai/vertex_sdk_llm_snippets.ipynb)
|
||||
|
||||
```
|
||||
Learn how to provide text input to Large Language Models available on Vertex AI to test, tune, and deploy generative AI language models.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Use the predict endpoints of Vertex AI PaLM API to receive generative AI responses to a message.
|
||||
- Use the text embedding endpoint to receive a vector representation of a message.
|
||||
- Perform prompt tuning of an LLM, based on input/output training data.
|
||||
|
||||
```
|
||||
|
||||
@@ -1,4 +1,21 @@
|
||||
|
||||
[Using Vertex AI Multimodal Embeddings and Vector Search](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_multimodal_embeddings.ipynb)
|
||||
|
||||
```
|
||||
Learn how to encode custom text embeddings, create an Approximate Nearest Neighbor index, and query against indexes.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
* Convert an image dataset to embeddings
|
||||
* Create an index
|
||||
* Upload embeddings to the index
|
||||
* Create an index endpoint
|
||||
* Deploy the index to the index endpoint
|
||||
* Perform an online query
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Using Vertex AI Matching Engine for StackOverflow Questions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_stack_overflow_embeddings.ipynb)
|
||||
|
||||
```
|
||||
@@ -16,6 +33,27 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).
|
||||
|
||||
|
||||
[Using Vertex AI Vector Search and Vertex AI Embeddings for Text for StackOverflow Questions](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_stack_overflow_embeddings_vertex.ipynb)
|
||||
|
||||
```
|
||||
Learn how to encode text embeddings, create an Approximate Nearest Neighbor index, and query against indexes.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
* Convert a BigQuery dataset to embeddings
|
||||
* Create an index
|
||||
* Upload embeddings to the index
|
||||
* Create an index endpoint
|
||||
* Deploy the index to the index endpoint
|
||||
* Perform an online query
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Vector Search](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).
|
||||
|
||||
Learn more about [Vertex AI Embeddings for Text](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings).
|
||||
|
||||
|
||||
[Using Vertex AI Matching Engine for Text-to-Image Embeddings](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_text_to_image_embeddings.ipynb)
|
||||
|
||||
```
|
||||
@@ -33,6 +71,23 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).
|
||||
|
||||
|
||||
[Using Vertex AI Vector Search for Text-to-Image Embeddings](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_create_text_to_image_embeddings.ipynb)
|
||||
|
||||
```
|
||||
Learn how to encode custom text embeddings, create an Approximate Nearest Neighbor index, and query against indexes.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
* Create ANN index
|
||||
* Create an index endpoint with VPC Network
|
||||
* Deploy ANN index
|
||||
* Perform online query
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Vector Search](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).
|
||||
|
||||
|
||||
[Create Vertex AI Matching Engine index](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/matching_engine/sdk_matching_engine_for_indexing.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -37,7 +37,7 @@ The steps performed include:
|
||||
Learn more about [Object detection for image data](https://cloud.google.com/vertex-ai/docs/training-overview#object_detection_for_images).
|
||||
|
||||
|
||||
[AutoML Video Object Tracking](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ15 Vertex SDK AutoML Object Tracking.ipynb)
|
||||
[AutoML Video Object Tracking](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-object-tracking-batch-prediction.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train a video model and use `Vertex AI Batch Prediction` to do batch predictions.
|
||||
@@ -119,7 +119,7 @@ The steps performed include the following:
|
||||
Learn more about [Entity extraction for text data](https://cloud.google.com/vertex-ai/docs/training-overview#entity_extraction_for_text).
|
||||
|
||||
|
||||
[AutoML Text Sentiment Analysis](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb)
|
||||
[AutoML Text Sentiment Analysis](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-text-sentiment-analysis-batch-prediction.ipynb)
|
||||
|
||||
```
|
||||
The objective of this notebook is to build a AutoML Text Sentiment Analysis model.
|
||||
@@ -140,7 +140,7 @@ The steps performed include the following:
|
||||
Learn more about [Sentiment analysis for text data](https://cloud.google.com/vertex-ai/docs/training-overview#sentiment_analysis_for_text).
|
||||
|
||||
|
||||
[AutoML Video Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ14 Vertex SDK AutoML Video Classification.ipynb)
|
||||
[AutoML Video Classification](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-automl-video-classification-batch-prediction.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `AutoML` to train a video model and use `Vertex AI Batch Prediction` to do batch predictions.
|
||||
|
||||
@@ -14,6 +14,8 @@ The steps performed include:
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
Learn more about [Vertex AI Experiments](https://cloud.google.com/vertex-ai/docs/experiments/intro-vertex-ai-experiments).
|
||||
|
||||
|
||||
[Track parameters and metrics for locally trained models](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/ml_metadata/sdk-metric-parameter-tracking-for-locally-trained-models.ipynb)
|
||||
|
||||
|
||||
@@ -45,22 +45,22 @@ The steps performed include:
|
||||
[AutoML text classification pipelines using google-cloud-pipeline-components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_text_classification_model_evaluation.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` text classification model.
|
||||
Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build and evaluate an `AutoML` text classification model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a Vertex AI `Dataset`.
|
||||
- Train a Automl Text Classification model on the `Dataset` resource.
|
||||
- Train an Automl Text Classification model on the `Dataset` resource.
|
||||
- Import the trained `AutoML model resource` into the pipeline.
|
||||
- Run a `Batch Prediction` job.
|
||||
- Evaulate the AutoML model using the `Classification Evaluation Component`.
|
||||
- Import the classification metrics to the AutoML model resource.
|
||||
- Evaluate the AutoML model using the `Classification Evaluation Component`.
|
||||
- Import the evaluation metrics to the AutoML model resource.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Model Evaluation](https://cloud.google.com/vertex-ai/docs/evaluation/introduction).
|
||||
|
||||
Learn more about [AutoML Text](https://cloud.google.com/vertex-ai/docs/text-data/classification/prepare-data).
|
||||
Learn more about [Classification on text data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_text).
|
||||
|
||||
|
||||
[Evaluating batch prediction results from AutoML Video classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_video_classification_model_evaluation.ipynb)
|
||||
@@ -84,7 +84,7 @@ The steps performed include:
|
||||
Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos).
|
||||
|
||||
|
||||
[Evaluating BatchPrediction results from a Custom Tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/custom_tabular_classification_model_evaluation.ipynb)
|
||||
[Evaluating BatchPrediction results from a custom tabular classification model](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/custom_tabular_classification_model_evaluation.ipynb)
|
||||
|
||||
```
|
||||
In this tutorial, you train a scikit-learn RandomForest model, save it in Vertex AI Model Registry and learn how to evaluate it through a Vertex AI pipeline job using `google_cloud_pipeline_components`.
|
||||
|
||||
@@ -240,26 +240,6 @@ The steps performed include:
|
||||
Learn more about [BigQuery ML components](https://cloud.google.com/vertex-ai/docs/pipelines/bigqueryml-component).
|
||||
|
||||
|
||||
[Loan eligibility prediction using `google-cloud-pipeline-components` and Spark ML](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_dataproc_tabular.ipynb)
|
||||
|
||||
```
|
||||
Learn how to build a Vertex AI pipeline and train a random-forest model using Spark ML for loan-eligibility classification problem.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
* Use the `DataprocPySparkBatchOp` to preprocess data.
|
||||
* Create a Vertex AI dataset resource on the training data.
|
||||
* Train a random forest model using PySpark.
|
||||
* Build a Vertex AI pipeline and run the training job.
|
||||
* Use the Spark serving image in order to deploy a Spark model on Vertex AI Endpoint.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction).
|
||||
|
||||
Learn more about [Dataproc components](https://cloud.google.com/vertex-ai/docs/pipelines/dataproc-component).
|
||||
|
||||
|
||||
[Model train, upload, and deploy using Google Cloud Pipeline Components](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/google_cloud_pipeline_components_model_train_upload_deploy.ipynb)
|
||||
|
||||
```
|
||||
@@ -301,6 +281,25 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Model components](https://cloud.google.com/vertex-ai/docs/pipelines/model-endpoint-component).
|
||||
|
||||
|
||||
[Vertex AI Pipelines with KFP 2.x](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/kfp2_pipeline.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI Pipelines` and KFP 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Create a KFP pipeline:
|
||||
- Create a `BigQuery Dataset` resource.
|
||||
- Export the dataset.
|
||||
- Train an XGBoost `Model` resource.
|
||||
- Create an `Endpoint` resource.
|
||||
- Deploys the `Model` resource to the `Endpoint` resource.
|
||||
- Compile the KFP pipeline.
|
||||
- Execute the KFP pipeline using `Vertex AI Pipelines`
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Lightweight Python function-based components, and component I/O](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/lightweight_functions_component_io_kfp.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -15,6 +15,61 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Batch Prediction](https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions).
|
||||
|
||||
|
||||
[Get started with Custom Prediction Routine (CPR)](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/get_started_with_cpr.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use Custom Prediction Routine for `Vertex AI Predictions`.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Write a custom data preprocessor.
|
||||
- Train the model.
|
||||
- Build a custom scikit-learn serving container with custom data preprocessing using the Custom Prediction Routine model server.
|
||||
- Test the model serving container locally.
|
||||
- Upload and deploy the model serving container to Vertex AI Endpoint.
|
||||
- Make a prediction request.
|
||||
- Build a custom scikit-learn serving container with custom predictor (post-processing) using the Custom Prediction Routine model server.
|
||||
- Implement custom predictor.
|
||||
- Test the model serving container locally.
|
||||
- Upload and deploy the model serving container to Vertex AI Endpoint.
|
||||
- Make a prediction request.
|
||||
- Build a custom scikit-learn serving container with custom predictor and HTTP request handler using the Custom Prediction Routine model server.
|
||||
- Implement a custom handler.
|
||||
- Test the model serving container locally.
|
||||
- Upload and deploy the model serving container to Vertex AI Endpoint.
|
||||
- Make a prediction request.
|
||||
- Customize the Dockerfile for a custom scikit-learn serving container with custom predictor and HTTP request handler using the Custom Prediction Routine model server.
|
||||
- Implement a custom Dockerfile.
|
||||
- Test the model serving container locally.
|
||||
- Upload and deploy the model serving container to Vertex AI Endpoint.
|
||||
- Make a prediction request.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Custom prediction routines](https://cloud.google.com/vertex-ai/docs/predictions/custom-prediction-routines).
|
||||
|
||||
|
||||
[Vertex AI LLM and streaming prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/llm_streaming_prediction.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use Vertex AI LLM to download pretrained LLM model, make predictions and finetuning the model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Load a pretrained text generation model.
|
||||
- Make a non-streaming prediction
|
||||
- Load a pretrained text generation model, which supports streaming.
|
||||
- Make a streaming prediction
|
||||
- Load a pretrained chat model.
|
||||
- Do a local interactive chat session.
|
||||
- Do a batch prediction with a text generation model.
|
||||
- Do a batch prediction with a text embedding model.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Language Models](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.language_models.TextGenerationModel#vertexai_language_models_TextGenerationModel_predict_streaming).
|
||||
|
||||
|
||||
[Serving PyTorch image models with prebuilt containers on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/pytorch_image_classification_with_prebuilt_serving_containers.ipynb)
|
||||
|
||||
```
|
||||
@@ -51,3 +106,21 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Vertex AI SDK 2.0 Vertex AI Remote Prediction](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/prediction/sdk2_remote_prediction.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- For scikit-learn, PyTorch, TensorFlow, PyTorch Lightning
|
||||
- Train the model remotely.
|
||||
- Uptrain the pretrained model remotely.
|
||||
- Evaluate both the pretrained and uptrained model.
|
||||
- Make a prediction remotely
|
||||
|
||||
```
|
||||
|
||||
|
||||
@@ -40,3 +40,51 @@ The steps performed include:
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Train a pytorch model with Vertex AI SDK 2.0 and Bigframes](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/sdk2_bigframes_pytorch.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Initialize a dataframe from a BigQuery table and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- Train the model remotely and evaluate the model locally
|
||||
|
||||
```
|
||||
|
||||
Learn more about [bigframes](https://cloud.google.com/bigquery/docs/).
|
||||
|
||||
|
||||
[Train a scikit-learn model with Vertex AI SDK 2.0 and Bigframes](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/sdk2_bigframes_sklearn.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Initialize a dataframe from a BigQuery table and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- Train the model remotely and evaluate the model locally
|
||||
|
||||
```
|
||||
|
||||
Learn more about [bigframes](https://cloud.google.com/bigquery/docs/).
|
||||
|
||||
|
||||
[Train a Tensorflow Keras model with Vertex AI SDK 2.0 and Bigframes](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/sdk/sdk2_bigframes_tensorflow.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Initialize a dataframe from a BigQuery table and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- Train the model remotely and evaluate the model locally
|
||||
|
||||
```
|
||||
|
||||
Learn more about [bigframes](https://cloud.google.com/bigquery/docs/).
|
||||
|
||||
|
||||
@@ -35,6 +35,21 @@ The steps performed include:
|
||||
Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet).
|
||||
|
||||
|
||||
[Train a TabNet model using Vertex AI Remote Training with Vertex AI SDK 2.0](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/sdk2_remote_tabnet_training.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download and split the dataset
|
||||
- Ingest the data in a Dataframe and perform transformations.
|
||||
- Train a tabular classification model.
|
||||
- Train a tabular regression model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Vertex AI TabNet](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/tabnet_vertex_tutorial.ipynb)
|
||||
|
||||
```
|
||||
|
||||
@@ -20,7 +20,7 @@ The steps performed are:
|
||||
[TabNet Pipeline](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabular_workflows/tabnet_on_vertex_pipelines.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create two classification models using Vertex AI TabNet Tabular Workflows.
|
||||
Learn how to create classification models on tabular data using two of the Vertex AI TabNet Tabular Workflows.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
|
||||
@@ -1,4 +1,20 @@
|
||||
|
||||
[Distributed Vertex AI Hyperparameter Tuning](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/distributed_hyperparameter_tuning.ipynb)
|
||||
|
||||
```
|
||||
In this notebook, you create a custom trained model from a Python script in a Docker container.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Training using a Python package.
|
||||
- Report accuracy when hyperparameter tuning.
|
||||
- Save the model artifacts to Cloud Storage using GCSFuse.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
|
||||
[Get started with Vertex AI Distributed Training](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/get_started_with_vertex_distributed_training.ipynb)
|
||||
|
||||
```
|
||||
@@ -33,6 +49,23 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
|
||||
[Vertex AI Hyperparameter Tuning for XGBoost](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/hyperparameter_tuning_xgboost.ipynb)
|
||||
|
||||
```
|
||||
Learn how to use `Vertex AI Hyperparameter Tuning` for training a XGBoost custom model.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Training using a Python package.
|
||||
- Report accuracy when hyperparameter tuning.
|
||||
- Save the model artifacts to Cloud Storage using GCSFuse.
|
||||
- Create a `Vertex AI Model` resource.
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Hyperparameter Tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview).
|
||||
|
||||
|
||||
[PyTorch image classification multi-node distributed data parallel training on cpu using Vertex training with custom container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/multi_node_ddp_gloo_vertex_training_with_custom_container.ipynb)
|
||||
|
||||
```
|
||||
@@ -43,7 +76,7 @@ The steps performed include:
|
||||
- Setting up your GCP project : Setting up the PROJECT_ID, REGION & SERVICE_ACCOUNT
|
||||
- Creating a cloud storage bucket
|
||||
- Building Custom Container using Artifact Registry and Docker
|
||||
- Create a Vertex AI tensorboard instance to store your Vertex AI experiment
|
||||
- Create a Vertex AI TensorBoard instance to store your Vertex AI experiment
|
||||
- Run a Vertex AI SDK CustomContainerTrainingJob
|
||||
|
||||
```
|
||||
@@ -51,22 +84,20 @@ The steps performed include:
|
||||
Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[PyTorch Image Classification Multi-Node Distributed Data Parallel Training on GPU using Vertex AI Training with Custom Container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/multi_node_ddp_nccl_vertex_training_with_custom_container.ipynb)
|
||||
[PyTorch image classification multi-node NCCL distributed data parallel training on cpu using Vertex training with custom container](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/multi_node_ddp_nccl_vertex_training_with_custom_container.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create a distributed PyTorch training job using Vertex AI SDK for Python and custom containers.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Setting up your GCP project : Setting up the PROJECT_ID, REGION & SERVICE_ACCOUNT
|
||||
- Creating a cloud storage bucket
|
||||
- Building Custom Container using Artifact Registry and Docker
|
||||
- Create a Vertex AI Tensorboard Instance to store your Vertex AI experiment
|
||||
- Create a Vertex AI tensorboard instance to store your Vertex AI experiment
|
||||
- Run a Vertex AI SDK CustomContainerTrainingJob
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Training, tuning and deploying a PyTorch text sentiment classification model on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/pytorch-text-sentiment-classification-custom-train-deploy.ipynb)
|
||||
@@ -89,10 +120,58 @@ The steps performed include:
|
||||
Learn more about [Custom training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Create a distributed custom training job](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb)
|
||||
[Train PyTorch model on Vertex AI with data from Cloud Storage](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/pytorch_gcs_data_training.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create a distributed training job using Vertex AI SDK for Python.
|
||||
Learn how to create a training job using PyTorch and a dataset stored on Cloud Storage.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Write a custom training script that creates your train & test datasets and trains the model.
|
||||
- Run a Vertex AI SDK `CustomTrainingJob`
|
||||
|
||||
```
|
||||
|
||||
Learn more about [PyTorch integration in Vertex AI](https://cloud.google.com/vertex-ai/docs/start/pytorch).
|
||||
|
||||
|
||||
[Vertex AI SDK 2.0 Vertex AI Remote Hyperparameter Tuning for OSS ML frameworks](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/sdk2_remote_hyperparameter_tuning.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- For scikit-learn, PyTorch, TensorFlow, PyTorch Lightning, Tabnet
|
||||
- Tune the model remotely.
|
||||
- Get the best model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Vertex AI SDK 2.0 Vertex AI Remote Training for OSS ML frameworks](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/sdk2_remote_training.ipynb)
|
||||
|
||||
```
|
||||
Learn to use `Vertex AI SDK 2.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Download and split the dataset
|
||||
- Perform transformations as a Vertex AI remote training.
|
||||
- For scikit-learn, PyTorch, TensorFlow, PyTorch Lightning
|
||||
- Train the model remotely.
|
||||
- Uptrain the pretrained model remotely.
|
||||
- Evaluate both the pretrained and uptrained model.
|
||||
|
||||
```
|
||||
|
||||
|
||||
[Distributed XGBoost training with Dask](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb)
|
||||
|
||||
```
|
||||
Learn how to create a distributed training job using XGBoost with Dask.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
|
||||
@@ -147,10 +147,6 @@ The steps performed include:
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [Vertex AI Training](https://cloud.google.com/vertex-ai/docs/training/custom-training).
|
||||
|
||||
|
||||
[Predictive Maintenance using Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/predictive_maintainance/predictive_maintenance_usecase.ipynb)
|
||||
|
||||
@@ -242,7 +238,7 @@ The steps performed are:
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [Dataproc](https://cloud.google.com/vertex-ai/docs/pipelines/dataproc-component).
|
||||
Learn more about [Dataproc Serverless for Spark](https://cloud.google.com/dataproc-serverless/docs/guides/bigquery-connector-spark-example).
|
||||
|
||||
|
||||
[SparkML with Dataproc and BigQuery](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/spark/spark_ml.ipynb)
|
||||
@@ -271,28 +267,3 @@ The steps performed are:
|
||||
|
||||
Learn more about [Dataproc](https://cloud.google.com/vertex-ai/docs/pipelines/dataproc-component).
|
||||
|
||||
|
||||
[Telecom subscriber churn prediction on Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/workbench/subscriber_churn_prediction/telecom-subscriber-churn-prediction.ipynb)
|
||||
|
||||
```
|
||||
This tutorial shows you how to do exploratory data analysis, preprocess data, train, deploy and get predictions from a churn prediction model on a tabular churn dataset.
|
||||
|
||||
The steps performed include:
|
||||
|
||||
- Load data from a Cloud Storage path
|
||||
- Perform exploratory data analysis (EDA)
|
||||
- Preprocess the data
|
||||
- Train a scikit-learn model
|
||||
- Evaluate the scikit-learn model
|
||||
- Save the model to a Cloud Storage path
|
||||
- Create a model and an endpoint in Vertex AI
|
||||
- Deploy the trained model to an endpoint
|
||||
- Generate predictions and explanations on test data from the hosted model
|
||||
- Undeploy the model resource
|
||||
|
||||
```
|
||||
|
||||
Learn more about [Vertex AI Workbench](https://cloud.google.com/vertex-ai/docs/workbench/introduction).
|
||||
|
||||
Learn more about [Vertex Explainable AI](https://cloud.google.com/vertex-ai/docs/explainable-ai/overview).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user