AutoML Tabular pipelines using google-cloud-pipeline-components
Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` tabular classification model.
The steps performed include:
- Create a KFP pipeline:
- Create a `Dataset` resource.
- Train an AutoML tabular classification `Model` resource.
- Create an `Endpoint` resource.
- Deploys the `Model` resource to the `Endpoint` resource.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Learn more about AutoML components.
Pipeline control structures using the KFP SDK
Learn how to use the KFP SDK to build pipelines that use loops and conditionals, including nested examples.
The steps performed include:
- Create a KFP pipeline:
- Use control flow components
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Custom training with pre-built Google Cloud Pipeline Components
Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build a custom model.
The steps performed include:
- Create a KFP pipeline:
- Train a custom model.
- Upload the trained model as a `Model` resource.
- Create an `Endpoint` resource.
- Deploy the `Model` resource to the `Endpoint` resource.
- Make a batch prediction request.
Learn more about Vertex AI Pipelines.
Learn more about Vertex AI Training components.
In this tutorial, you train a scikit-learn tabular classification model and create batch prediction job for it through a Vertex AI pipeline using `google_cloud_pipeline_components`.
The steps performed include:
- Create a dataset in BigQuery.
- Set some data aside from the source dataset for batch prediction.
- Create a custom python package for training application.
- Upload the python package to Cloud Storage.
- Create a Vertex AI Pipeline that:
- creates a Vertex AI Dataset from the source dataset.
- trains a scikit-learn RandomForest classification model on the dataset.
- uploads the trained model to Vertex AI Model Registry.
- runs a batch prediction job with the model on the test data.
- Check the prediction results from the destination table in BigQuery.
- Clean up the resources created in this notebook.
Learn more about Vertex AI Pipelines.
Learn more about Vertex AI Batch Prediction components.
AutoML image classification pipelines using google-cloud-pipeline-components
Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` image classification model.
The steps performed include:
- Create a KFP pipeline:
- Create a `Dataset` resource.
- Train an AutoML image classification `Model` resource.
- Create an `Endpoint` resource.
- Deploys the `Model` resource to the `Endpoint` resource.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Learn more about AutoML components.
AutoML tabular regression pipelines using google-cloud-pipeline-components
Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` tabular regression model.
The steps performed include:
- Create a KFP pipeline:
- Create a `Dataset` resource.
- Train an AutoML tabular regression `Model` resource.
- Create an `Endpoint` resource.
- Deploys the `Model` resource to the `Endpoint` resource.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Learn more about AutoML components.
AutoML text classification pipelines using google-cloud-pipeline-components
Learn to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build an `AutoML` text classification model.
The steps performed include:
- Create a KFP pipeline:
- Create a `Dataset` resource.
- Train an AutoML text classification `Model` resource.
- Create an `Endpoint` resource.
- Deploys the `Model` resource to the `Endpoint` resource.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Learn more about AutoML components.
Training an acquisition-prediction model using Swivel, BigQuery ML and Vertex AI Pipelines
Learn how to build a simple BigQuery ML pipeline using Vertex AI pipelines in order to calculate text embeddings of content from articles and classify them
into the *corporate acquisitions* category.
The steps performed include:
- Creating a component for Dataflow job that ingests data to BigQuery.
- Creating a component for preprocessing steps to run on the data in BigQuery.
- Creating a component for training a logistic regression model using BigQuery ML.
- Building and configuring a Kubeflow DSL pipeline with all the created components.
- Compiling and running the pipeline in Vertex AI Pipelines.
Learn more about Vertex AI Pipelines.
Learn more about BigQuery ML components.
Loan eligibility prediction using google-cloud-pipeline-components and Spark ML
Learn how to build a Vertex AI pipeline and train a random-forest model using Spark ML for loan-eligibility classification problem.
The steps performed include:
* Use the `DataprocPySparkBatchOp` to preprocess data.
* Create a Vertex AI dataset resource on the training data.
* Train a random forest model using PySpark.
* Build a Vertex AI pipeline and run the training job.
* Use the Spark serving image in order to deploy a Spark model on Vertex AI Endpoint.
Learn more about Vertex AI Pipelines.
Learn more about Dataproc components.
Model train, upload, and deploy using Google Cloud Pipeline Components
Learn how to use `Vertex AI Pipelines` and `Google Cloud Pipeline Components` to build and deploy a custom model.
The steps performed include:
- Create a KFP pipeline:
- Train a custom model.
- Uploads the trained model as a `Model` resource.
- Creates an `Endpoint` resource.
- Deploys the `Model` resource to the `Endpoint` resource.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Learn more about Vertex AI Training components.
Model upload, predict, and evaluate using google-cloud-pipeline-components
Learn how to evaluate a custom model using a pipeline with components from `google_cloud_pipeline_components` and a custom pipeline component you build.
The steps performed include:
- Upload a pre-trained model as a `Model` resource.
- Run a `BatchPredictionJob` on the `Model` resource with ground truth data.
- Generate evaluation `Metrics` artifact about the `Model` resource.
- Compare the evaluation metrics to a threshold.
Learn more about Vertex AI Pipelines.
Learn more about Vertex AI Model components.
Lightweight Python function-based components, and component I/O
Learn to use the KFP SDK to build lightweight Python function-based components, and then you learn to use `Vertex AI Pipelines` to execute the pipeline.
The steps performed include:
- Build Python function-based KFP components.
- Construct a KFP pipeline.
- Pass *Artifacts* and *parameters* between components, both by path reference and by value.
- Use the `kfp.dsl.importer` method.
- Compile the KFP pipeline.
- Execute the KFP pipeline using `Vertex AI Pipelines`
Learn more about Vertex AI Pipelines.
Metrics visualization and run comparison using the KFP SDK
Learn how to use the KFP SDK to build pipelines that generate evaluation metrics.
The steps performed include:
- Create KFP components:
- Generate ROC curve and confusion matrix visualizations for classification results
- Write metrics
- Create KFP pipelines.
- Execute KFP pipelines
- Compare metrics across pipeline runs
Learn more about Vertex AI Pipelines.
Pipelines introduction for KFP
Learn how to use the KFP SDK to build pipelines that generate evaluation metrics.
The steps performed include:
- Define and compile a `Vertex AI` pipeline.
- Specify which service account to use for a pipeline run.
Learn more about Vertex AI Pipelines.
BQML and AutoML - Experimenting with Vertex AI
Learn how to use `Vertex AI Predictions` for rapid prototyping a model.
The steps performed include:
- Creating a BigQuery and Vertex AI training dataset.
- Training a BigQuery ML and AutoML model.
- Extracting evaluation metrics from the BigQueryML and AutoML models.
- Selecting the best trained model.
- Deploying the best trained model.
- Testing the deployed model infrastructure.
Learn more about AutoML components.
Learn more about BigQuery ML components.