xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: rebrand (#2343)

Browse Source
This commit is contained in:
Andrew Ferlitsch
2023-10-06 13:43:37 +00:00
committed by GitHub
parent 0041b7da91
commit 2b94e8b51e
1 changed files with 15 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/community/ml_ops/stage6/get_started_with_matching_engine_swivel.ipynb
+15 -15
View File
@@ -38,7 +38,7 @@
"id": "JAPoU8Sm5E6e"
},
"source": [
"# E2E ML on GCP: MLOps stage 6 : serving: get started with Vertex AI Matching Engine and Swivel builtin algorithm\n",
"# E2E ML on GCP: MLOps stage 6 : serving: get started with Vertex AI Vector Search and Swivel builtin algorithm\n",
"\n",
"<table align=\"left\">\n",
"\n",
@@ -84,12 +84,12 @@
"source": [
"### Objective\n",
"\n",
"In this notebook, you learn how to train custom embeddings using Vertex AI Pipelines and subsequently train and deploy a matching engine index using the embeddings.\n",
"In this notebook, you learn how to train custom embeddings using Vertex AI Pipelines and subsequently train and deploy a vector search index using the embeddings.\n",
"\n",
"This tutorial uses the following Google Cloud ML services:\n",
"\n",
"- `Vertex AI Swivel` builtin algorithm\n",
"- `Vertex AI Matching Engine`\n",
"- `Vertex AI Vector Search`\n",
"- `Vertex AI Batch Prediction`\n",
"\n",
"The steps performed include:\n",
@@ -97,10 +97,10 @@
"1. Train the `Swivel` algorithm to generate embeddings (encoder) for the dataset.\n",
"2. Make example predictions (embeddings) from then trained encoder.\n",
"3. Generate embeddings using the trained `Swivel` builtin algorithm.\n",
"4. Store embeddings to format supported by `Matching Engine`.\n",
"5. Create a `Matching Engine Index` for the embeddings.\n",
"6. Deploy the `Matching Engine Index` to a `Index Endpoint`.\n",
"7. Make a matching engine prediction request."
"4. Store embeddings to format supported by `Vector Search`.\n",
"5. Create a `Vector Search Index` for the embeddings.\n",
"6. Deploy the `Vector Search Index` to a `Index Endpoint`.\n",
"7. Make a vector search prediction request."
]
},
{
@@ -1336,11 +1336,11 @@
"source": [
"### Make a batch prediction\n",
"\n",
"[Batch prediction](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions) is used to asynchronously make predictions on a batch of input data. This is recommended if you have a large input size and do not need an immediate response, such as getting embeddings for candidate objects in order to create an index for a nearest neighbor search service such as [Vertex Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).\n",
"[Batch prediction](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions) is used to asynchronously make predictions on a batch of input data. This is recommended if you have a large input size and do not need an immediate response, such as getting embeddings for candidate objects in order to create an index for a nearest neighbor search service such as [Vertex AI Vector Search](https://cloud.google.com/vertex-ai/docs/matching-engine/overview).\n",
"\n",
"### Create the batch input file\n",
"\n",
"Next, you generate the batch input file to generate embeddings for the dataset, which you subsequently use to create an index with `Vertex AI Matching Engine`. In this example, the dataset contains a 200000 unique identifiers (1...200000). You will use the trained encoder to generate a predicted embedding for each unique identifier.\n",
"Next, you generate the batch input file to generate embeddings for the dataset, which you subsequently use to create an index with `Vertex AI Vector Search`. In this example, the dataset contains a 200000 unique identifiers (1...200000). You will use the trained encoder to generate a predicted embedding for each unique identifier.\n",
"\n",
"The input data needs to be on Cloud Storage and in JSONL format. You can use the sample query object file provided below. Like with online prediction, it's recommended to have the `key` field so that you can associate each output embedding with its corresponding input."
]
@@ -1544,7 +1544,7 @@
"id": "qhIBCQ7dDSbW"
},
"source": [
"### Create Matching Engine Index\n",
"### Create Vector Search Index\n",
"\n",
"Next, you create the index for your embeddings. Currently, two indexing algorithms are supported:\n",
"\n",
@@ -1569,7 +1569,7 @@
"\n",
"This may take upto 30 minutes.\n",
"\n",
"Learn more about [Configuring Matching Engine Indexes](https://cloud.google.com/vertex-ai/docs/matching-engine/configuring-indexes)."
"Learn more about [Configuring Vector Search Indexes](https://cloud.google.com/vertex-ai/docs/matching-engine/configuring-indexes)."
]
},
{
@@ -1608,7 +1608,7 @@
"source": [
"## Setup VPC peering network\n",
"\n",
"To use a `Matching Engine Index`, you setup a VPC peering network between your project and the `Vertex AI Matching Engine` service project. This eliminates additional hops in network traffic and allows using efficient gRPC protocol.\n",
"To use a `Vector Search Index`, you setup a VPC peering network between your project and the `Vertex AI Vector Search` service project. This eliminates additional hops in network traffic and allows using efficient gRPC protocol.\n",
"\n",
"Learn more about [VPC peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering).\n",
"\n",
@@ -1713,7 +1713,7 @@
"source": [
"#### Construct the full network name\n",
"\n",
"You need to have the full network resource name when you subsequently create an `Matching Engine Index Endpoint` resource for VPC peering."
"You need to have the full network resource name when you subsequently create an `Vector Search Index Endpoint` resource for VPC peering."
]
},
{
@@ -1735,7 +1735,7 @@
"source": [
"### Create an IndexEndpoint with VPC Network\n",
"\n",
"Next, you create a `Matching Engine Index Endpoint`, similar to the concept of creating a `Private Endpoint` for prediction with a peer-to-peer network.\n",
"Next, you create a `Vector Search Index Endpoint`, similar to the concept of creating a `Private Endpoint` for prediction with a peer-to-peer network.\n",
"\n",
"To create the `Index Endpoint` resource, you call the method `create()` with the following parameters:\n",
"\n",
@@ -1768,7 +1768,7 @@
"id": "8Ew1UgcIIiJG"
},
"source": [
"### Deploy the `Matching Engine Index` to the `Index Endpoint` resource\n",
"### Deploy the `Vector Search Index` to the `Index Endpoint` resource\n",
"\n",
"Next, deploy your index to the `Index Endpoint` using the method `deploy_index()` with the following parameters:\n",
"\n",