mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ivanmkc
|
5657ccb83b | ||
|
ivanmkc
|
572cce3bf1 | ||
|
ivanmkc
|
516dea7436 |
@@ -97,11 +97,9 @@
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"* **Prepare a VPC network**. To reduce any network overhead that might lead to unnecessary increase in overhead latency, it is best to call the ANN endpoints from your VPC via a direct [VPC Peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering) connection. The following section describes how to setup a VPC Peering connection if you don't have one. This is a one-time initial setup task. You can also reuse existing VPC network and skip this section.\n",
|
||||
"* **WARNING:** The MatchingIndexEndpoint.match method (to create online queries against your deployed index) has to be executed in a Vertex AI Workbench notebook instance that is created with the following requirements:\n",
|
||||
" * **In the same region as where your ANN service is deployed** (for example, if you set `REGION = \"us-central1\"` as same as the tutorial, the notebook instance has to be in `us-central1`).\n",
|
||||
" * **Make sure you select the VPC network you created for ANN service** (instead of using the \"default\" one). That is, you will have to create the VPC network below and then create a new notebook instance that uses that VPC. \n",
|
||||
" * If you run it in the colab or a Vertex AI Workbench notebook instance in a different VPC network or region, the gRPC API will fail to peer the network (InactiveRPCError)."
|
||||
"* **Prepare a VPC network**. To reduce any network overhead that might lead to unnecessary increase in overhead latency, it is best to call the ANN endpoints from your VPC via a direct [VPC Peering](https://cloud.google.com/vertex-ai/docs/general/vpc-peering) connection. \n",
|
||||
" * The following section describes how to setup a VPC Peering connection if you don't have one. \n",
|
||||
" * This is a one-time initial setup task. You can also reuse existing VPC network and skip this section."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -112,11 +110,11 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"<your_project_id>\" # @param {type:\"string\"}\n",
|
||||
"PROJECT_ID = \"python-docs-samples-tests\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"NETWORK_NAME = \"my-vpc-network\" # @param {type:\"string\"}\n",
|
||||
"NETWORK_NAME = \"ann-vpc-network\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"PEERING_RANGE_NAME = \"my-haystack-range\""
|
||||
"PEERING_RANGE_NAME = \"ann-haystack-range\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -143,6 +141,7 @@
|
||||
"! gcloud compute addresses create {PEERING_RANGE_NAME} --global --prefix-length=16 --network={NETWORK_NAME} --purpose=VPC_PEERING --project={PROJECT_ID} --description=\"peering range\"\n",
|
||||
"\n",
|
||||
"# Set up peering with service networking\n",
|
||||
"# Your account must have the \"Compute Network Admin\" role to run the following.\n",
|
||||
"! gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --network={NETWORK_NAME} --ranges={PEERING_RANGE_NAME} --project={PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
@@ -155,6 +154,20 @@
|
||||
"* Authentication: Rerun the `gcloud auth login` command in the Vertex AI Workbench notebook terminal when you are logged out and need the credential again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d5de53b31bf1"
|
||||
},
|
||||
"source": [
|
||||
"## Make sure the following cells are run from inside the VPC network that you created in the previous step.\n",
|
||||
"\n",
|
||||
"* **WARNING:** The MatchingIndexEndpoint.match method (to create online queries against your deployed index) has to be executed in a Vertex AI Workbench notebook instance that is created with the following requirements:\n",
|
||||
" * **In the same region as where your ANN service is deployed** (for example, if you set `REGION = \"us-central1\"` as same as the tutorial, the notebook instance has to be in `us-central1`).\n",
|
||||
" * **Make sure you select the VPC network you created for ANN service** (instead of using the \"default\" one). That is, you will have to create the VPC network below and then create a new notebook instance that uses that VPC. \n",
|
||||
" * If you run it in the colab or a Vertex AI Workbench notebook instance in a different VPC network or region, \"Create Online Queries\" section will fail."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -271,7 +284,7 @@
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"PROJECT_ID = \"\"\n",
|
||||
"PROJECT_ID = \"python-docs-samples-tests\"\n",
|
||||
"\n",
|
||||
"# Get your Google Cloud project ID from gcloud\n",
|
||||
"if not os.getenv(\"IS_TESTING\"):\n",
|
||||
@@ -734,6 +747,28 @@
|
||||
"INDEX_RESOURCE_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "0f1a9fbecabb"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"Using the resource name, you can retrieve an existing MatchingEngineIndex."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1ddb70647d98"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tree_ah_index = aiplatform.MatchingEngineIndex(INDEX_RESOURCE_NAME)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -742,7 +777,7 @@
|
||||
"source": [
|
||||
"### Create Brute Force Index (for Ground Truth)\n",
|
||||
"\n",
|
||||
"The brute force index uses a naive brute force method to find the nearest neighbors. This method is not fast or efficient. Hence brute force indices are not recommended for production usage. They are to be used to find the \"ground truth\" set of neighbors, so that the \"ground truth\" set can be used to measure recall of the indices being tuned for production usage. To ensure an apples to apples comparison, the `distanceMeasureType` and `featureNormType`, `dimensions` of the brute force index should match those of the production indices being tuned.\n",
|
||||
"The brute force index uses a naive brute force method to find the nearest neighbors. This method is not fast or efficient. Hence brute force indices are not recommended for production usage. They are to be used to find the \"ground truth\" set of neighbors, so that the \"ground truth\" set can be used to measure recall of the indices being tuned for production usage. To ensure an apples to apples comparison, the `distanceMeasureType` and `dimensions` of the brute force index should match those of the production indices being tuned.\n",
|
||||
"\n",
|
||||
"Create the brute force index configuration:"
|
||||
]
|
||||
@@ -777,6 +812,19 @@
|
||||
"INDEX_BRUTE_FORCE_RESOURCE_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "865fcad494d7"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"brute_force_index = aiplatform.MatchingEngineIndex(\n",
|
||||
" \"projects/1012616486416/locations/us-central1/indexes/6738176690918260736\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -941,7 +989,7 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"DEPLOYED_INDEX_ID = \"tree_ah_glove_deployed\""
|
||||
"DEPLOYED_INDEX_ID = f\"tree_ah_glove_deployed_{TIMESTAMP}\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -976,7 +1024,7 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"DEPLOYED_BRUTE_FORCE_INDEX_ID = \"glove_brute_force_deployed\""
|
||||
"DEPLOYED_BRUTE_FORCE_INDEX_ID = f\"glove_brute_force_deployed_{TIMESTAMP}\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1023,344 +1071,13 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Test query\n",
|
||||
"query = [\n",
|
||||
" -0.11333,\n",
|
||||
" 0.48402,\n",
|
||||
" 0.090771,\n",
|
||||
" -0.22439,\n",
|
||||
" 0.034206,\n",
|
||||
" -0.55831,\n",
|
||||
" 0.041849,\n",
|
||||
" -0.53573,\n",
|
||||
" 0.18809,\n",
|
||||
" -0.58722,\n",
|
||||
" 0.015313,\n",
|
||||
" -0.014555,\n",
|
||||
" 0.80842,\n",
|
||||
" -0.038519,\n",
|
||||
" 0.75348,\n",
|
||||
" 0.70502,\n",
|
||||
" -0.17863,\n",
|
||||
" 0.3222,\n",
|
||||
" 0.67575,\n",
|
||||
" 0.67198,\n",
|
||||
" 0.26044,\n",
|
||||
" 0.4187,\n",
|
||||
" -0.34122,\n",
|
||||
" 0.2286,\n",
|
||||
" -0.53529,\n",
|
||||
" 1.2582,\n",
|
||||
" -0.091543,\n",
|
||||
" 0.19716,\n",
|
||||
" -0.037454,\n",
|
||||
" -0.3336,\n",
|
||||
" 0.31399,\n",
|
||||
" 0.36488,\n",
|
||||
" 0.71263,\n",
|
||||
" 0.1307,\n",
|
||||
" -0.24654,\n",
|
||||
" -0.52445,\n",
|
||||
" -0.036091,\n",
|
||||
" 0.55068,\n",
|
||||
" 0.10017,\n",
|
||||
" 0.48095,\n",
|
||||
" 0.71104,\n",
|
||||
" -0.053462,\n",
|
||||
" 0.22325,\n",
|
||||
" 0.30917,\n",
|
||||
" -0.39926,\n",
|
||||
" 0.036634,\n",
|
||||
" -0.35431,\n",
|
||||
" -0.42795,\n",
|
||||
" 0.46444,\n",
|
||||
" 0.25586,\n",
|
||||
" 0.68257,\n",
|
||||
" -0.20821,\n",
|
||||
" 0.38433,\n",
|
||||
" 0.055773,\n",
|
||||
" -0.2539,\n",
|
||||
" -0.20804,\n",
|
||||
" 0.52522,\n",
|
||||
" -0.11399,\n",
|
||||
" -0.3253,\n",
|
||||
" -0.44104,\n",
|
||||
" 0.17528,\n",
|
||||
" 0.62255,\n",
|
||||
" 0.50237,\n",
|
||||
" -0.7607,\n",
|
||||
" -0.071786,\n",
|
||||
" 0.0080131,\n",
|
||||
" -0.13286,\n",
|
||||
" 0.50097,\n",
|
||||
" 0.18824,\n",
|
||||
" -0.54722,\n",
|
||||
" -0.42664,\n",
|
||||
" 0.4292,\n",
|
||||
" 0.14877,\n",
|
||||
" -0.0072514,\n",
|
||||
" -0.16484,\n",
|
||||
" -0.059798,\n",
|
||||
" 0.9895,\n",
|
||||
" -0.61738,\n",
|
||||
" 0.054169,\n",
|
||||
" 0.48424,\n",
|
||||
" -0.35084,\n",
|
||||
" -0.27053,\n",
|
||||
" 0.37829,\n",
|
||||
" 0.11503,\n",
|
||||
" -0.39613,\n",
|
||||
" 0.24266,\n",
|
||||
" 0.39147,\n",
|
||||
" -0.075256,\n",
|
||||
" 0.65093,\n",
|
||||
" -0.20822,\n",
|
||||
" -0.17456,\n",
|
||||
" 0.53571,\n",
|
||||
" -0.16537,\n",
|
||||
" 0.13582,\n",
|
||||
" -0.56016,\n",
|
||||
" 0.016964,\n",
|
||||
" 0.1277,\n",
|
||||
" 0.94071,\n",
|
||||
" -0.22608,\n",
|
||||
" -0.021106,\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"response = my_index_endpoint.match(\n",
|
||||
" deployed_index_id=DEPLOYED_INDEX_ID, queries=[query], num_neighbors=NUM_NEIGHBOURS\n",
|
||||
" deployed_index_id=DEPLOYED_INDEX_ID, queries=test[:1], num_neighbors=NUM_NEIGHBOURS\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "_mNwdU9_B_Ez"
|
||||
},
|
||||
"source": [
|
||||
"### Batch Query\n",
|
||||
"\n",
|
||||
"You can run multiple queries in a single match call:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "A0XL0PJ1GoM9"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Test query\n",
|
||||
"queries = [\n",
|
||||
" [\n",
|
||||
" -0.11333,\n",
|
||||
" 0.48402,\n",
|
||||
" 0.090771,\n",
|
||||
" -0.22439,\n",
|
||||
" 0.034206,\n",
|
||||
" -0.55831,\n",
|
||||
" 0.041849,\n",
|
||||
" -0.53573,\n",
|
||||
" 0.18809,\n",
|
||||
" -0.58722,\n",
|
||||
" 0.015313,\n",
|
||||
" -0.014555,\n",
|
||||
" 0.80842,\n",
|
||||
" -0.038519,\n",
|
||||
" 0.75348,\n",
|
||||
" 0.70502,\n",
|
||||
" -0.17863,\n",
|
||||
" 0.3222,\n",
|
||||
" 0.67575,\n",
|
||||
" 0.67198,\n",
|
||||
" 0.26044,\n",
|
||||
" 0.4187,\n",
|
||||
" -0.34122,\n",
|
||||
" 0.2286,\n",
|
||||
" -0.53529,\n",
|
||||
" 1.2582,\n",
|
||||
" -0.091543,\n",
|
||||
" 0.19716,\n",
|
||||
" -0.037454,\n",
|
||||
" -0.3336,\n",
|
||||
" 0.31399,\n",
|
||||
" 0.36488,\n",
|
||||
" 0.71263,\n",
|
||||
" 0.1307,\n",
|
||||
" -0.24654,\n",
|
||||
" -0.52445,\n",
|
||||
" -0.036091,\n",
|
||||
" 0.55068,\n",
|
||||
" 0.10017,\n",
|
||||
" 0.48095,\n",
|
||||
" 0.71104,\n",
|
||||
" -0.053462,\n",
|
||||
" 0.22325,\n",
|
||||
" 0.30917,\n",
|
||||
" -0.39926,\n",
|
||||
" 0.036634,\n",
|
||||
" -0.35431,\n",
|
||||
" -0.42795,\n",
|
||||
" 0.46444,\n",
|
||||
" 0.25586,\n",
|
||||
" 0.68257,\n",
|
||||
" -0.20821,\n",
|
||||
" 0.38433,\n",
|
||||
" 0.055773,\n",
|
||||
" -0.2539,\n",
|
||||
" -0.20804,\n",
|
||||
" 0.52522,\n",
|
||||
" -0.11399,\n",
|
||||
" -0.3253,\n",
|
||||
" -0.44104,\n",
|
||||
" 0.17528,\n",
|
||||
" 0.62255,\n",
|
||||
" 0.50237,\n",
|
||||
" -0.7607,\n",
|
||||
" -0.071786,\n",
|
||||
" 0.0080131,\n",
|
||||
" -0.13286,\n",
|
||||
" 0.50097,\n",
|
||||
" 0.18824,\n",
|
||||
" -0.54722,\n",
|
||||
" -0.42664,\n",
|
||||
" 0.4292,\n",
|
||||
" 0.14877,\n",
|
||||
" -0.0072514,\n",
|
||||
" -0.16484,\n",
|
||||
" -0.059798,\n",
|
||||
" 0.9895,\n",
|
||||
" -0.61738,\n",
|
||||
" 0.054169,\n",
|
||||
" 0.48424,\n",
|
||||
" -0.35084,\n",
|
||||
" -0.27053,\n",
|
||||
" 0.37829,\n",
|
||||
" 0.11503,\n",
|
||||
" -0.39613,\n",
|
||||
" 0.24266,\n",
|
||||
" 0.39147,\n",
|
||||
" -0.075256,\n",
|
||||
" 0.65093,\n",
|
||||
" -0.20822,\n",
|
||||
" -0.17456,\n",
|
||||
" 0.53571,\n",
|
||||
" -0.16537,\n",
|
||||
" 0.13582,\n",
|
||||
" -0.56016,\n",
|
||||
" 0.016964,\n",
|
||||
" 0.1277,\n",
|
||||
" 0.94071,\n",
|
||||
" -0.22608,\n",
|
||||
" -0.021106,\n",
|
||||
" ],\n",
|
||||
" [\n",
|
||||
" -0.99544,\n",
|
||||
" -2.3651,\n",
|
||||
" -0.24332,\n",
|
||||
" -1.0321,\n",
|
||||
" 0.42052,\n",
|
||||
" -1.1817,\n",
|
||||
" -0.16451,\n",
|
||||
" -1.683,\n",
|
||||
" 0.49673,\n",
|
||||
" -0.27258,\n",
|
||||
" -0.025397,\n",
|
||||
" 0.34188,\n",
|
||||
" 1.5523,\n",
|
||||
" 1.3532,\n",
|
||||
" 0.33297,\n",
|
||||
" -0.0056677,\n",
|
||||
" -0.76525,\n",
|
||||
" 0.49587,\n",
|
||||
" 1.2211,\n",
|
||||
" 0.83394,\n",
|
||||
" -0.20031,\n",
|
||||
" -0.59657,\n",
|
||||
" 0.38485,\n",
|
||||
" -0.23487,\n",
|
||||
" -1.0725,\n",
|
||||
" 0.95856,\n",
|
||||
" 0.16161,\n",
|
||||
" -1.2496,\n",
|
||||
" 1.6751,\n",
|
||||
" 0.73899,\n",
|
||||
" 0.051347,\n",
|
||||
" -0.42702,\n",
|
||||
" 0.16257,\n",
|
||||
" -0.16772,\n",
|
||||
" 0.40146,\n",
|
||||
" 0.29837,\n",
|
||||
" 0.96204,\n",
|
||||
" -0.36232,\n",
|
||||
" -0.47848,\n",
|
||||
" 0.78278,\n",
|
||||
" 0.14834,\n",
|
||||
" 1.3407,\n",
|
||||
" 0.47834,\n",
|
||||
" -0.39083,\n",
|
||||
" -1.037,\n",
|
||||
" -0.24643,\n",
|
||||
" -0.75841,\n",
|
||||
" 0.7669,\n",
|
||||
" -0.37363,\n",
|
||||
" 0.52741,\n",
|
||||
" 0.018563,\n",
|
||||
" -0.51301,\n",
|
||||
" 0.97674,\n",
|
||||
" 0.55232,\n",
|
||||
" 1.1584,\n",
|
||||
" 0.73715,\n",
|
||||
" 1.3055,\n",
|
||||
" -0.44743,\n",
|
||||
" -0.15961,\n",
|
||||
" 0.85006,\n",
|
||||
" -0.34092,\n",
|
||||
" -0.67667,\n",
|
||||
" 0.2317,\n",
|
||||
" 1.5582,\n",
|
||||
" 1.2308,\n",
|
||||
" -0.62213,\n",
|
||||
" -0.032801,\n",
|
||||
" 0.1206,\n",
|
||||
" -0.25899,\n",
|
||||
" -0.02756,\n",
|
||||
" -0.52814,\n",
|
||||
" -0.93523,\n",
|
||||
" 0.58434,\n",
|
||||
" -0.24799,\n",
|
||||
" 0.37692,\n",
|
||||
" 0.86527,\n",
|
||||
" 0.069626,\n",
|
||||
" 1.3096,\n",
|
||||
" 0.29975,\n",
|
||||
" -1.3651,\n",
|
||||
" -0.32048,\n",
|
||||
" -0.13741,\n",
|
||||
" 0.33329,\n",
|
||||
" -1.9113,\n",
|
||||
" -0.60222,\n",
|
||||
" -0.23921,\n",
|
||||
" 0.12664,\n",
|
||||
" -0.47961,\n",
|
||||
" -0.89531,\n",
|
||||
" 0.62054,\n",
|
||||
" 0.40869,\n",
|
||||
" -0.08503,\n",
|
||||
" 0.6413,\n",
|
||||
" -0.84044,\n",
|
||||
" -0.74325,\n",
|
||||
" -0.19426,\n",
|
||||
" 0.098722,\n",
|
||||
" 0.32648,\n",
|
||||
" -0.67621,\n",
|
||||
" -0.62692,\n",
|
||||
" ],\n",
|
||||
"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@@ -1369,7 +1086,7 @@
|
||||
"source": [
|
||||
"### Compute Recall\n",
|
||||
"\n",
|
||||
"Use deployed brute force Index as the ground truth to calculate the recall of ANN Index:"
|
||||
"Use the deployed brute force Index as the ground truth to calculate the recall of ANN Index. Note that you can run multiple queries in a single match call."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1402,18 +1119,20 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Calculate recall by determining how many neighbors were correctly retrieved as compared to the brute-force option.\n",
|
||||
"correct_neighbors = 0\n",
|
||||
"recalled_neighbors = 0\n",
|
||||
"for tree_ah_neighbors, brute_force_neighbors in zip(\n",
|
||||
" tree_ah_response_test, brute_force_response_test\n",
|
||||
"):\n",
|
||||
" tree_ah_neighbor_ids = [neighbor.id for neighbor in tree_ah_neighbors]\n",
|
||||
" brute_force_neighbor_ids = [neighbor.id for neighbor in brute_force_neighbors]\n",
|
||||
"\n",
|
||||
" correct_neighbors += len(\n",
|
||||
" recalled_neighbors += len(\n",
|
||||
" set(tree_ah_neighbor_ids).intersection(brute_force_neighbor_ids)\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"recall = correct_neighbors / (len(test) * NUM_NEIGHBOURS)\n",
|
||||
"recall = recalled_neighbors / len(\n",
|
||||
" [neighbor for neighbors in brute_force_response_test for neighbor in neighbors]\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(\"Recall: {}\".format(recall))"
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user