mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
refactor, chore (Egen) : Fixes headings, removes boilerplate and other corrections from template (#3044)
* refactor: removes boilerplate code, and fixes heading * refactor: change region to location * Removes IS_TESTING variable * cleanup of changes * chore: lint test done * chore: aligns the icons to the center * chore:verbiage changes and end to end code execution * chore: reformatted by lint test * chore: edits future tenses and reformatted by lint test * chorE: address review comments and change import statement based on lint test * fix: error rectification, remove vague testing variables * fix: rectifies testing induced error in notebook * chore: lint
This commit is contained in:
Kaushik Koilada
+104
-193
@@ -29,27 +29,32 @@
|
||||
"id": "awoLZ5dc5bcG"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI Feature Store Based LLM Grounding Tutorial\n",
|
||||
"# Vertex AI Feature Store Based LLM Grounding tutorial\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
"\n",
|
||||
" <td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/vertex_ai_feature_store_based_llm_grounding_tutorial.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"><br> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/vertex_ai_feature_store_based_llm_grounding_tutorial.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Ffeature_store%2Fvertex_ai_feature_store_based_llm_grounding_tutorial.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" </td> \n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/feature_store/vertex_ai_feature_store_based_llm_grounding_tutorial.ipynb.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br>\n",
|
||||
" Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" </td> \n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/vertex_ai_feature_store_based_llm_grounding_tutorial.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br>\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
@@ -76,7 +81,7 @@
|
||||
"\n",
|
||||
"In this tutorial, you learn how to create and use an online feature store instance to host and serve data in BigQuery with Vertex AI Feature Store in an end to end workflow of features serving and vector retrieval user journey.\n",
|
||||
"\n",
|
||||
"This tutorial uses the following Google Cloud ML services and resources:\n",
|
||||
"This tutorial uses the following Google Vertex AI services and resources:\n",
|
||||
"\n",
|
||||
"- Vertex AI Feature Store\n",
|
||||
"\n",
|
||||
@@ -128,15 +133,22 @@
|
||||
"to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d1ea81ac77f0"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yTJiDCrYsOmT"
|
||||
},
|
||||
"source": [
|
||||
"## Installation\n",
|
||||
"\n",
|
||||
"Install the following packages required to run this notebook."
|
||||
"### Install Vertex AI SDK for Python and other required packages"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -161,7 +173,9 @@
|
||||
"id": "np60_uuCs7X5"
|
||||
},
|
||||
"source": [
|
||||
"### Colab only: Uncomment the following cell to restart the kernel."
|
||||
"### Restart runtime (Colab only)\n",
|
||||
"\n",
|
||||
"To use the newly installed packages, you must restart the runtime on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -172,46 +186,62 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# # Automatically restart the kernel after installation so that your environment can access the new packages\n",
|
||||
"# import IPython\n",
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"# app = IPython.Application.instance()\n",
|
||||
"# app.kernel.do_shutdown(True)"
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" import IPython\n",
|
||||
"\n",
|
||||
" app = IPython.Application.instance()\n",
|
||||
" app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mdqw6ADTtJRI"
|
||||
"id": "01e1cc61b578"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"### Set up your Google Cloud project\n",
|
||||
"\n",
|
||||
"**The following steps are required, regardless of your notebook environment.**\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"3. [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"4. If you're running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
|
||||
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "bFANidV0tPbo"
|
||||
"id": "ff666ce4051c"
|
||||
},
|
||||
"source": [
|
||||
"#### Set your project ID\n",
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"**If you don't know your project ID**, try the following:\n",
|
||||
"* Run `gcloud config list`.\n",
|
||||
"* Run `gcloud projects list`.\n",
|
||||
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
|
||||
"Authenticate your environment on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "cc7251520a07"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "60763ee24ce0"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information and initialize Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -223,108 +253,14 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"LOCATION = \"us-central1\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# Set the project id\n",
|
||||
"! gcloud config set project {PROJECT_ID}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "lfY8yWnbtZ0K"
|
||||
},
|
||||
"source": [
|
||||
"#### Region\n",
|
||||
"\n",
|
||||
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations). Note that the new Feature Store capability showed in the colab is currently only available in the following regions:\n",
|
||||
"* `us-central1`\n",
|
||||
"* `us-east1`\n",
|
||||
"* `us-west1`\n",
|
||||
"* `europe-west4`\n",
|
||||
"* `asia-southeast1`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "G6iMMALZthFM"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"REGION = \"us-central1\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Ni5jx6RGtzG3"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your Google Cloud account\n",
|
||||
"from google.cloud import aiplatform\n",
|
||||
"\n",
|
||||
"Depending on your Jupyter environment, you might have to manually authenticate. Follow these instructions:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "UzsMphY2t4-v"
|
||||
},
|
||||
"source": [
|
||||
"**1. Vertex AI Workbench**\n",
|
||||
"* Do nothing as you are already authenticated."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R_OnJm_Yt8bw"
|
||||
},
|
||||
"source": [
|
||||
"**2. Local JupyterLab instance, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "x4ybIfzhuAOc"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ! gcloud auth login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gmnRqX6BuBnx"
|
||||
},
|
||||
"source": [
|
||||
"**3. Colab, uncomment and run:**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "uN9JoC1buE9P"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# from google.colab import auth\n",
|
||||
"# auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "YNAvMVJjuH5b"
|
||||
},
|
||||
"source": [
|
||||
"**4. Service account or other**\n",
|
||||
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
|
||||
"aiplatform.init(project=PROJECT_ID, location=LOCATION)\n",
|
||||
"\n",
|
||||
"API_ENDPOINT = f\"{LOCATION}-aiplatform.googleapis.com\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -346,7 +282,7 @@
|
||||
"source": [
|
||||
"import uuid\n",
|
||||
"\n",
|
||||
"from google.cloud import aiplatform, bigquery\n",
|
||||
"from google.cloud import bigquery\n",
|
||||
"from google.cloud.aiplatform_v1 import (FeatureOnlineStoreAdminServiceClient,\n",
|
||||
" FeatureOnlineStoreServiceClient)\n",
|
||||
"from google.cloud.aiplatform_v1.types import NearestNeighborQuery\n",
|
||||
@@ -356,32 +292,7 @@
|
||||
" feature_online_store_admin_service as \\\n",
|
||||
" feature_online_store_admin_service_pb2\n",
|
||||
"from google.cloud.aiplatform_v1.types import \\\n",
|
||||
" feature_online_store_service as feature_online_store_service_pb2\n",
|
||||
"from google.cloud.aiplatform_v1.types import feature_view as feature_view_pb2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6ooJNSOvu6Q5"
|
||||
},
|
||||
"source": [
|
||||
"### Initialize Vertex AI SDK for Python\n",
|
||||
"\n",
|
||||
"Initialize the Vertex AI SDK for Python for your project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "mLTm3pquu9ar"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION)\n",
|
||||
"\n",
|
||||
"API_ENDPOINT = f\"{REGION}-aiplatform.googleapis.com\""
|
||||
" feature_online_store_service as feature_online_store_service_pb2"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -425,7 +336,7 @@
|
||||
"id": "_OV9dADJb63o"
|
||||
},
|
||||
"source": [
|
||||
"**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket.\n"
|
||||
"**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -436,7 +347,7 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! gsutil mb -l {REGION} -p {PROJECT_ID} {GCS_BUCKET}"
|
||||
"! gsutil mb -l {LOCATION} -p {PROJECT_ID} {GCS_BUCKET}"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -504,7 +415,7 @@
|
||||
"\n",
|
||||
"BQ_DATASET_ID = \"fs_grounding\" # @param {type:\"string\"}\n",
|
||||
"dataset = bigquery.Dataset(f\"{PROJECT_ID}.{BQ_DATASET_ID}\")\n",
|
||||
"dataset.location = REGION\n",
|
||||
"dataset.location = LOCATION\n",
|
||||
"dataset = bq_client.create_dataset(\n",
|
||||
" dataset, exists_ok=True, timeout=30\n",
|
||||
") # Make an API request.\n",
|
||||
@@ -538,7 +449,7 @@
|
||||
"\n",
|
||||
"PARAMS = {\n",
|
||||
" \"project\": PROJECT_ID,\n",
|
||||
" \"location\": REGION,\n",
|
||||
" \"location\": LOCATION,\n",
|
||||
" \"bigquery_bp_input_uri\": BIGQUERY_BP_INPUT_URI,\n",
|
||||
" \"bigquery_bp_output_uri\": BIGQUERY_BP_OUTPUT_URI,\n",
|
||||
" \"input_text_gcs_dir\": INPUT_TEXT_GCS_DIR,\n",
|
||||
@@ -582,7 +493,7 @@
|
||||
" parameters=PARAMS,\n",
|
||||
" project=PROJECT_ID,\n",
|
||||
" pipeline_root=f\"{GCS_BUCKET}/fs_based/pipeline_root\",\n",
|
||||
" location=REGION,\n",
|
||||
" location=LOCATION,\n",
|
||||
")\n",
|
||||
"job.wait()"
|
||||
]
|
||||
@@ -611,7 +522,7 @@
|
||||
"\n",
|
||||
" Args:\n",
|
||||
" bigquery_bp_input_uri: The URI to a bigquery table as the input for the\n",
|
||||
" batch prediction component. The chunking component will populate data to\n",
|
||||
" batch prediction component. The chunking component populates data to\n",
|
||||
" this uri first before batch prediction.\n",
|
||||
" bigquery_bp_output_uri: The URI to a bigquery table as the output for the\n",
|
||||
" batch prediction component.\n",
|
||||
@@ -711,7 +622,7 @@
|
||||
"\n",
|
||||
"create_store_lro = admin_client.create_feature_online_store(\n",
|
||||
" feature_online_store_admin_service_pb2.CreateFeatureOnlineStoreRequest(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}\",\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}\",\n",
|
||||
" feature_online_store_id=FEATURE_ONLINE_STORE_ID,\n",
|
||||
" feature_online_store=online_store_config,\n",
|
||||
" )\n",
|
||||
@@ -762,7 +673,7 @@
|
||||
"source": [
|
||||
"# Use get to verify the store is created.\n",
|
||||
"admin_client.get_feature_online_store(\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -785,7 +696,7 @@
|
||||
"source": [
|
||||
"# Use list to verify the store is created.\n",
|
||||
"admin_client.list_feature_online_stores(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}\"\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -802,7 +713,7 @@
|
||||
"* A data source (BigQuery table or view URI or `FeatureGroup/features`) synced to the `FeatureOnlineStore` instance for serving.\n",
|
||||
"* The [cron](https://en.wikipedia.org/wiki/Cron) schedule to run the sync pipeline.\n",
|
||||
"\n",
|
||||
"During feature view creation, a sync job will be scheduled, and either started immediately or following the cron schedule. In the sync job, data is exported, a index is built and deployed to GKE cluster."
|
||||
"During feature view creation, a sync job is scheduled, and either started immediately or following the cron schedule. In the sync job, data is exported, an index is built and deployed to GKE cluster."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -814,7 +725,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"FEATURE_VIEW_ID = \"fs_grounding_test_new\" # @param {type: \"string\"}\n",
|
||||
"# A schedule will be created based on cron setting.\n",
|
||||
"# A schedule is created based on cron setting.\n",
|
||||
"# If cron is unspecified, a sync job is started immediately.\n",
|
||||
"CRON_SCHEDULE = \"TZ=America/Los_Angeles 00 13 11 8 *\" # @param {type: \"string\"}"
|
||||
]
|
||||
@@ -857,6 +768,8 @@
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud.aiplatform_v1.types import feature_view as feature_view_pb2\n",
|
||||
"\n",
|
||||
"big_query_source = feature_view_pb2.FeatureView.BigQuerySource(\n",
|
||||
" uri=DATA_SOURCE, entity_id_columns=[\"vertex_generated_chunk_id\"]\n",
|
||||
")\n",
|
||||
@@ -875,7 +788,7 @@
|
||||
"\n",
|
||||
"create_view_lro = admin_client.create_feature_view(\n",
|
||||
" feature_online_store_admin_service_pb2.CreateFeatureViewRequest(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\",\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\",\n",
|
||||
" feature_view_id=FEATURE_VIEW_ID,\n",
|
||||
" feature_view=feature_view_pb2.FeatureView(\n",
|
||||
" big_query_source=big_query_source,\n",
|
||||
@@ -926,7 +839,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"admin_client.get_feature_view(\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -936,7 +849,7 @@
|
||||
"id": "sqpu4nHAO4pW"
|
||||
},
|
||||
"source": [
|
||||
"Verify that the FeatureView instance is created by listing all the feature views within the online store."
|
||||
"Verify that the `FeatureView` instance is created by listing all the feature views within the online store."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -948,7 +861,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"admin_client.list_feature_views(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -962,7 +875,7 @@
|
||||
"source": [
|
||||
"# Optional: Delete feature views to avoid exceeding the deployed index nodes quota.\n",
|
||||
"# views = admin_client.list_feature_views(\n",
|
||||
"# parent=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
"# parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
"# )\n",
|
||||
"# for view in views:\n",
|
||||
"# admin_client.delete_feature_view(name=view.name)"
|
||||
@@ -990,7 +903,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sync_response = admin_client.sync_feature_view(\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -1048,7 +961,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"admin_client.list_feature_view_syncs(\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
" parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
@@ -1075,7 +988,7 @@
|
||||
"source": [
|
||||
"# Verify online store creation.\n",
|
||||
"featore_online_store_instance = admin_client.get_feature_online_store(\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\"\n",
|
||||
")\n",
|
||||
"PUBLIC_ENDPOINT = (\n",
|
||||
" featore_online_store_instance.dedicated_serving_endpoint.public_endpoint_domain_name\n",
|
||||
@@ -1147,7 +1060,7 @@
|
||||
"# A vertex_generated_chunk_id for testing\n",
|
||||
"data_client.search_nearest_entities(\n",
|
||||
" request=feature_online_store_service_pb2.SearchNearestEntitiesRequest(\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" query=NearestNeighborQuery(\n",
|
||||
" entity_id=ENTITY_ID,\n",
|
||||
" neighbor_count=5,\n",
|
||||
@@ -1187,7 +1100,7 @@
|
||||
"source": [
|
||||
"data_client.search_nearest_entities(\n",
|
||||
" request=feature_online_store_service_pb2.SearchNearestEntitiesRequest(\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" query=NearestNeighborQuery(\n",
|
||||
" embedding=NearestNeighborQuery.Embedding(value=EMBEDDINGS),\n",
|
||||
" neighbor_count=10,\n",
|
||||
@@ -1216,7 +1129,7 @@
|
||||
"source": [
|
||||
"data_client.fetch_feature_values(\n",
|
||||
" request=feature_online_store_service_pb2.FetchFeatureValuesRequest(\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\",\n",
|
||||
" data_key=feature_online_store_service_pb2.FeatureViewDataKey(key=ENTITY_ID),\n",
|
||||
" )\n",
|
||||
")"
|
||||
@@ -1246,20 +1159,18 @@
|
||||
"source": [
|
||||
"# Delete Feature View\n",
|
||||
"admin_client.delete_feature_view(\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Delete Feature Online Store\n",
|
||||
"admin_client.delete_feature_online_store(\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{REGION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\",\n",
|
||||
" name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}\",\n",
|
||||
" force=True,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Delete Cloud Storage objects that were created\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"delete_bucket = False\n",
|
||||
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
|
||||
"delete_bucket = True\n",
|
||||
"if delete_bucket:\n",
|
||||
" ! gsutil -m rm -r $GCS_BUCKET"
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user