mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Add vertex_ai_feature_store_iam_policy notebook. (#2913)
* Add vertex_ai_feature_store_iam_policy notebook. * update CODEOWNERS * Fix formatting of vertex_ai_feature_store_iam_policy.ipynb
This commit is contained in:
ethan-gordon
@@ -27,6 +27,7 @@
|
||||
/training/xgboost_data_parallel_training_on_cpu_using_dask.ipynb @halio-g
|
||||
/feature_store/gapic-feature-store.ipynb @diemtvu
|
||||
/feature_store/vertex_ai_feature_store_feature_view_service_agents.ipynb @ethan-gordon
|
||||
/feature_store/vertex_ai_feature_store_iam_policy.ipynb @ethan-gordon
|
||||
/managed_notebooks @GoogleCloudPlatform/notebooks-team
|
||||
/pipelines/google_cloud_pipeline_components_bqml_text.ipynb @inardini
|
||||
/pipelines/google_cloud_pipelines_dataproc_tabular @inardini
|
||||
@@ -53,4 +54,4 @@
|
||||
/model_evaluation/model_based_llm_evaluation/autosxs_check_alignment_against_human_preference_data.ipynb @sipingh
|
||||
/ray_on_vertex_ai/get_started_with_pytorch_rov.ipynb @inardini
|
||||
/training/tpuv5e_gemma_peft_finetuning_and_serving.ipynb @brianchunkang
|
||||
/training/tpuv5e_llama2_pytorch_finetuning_and_serving.ipynb @brianchunkang @chiefkarlin
|
||||
/training/tpuv5e_llama2_pytorch_finetuning_and_serving.ipynb @brianchunkang @chiefkarlin
|
||||
|
||||
@@ -0,0 +1,708 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2024 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Configure IAM Policy in Vertex AI Feature Store\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/feature_store/vertex_ai_feature_store_iam_policy.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fblob%2Fmain%2Fnotebooks%2Fofficial%2Ffeature_store%2Fvertex_ai_feature_store_iam_policy.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td> \n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/feature_store/vertex_ai_feature_store_iam_policy.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/feature_store/vertex_ai_feature_store_iam_policy.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This tutorial demonstrates how to configure an IAM Policy to control access to resources and data stored within Vertex AI Feature Store.\n",
|
||||
"\n",
|
||||
"Learn more about [Vertex AI Feature Store](https://cloud.google.com/vertex-ai/docs/featurestore/overview).\n",
|
||||
"\n",
|
||||
"The steps performed include:\n",
|
||||
"- Create a BigQuery dataset and set up a `FeatureView` to run periodic sync jobs.\n",
|
||||
"- Provision an online store instance and set up online serving.\n",
|
||||
"- Configure an IAM Policy to manage access controls."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "61RBz8LLbxCR"
|
||||
},
|
||||
"source": [
|
||||
"## Getting Started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "No17Cw5hgx12"
|
||||
},
|
||||
"source": [
|
||||
"### Install Vertex AI SDK and other required packages\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "tFy3H3aPgx12"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! pip3 install --upgrade --user --quiet google-cloud-aiplatform google-cloud-bigquery"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R5Xep4W9lq-Z"
|
||||
},
|
||||
"source": [
|
||||
"### Restart runtime (Colab only)\n",
|
||||
"\n",
|
||||
"To use the newly installed packages, you must restart the runtime on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "XRvKdaPDTznN"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" import IPython\n",
|
||||
"\n",
|
||||
" app = IPython.Application.instance()\n",
|
||||
" app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "SbmM4z7FOBpM"
|
||||
},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-warning\">\n",
|
||||
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
|
||||
"</div>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dmWOrTJ3gx13"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"Authenticate your environment on Google Colab.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NyKGtVQjgx13"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DF4l8DTdWgPY"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information and initialize Vertex AI SDK\n",
|
||||
"\n",
|
||||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Nqwi-5ufWp_B"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
|
||||
"LOCATION = \"us-central1\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"import vertexai\n",
|
||||
"\n",
|
||||
"vertexai.init(project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "EdvJRUWRNGHE"
|
||||
},
|
||||
"source": [
|
||||
"## Configure IAM Policy in Vertex AI Feature Store"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "GKnDEUulaLLy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"LOCATION_PARENT = \"projects/\" + PROJECT_ID + \"/locations/\" + LOCATION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6vbsjb91aJ1V"
|
||||
},
|
||||
"source": [
|
||||
"Configure the necessary service clients:\n",
|
||||
"- [FeaturestoreServiceClient](https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform_v1.services.featurestore_service.FeaturestoreServiceClient)\n",
|
||||
"- [FeatureOnlineStoreAdminServiceClient](https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform_v1beta1.services.feature_online_store_admin_service.FeatureOnlineStoreAdminServiceClient)\n",
|
||||
"- [FeatureRegistryServiceClient](https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform_v1beta1.services.feature_registry_service.FeatureRegistryServiceClient)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "zL8aOMAtaEUg"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud.aiplatform_v1beta1 import (\n",
|
||||
" FeatureOnlineStoreAdminServiceClient, FeatureOnlineStoreServiceClient,\n",
|
||||
" FeatureRegistryServiceClient, FeaturestoreServiceClient)\n",
|
||||
"from google.cloud.aiplatform_v1beta1.types import \\\n",
|
||||
" feature_online_store_service as feature_online_store_service_pb2\n",
|
||||
"from google.iam.v1 import iam_policy_pb2, policy_pb2\n",
|
||||
"\n",
|
||||
"ENDPOINT = LOCATION + \"-aiplatform.googleapis.com\"\n",
|
||||
"featurestore_service_client = FeaturestoreServiceClient(\n",
|
||||
" client_options={\"api_endpoint\": ENDPOINT}\n",
|
||||
")\n",
|
||||
"feature_online_store_admin_service_client = FeatureOnlineStoreAdminServiceClient(\n",
|
||||
" client_options={\"api_endpoint\": ENDPOINT}\n",
|
||||
")\n",
|
||||
"feature_registry_service_client = FeatureRegistryServiceClient(\n",
|
||||
" client_options={\"api_endpoint\": ENDPOINT}\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "NkxyUncIaCca"
|
||||
},
|
||||
"source": [
|
||||
"Generate sample data for this colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "XWAHq1fcZKsJ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"DATASET_ID = \"test_data\"+\"_\"+LOCATION.replace('-', '_') # @param {type:\"string\"}\n",
|
||||
"TABLE_ID = \"tableA\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"!bq mk --dataset_id={DATASET_ID} --location={LOCATION}\n",
|
||||
"!bq query --nouse_legacy_sql \\\n",
|
||||
"\"CREATE TABLE {DATASET_ID}.{TABLE_ID} AS (\" \\\n",
|
||||
"\"SELECT * FROM UNNEST(ARRAY<STRUCT<entity_id STRING, feature_timestamp TIMESTAMP, feature1 INT64, feature2 INT64>>[\" \\\n",
|
||||
"\"('test', TIMESTAMP('2024-02-26 08:00:00 UTC'), 10, 20),\" \\\n",
|
||||
"\"('test', TIMESTAMP('2024-02-27 08:00:00 UTC'), 30, 40),\" \\\n",
|
||||
"\"('test', TIMESTAMP('2024-02-28 08:00:00 UTC'), 50, 60)]))\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZCS7_Zr4ZL6y"
|
||||
},
|
||||
"source": [
|
||||
"### Create `FeatureOnlineStore` and `FeatureView` resources"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3KH56EmWZJnp"
|
||||
},
|
||||
"source": [
|
||||
"Provision a `FeatureOnlineStore` instance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "jrRqB3VpZF3C"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"FEATURE_ONLINE_STORE_ID = \"test_feature_online_store\"\n",
|
||||
"\n",
|
||||
"feature_online_store_admin_service_client.create_feature_online_store(\n",
|
||||
" {\n",
|
||||
" \"parent\": LOCATION_PARENT,\n",
|
||||
" \"feature_online_store_id\": FEATURE_ONLINE_STORE_ID,\n",
|
||||
" \"feature_online_store\": {\n",
|
||||
" \"bigtable\": {\n",
|
||||
" \"auto_scaling\": {\n",
|
||||
" \"min_node_count\": 1,\n",
|
||||
" \"max_node_count\": 3,\n",
|
||||
" \"cpu_utilization_target\": 50,\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" }\n",
|
||||
").result()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QK6Q3tNHZFa8"
|
||||
},
|
||||
"source": [
|
||||
"Configure periodic data synchronization from the offline store in BigQuery to the `FeatureOnlineStore` instance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "kKl7vb8fZDyu"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"FEATURE_VIEW_ID = \"test_feature_view\"\n",
|
||||
"FEATURE_ONLINE_STORE_RESOURCE_ID = (\n",
|
||||
" LOCATION_PARENT + \"/featureOnlineStores/\" + FEATURE_ONLINE_STORE_ID\n",
|
||||
")\n",
|
||||
"FEATURE_VIEW_RESOURCE_ID = (\n",
|
||||
" FEATURE_ONLINE_STORE_RESOURCE_ID + \"/featureViews/\" + FEATURE_VIEW_ID\n",
|
||||
")\n",
|
||||
"create_feature_view_result = (\n",
|
||||
" feature_online_store_admin_service_client.create_feature_view(\n",
|
||||
" parent=FEATURE_ONLINE_STORE_RESOURCE_ID,\n",
|
||||
" feature_view_id=FEATURE_VIEW_ID,\n",
|
||||
" feature_view={\n",
|
||||
" \"big_query_source\": {\n",
|
||||
" \"uri\": f\"bq://{PROJECT_ID}.{DATASET_ID}.{TABLE_ID}\",\n",
|
||||
" \"entity_id_columns\": [\"entity_id\"],\n",
|
||||
" },\n",
|
||||
" \"sync_config\": {\"cron\": \"0 12 * * *\"},\n",
|
||||
" },\n",
|
||||
" ).result()\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BU1_rOgVZpNj"
|
||||
},
|
||||
"source": [
|
||||
"## Configure an IAM Policy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "79enQT97ZBBK"
|
||||
},
|
||||
"source": [
|
||||
"Configure IAM Policy for a feature online store."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ST1M5no-ZAgV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# A specific user email or a group email can be used to add members to the IAM Policy.\n",
|
||||
"EXAMPLE_RESOURCE_VIEWER_EMAIL = (\n",
|
||||
" \"google-cloud-eng-fte@google.com\" # @param {type:\"string\"}\n",
|
||||
")\n",
|
||||
"EXAMPLE_DATA_VIEWER_EMAIL = \"python-sample-owners@google.com\" # @param {type:\"string\"}\n",
|
||||
"# A Service Account can also be added to the IAM Policy.\n",
|
||||
"EXAMPLE_ADMIN_SERVICE_ACCOUNT = \"samples@python-docs-samples-tests.iam.gserviceaccount.com\" # @param {type:\"string\"}\n",
|
||||
"feature_online_store_admin_service_client.set_iam_policy(\n",
|
||||
" request=iam_policy_pb2.SetIamPolicyRequest(\n",
|
||||
" resource=FEATURE_ONLINE_STORE_RESOURCE_ID,\n",
|
||||
" policy=policy_pb2.Policy(\n",
|
||||
" bindings=[\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.featurestoreResourceViewer\",\n",
|
||||
" members=[f\"group:{EXAMPLE_RESOURCE_VIEWER_EMAIL}\"],\n",
|
||||
" ),\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.featurestoreDataViewer\",\n",
|
||||
" members=[f\"group:{EXAMPLE_DATA_VIEWER_EMAIL}\"],\n",
|
||||
" ),\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.admin\",\n",
|
||||
" members=[f\"serviceAccount:{EXAMPLE_ADMIN_SERVICE_ACCOUNT}\"],\n",
|
||||
" ),\n",
|
||||
" ]\n",
|
||||
" ),\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3ct-qQeaY-HQ"
|
||||
},
|
||||
"source": [
|
||||
"Confirm the updated IAM Policy of the feature online store."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nPkYu3D_Y9q1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"feature_online_store_admin_service_client.get_iam_policy(\n",
|
||||
" request=iam_policy_pb2.GetIamPolicyRequest(\n",
|
||||
" resource=FEATURE_ONLINE_STORE_RESOURCE_ID\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gOldDbeVY6SK"
|
||||
},
|
||||
"source": [
|
||||
"Configure IAM Policy for a feature view."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "IuMXMJQjY32o"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# A specific user email or a group email can be used to add members to the IAM Policy.\n",
|
||||
"EXAMPLE_RESOURCE_VIEWER_EMAIL = (\n",
|
||||
" \"google-cloud-eng-fte@google.com\" # @param {type:\"string\"}\n",
|
||||
")\n",
|
||||
"EXAMPLE_DATA_VIEWER_EMAIL = \"python-sample-owners@google.com\" # @param {type:\"string\"}\n",
|
||||
"# A Service Account can also be added to the IAM Policy.\n",
|
||||
"EXAMPLE_ADMIN_SERVICE_ACCOUNT = \"samples@python-docs-samples-tests.iam.gserviceaccount.com\" # @param {type:\"string\"}\n",
|
||||
"feature_online_store_admin_service_client.set_iam_policy(\n",
|
||||
" request=iam_policy_pb2.SetIamPolicyRequest(\n",
|
||||
" resource=FEATURE_VIEW_RESOURCE_ID,\n",
|
||||
" policy=policy_pb2.Policy(\n",
|
||||
" bindings=[\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.featurestoreResourceViewer\",\n",
|
||||
" members=[f\"group:{EXAMPLE_RESOURCE_VIEWER_EMAIL}\"],\n",
|
||||
" ),\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.featurestoreDataViewer\",\n",
|
||||
" members=[f\"group:{EXAMPLE_DATA_VIEWER_EMAIL}\"],\n",
|
||||
" ),\n",
|
||||
" policy_pb2.Binding(\n",
|
||||
" role=\"roles/aiplatform.admin\",\n",
|
||||
" members=[f\"serviceAccount:{EXAMPLE_ADMIN_SERVICE_ACCOUNT}\"],\n",
|
||||
" ),\n",
|
||||
" ]\n",
|
||||
" ),\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "IdDAiz6TEWMI"
|
||||
},
|
||||
"source": [
|
||||
"Wait some time for the newly-added IAM policy binding to take effect.\n",
|
||||
"\n",
|
||||
"* Note: The policy change typically takes 2 minutes, so this delay may be reduced. See IAM docs for more details."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "yGeqDWFgEX3s"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Wait 10 minutes for the newly updated IAM policy binding to become effective.\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"sleep(600)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RVhAFV0MY3Ko"
|
||||
},
|
||||
"source": [
|
||||
"Confirm the updated IAM Policy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "awfeS9z5Y0sy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"feature_online_store_admin_service_client.get_iam_policy(\n",
|
||||
" request=iam_policy_pb2.GetIamPolicyRequest(resource=FEATURE_VIEW_RESOURCE_ID)\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "t6A7fWzlY0Pc"
|
||||
},
|
||||
"source": [
|
||||
"Run on-demand batch sync."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "YLJUPFPmYsa3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sync_response = feature_online_store_admin_service_client.sync_feature_view(\n",
|
||||
" feature_view=f\"projects/{PROJECT_ID}/locations/{LOCATION}/featureOnlineStores/{FEATURE_ONLINE_STORE_ID}/featureViews/{FEATURE_VIEW_ID}\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Wait for sync completion\n",
|
||||
"while True:\n",
|
||||
" feature_view_sync = feature_online_store_admin_service_client.get_feature_view_sync(\n",
|
||||
" name=sync_response.feature_view_sync\n",
|
||||
" )\n",
|
||||
" if feature_view_sync.run_time.end_time.seconds > 0:\n",
|
||||
" status = \"Succeed\" if feature_view_sync.final_status.code == 0 else \"Failed\"\n",
|
||||
" print(f\"Sync {status} for {feature_view_sync.name}.\")\n",
|
||||
" break\n",
|
||||
" else:\n",
|
||||
" print(\"Sync ongoing, waiting for 30 seconds.\")\n",
|
||||
" sleep(30)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-k4qNUFcrYzC"
|
||||
},
|
||||
"source": [
|
||||
"Confirm the status of batch sync."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6mnM08ZLrT8f"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"feature_online_store_admin_service_client.get_feature_view_sync(\n",
|
||||
" name=sync_response.feature_view_sync\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "z-PuxGYDYph2"
|
||||
},
|
||||
"source": [
|
||||
"### Start online serving"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uyZrlw9pMGze"
|
||||
},
|
||||
"source": [
|
||||
"After the data sync is complete, use the `FetchFeatureValues` API to retrieve the data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PmOPvky1Ylbv"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"data_client = FeatureOnlineStoreServiceClient(client_options={\"api_endpoint\": ENDPOINT})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27e6iqlTMNDv"
|
||||
},
|
||||
"source": [
|
||||
"Read the synced data from feature online store."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1m4OpNPsMMqP"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"data_client.fetch_feature_values(\n",
|
||||
" request=feature_online_store_service_pb2.FetchFeatureValuesRequest(\n",
|
||||
" feature_view=FEATURE_VIEW_RESOURCE_ID,\n",
|
||||
" data_key=feature_online_store_service_pb2.FeatureViewDataKey(key=\"test\"),\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2a4e033321ad"
|
||||
},
|
||||
"source": [
|
||||
"## Cleaning up\n",
|
||||
"\n",
|
||||
"Cleaning up\n",
|
||||
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
||||
"\n",
|
||||
"Otherwise, you can delete the cluster you created in this tutorial."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "aqLqumA9YeBh"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Delete feature view\n",
|
||||
"feature_online_store_admin_service_client.delete_feature_view(\n",
|
||||
" name=FEATURE_VIEW_RESOURCE_ID\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Delete online store\n",
|
||||
"feature_online_store_admin_service_client.delete_feature_online_store(\n",
|
||||
" name=FEATURE_ONLINE_STORE_RESOURCE_ID\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Delete test data\n",
|
||||
"!bq rm -f {DATASET_ID}.{TABLE_ID}"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "vertex_ai_feature_store_iam_policy.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Reference in New Issue
Block a user