Compare commits

...
1 Commits
Author SHA1 Message Date
Andrew Ferlitsch 75dcab6c20 fix: reported branding issues 2022-10-12 19:48:23 +00:00
@@ -29,7 +29,7 @@
"id": "JAPoU8Sm5E6e"
},
"source": [
"# Using Vertex AI Feature Store with pandas DataFrame\n",
"# Using Vertex AI Feature Store with pandas dataFrame\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
@@ -62,7 +62,7 @@
"source": [
"## Overview\n",
"\n",
"This notebook introduces Pandas support for Feature Store using Vertex AI SDK. For pre-requisites and introduction on Vertex AI SDK and Feature Store native support, please go through this [Colab notebook](https://colab.sandbox.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/feature_store/sdk-feature-store.ipynb). "
"This notebook introduces pandas support for `Vertex AI Feature Store` using Vertex AI SDK. For pre-requisites and introduction on Vertex AI SDK and Feature Store native support, please go through this [Colab notebook](https://colab.sandbox.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/feature_store/sdk-feature-store.ipynb). "
]
},
{
@@ -73,13 +73,20 @@
"source": [
"### Objective\n",
"\n",
"In this notebook, you learn how to use `Vertex AI Feature Store` with pandas DataFrame.\n",
"In this notebook, you learn how to use `Vertex AI Feature Store` with pandas dataFrame.\n",
"\n",
"This tutorial uses the following Google Cloud ML services:\n",
"\n",
"- `Vertex AI Feature Store`\n",
"\n",
"The steps performed include:\n",
"\n",
"- Ingest Feature values from Pandas DataFrame into Feature Store's Entity types.\n",
"- Read Entity Feature values from Online Feature Store into Pandas DataFrame.\n",
"- Batch serve Feature values from your Feature Store into Pandas DataFrame.\n",
"- Create a Vertex AI Feature Store.\n",
"- Create Vertex AI Entity Type resources for the Vertex AI Feature Store.\n",
"- Create Vertex AI Feature resources for corresponding Vertex AI Entity Type resources.\n",
"- Ingest feature values from pandas dataFrame into Vertex AI Feature Store's Entity Type resources.\n",
"- Read Vertex AI Entity Types Feature values from online Feature Store into pandas dataFrame.\n",
"- Batch serve feature values from your Feature Store into pandas dataFrame.\n",
"\n",
"You also learn how Vertex AI Feature Store can be useful in the below scenarios:\n",
"\n",
@@ -146,12 +153,15 @@
"source": [
"import os\n",
"\n",
"# The Google Cloud Notebook product has specific requirements\n",
"IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n",
"# The Vertex AI Workbench Notebook product has specific requirements\n",
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\") and not os.getenv(\"VIRTUAL_ENV\")\n",
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
" \"/opt/deeplearning/metadata/env_version\"\n",
")\n",
"\n",
"# Google Cloud Notebook requires dependencies to be installed with '--user'\n",
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
"USER_FLAG = \"\"\n",
"if IS_GOOGLE_CLOUD_NOTEBOOK:\n",
"if IS_WORKBENCH_NOTEBOOK:\n",
" USER_FLAG = \"--user\"\n",
" \n",
"! pip install -U {USER_FLAG} --upgrade google-cloud-aiplatform \\\n",
@@ -307,7 +317,7 @@
"### Authenticate your Google Cloud account\n",
"\n",
"**If you are using Google Cloud Notebooks**, your environment is already\n",
"authenticated. Skip this step."
"authenticated. "
]
},
{
@@ -348,19 +358,19 @@
},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"\n",
"# If you are running this notebook in Colab, run this cell and follow the\n",
"# instructions to authenticate your GCP account. This provides access to your\n",
"# Cloud Storage bucket and lets you submit training jobs and prediction\n",
"# requests.\n",
"\n",
"# The Google Cloud Notebook product has specific requirements\n",
"IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n",
"import os\n",
"import sys\n",
"\n",
"# If on Google Cloud Notebooks, then don't execute this code\n",
"if not IS_GOOGLE_CLOUD_NOTEBOOK:\n",
"# If on Vertex AI Workbench, then don't execute this code\n",
"IS_COLAB = \"google.colab\" in sys.modules\n",
"if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
" \"DL_ANACONDA_HOME\"\n",
"):\n",
" if \"google.colab\" in sys.modules:\n",
" from google.colab import auth as google_auth\n",
"\n",
@@ -404,7 +414,7 @@
"id": "9UvxYyGUimKw"
},
"source": [
"## Create Feature Store Resources"
"## Create Vertex AI Feature Store resources"
]
},
{
@@ -413,9 +423,9 @@
"id": "buQBIv3ZL3A0"
},
"source": [
"### Create Feature Store\n",
"### Create a Vertex AI Feature Store\n",
"\n",
"The method to create a Feature Store returns a\n",
"The method to create a Vertex AI Feature Store returns a\n",
"[long-running operation](https://google.aip.dev/151) (LRO). An LRO starts an asynchronous job. LROs are returned for other API\n",
"methods too, such as updating or deleting a featurestore. Running the code cell creates a featurestore and prints the process logs."
]
@@ -440,9 +450,9 @@
"id": "EpmJq75zXjmT"
},
"source": [
"### Create Entity Types\n",
"### Create the Vertex AI Entity Type resources\n",
"\n",
"Entity types can be created within the Featurestore class. Below, you create the `Users` entity type and `Movies` entity type. Process logs are printed in the output for each cell."
"Vertex AI Entity Types can be created within a Vertex AI Feature Store resource. Below, you create the `Users` entity type and `Movies` entity type. Process logs are printed in the output for each cell."
]
},
{
@@ -479,8 +489,8 @@
"id": "FJW4q-0jO2Xf"
},
"source": [
"### Create Features\n",
"Features can be created within each entity type. Add defining features to the `Users` entity type and `Movies` entity type by using the following methods."
"### Create Vertex AI Feature resources for the Vertex AI Entity Type resources\n",
"Vertex AI Feature resources can be created within each Vertex AI Entity Type. Add defining features to the `Users` entity type and `Movies` entity type by using the following methods."
]
},
{
@@ -553,9 +563,9 @@
"id": "K3n5XdK8Xjmw"
},
"source": [
"## Ingest Feature Values into Entity Type from a Pandas DataFrame\n",
"## Ingest feature values into Entity Type from a pandas dataFrame\n",
"\n",
"You need to ingest feature values into your entity type containing the features, so you can later `read` (online) or `batch serve` (offline) the feature values from the entity type. In this step, you will learn how to ingest feature values from a Pandas DataFrame into an entity type. We can also import feature values from BigQuery or Google Cloud Storage.\n"
"You need to ingest feature values into your entity type containing the features, so you can later `read` (online) or `batch serve` (offline) the feature values from the entity type. In this step, you learn how to ingest feature values from a Pandas DataFrame into an entity type. You can also import feature values from BigQuery or Cloud Storage.\n"
]
},
{
@@ -604,7 +614,7 @@
"id": "Fd6Z0jfR5OW5"
},
"source": [
"#### Load Avro Files into Pandas DataFrames"
"#### Load Avro files into pandas dataFrames"
]
},
{
@@ -660,7 +670,7 @@
"id": "bgb0WGwX5OW6"
},
"source": [
"#### Ingest Feature Values into _Users_ Entity Type"
"#### Ingest feature values into *Users* Entity Type resource"
]
},
{
@@ -685,7 +695,7 @@
"id": "PCAdQ3cF5OW6"
},
"source": [
"#### Ingest Feature Values into _Movies_ Entity Type"
"#### Ingest Feature Values into *Movies* Entity Type resource"
]
},
{
@@ -710,9 +720,9 @@
"id": "pIYLZwao5OW6"
},
"source": [
"## Read/Online Serve Entity's Feature Values from Vertex AI Online Feature Store\n",
"## Online serving feature values from Vertex AI Feature Store\n",
"\n",
"Feature Store allows [online serving](https://cloud.google.com/vertex-ai/docs/featurestore/serving-online)\n",
"Vertex AI Feature Store allows [online serving](https://cloud.google.com/vertex-ai/docs/featurestore/serving-online)\n",
"which lets you read feature values for small batches of entities. It works well when you want to read values of selected features from an entity or multiple entities in an entity type."
]
},
@@ -751,9 +761,9 @@
"id": "AK2Glzkq5OW7"
},
"source": [
"## Batch Serve Feature Values from Vertex AI Feature Store\n",
"## Batch serve featurevalues from Vertex AI Feature Store\n",
"\n",
"Batch Serving is used to fetch a large batch of feature values for high-throughput, and is typically used for training a model or batch prediction. In this section, you learn how to prepare training examples by using the Feature Store's batch serve function."
"Batch serving is used to fetch a large batch of feature values for high-throughput, and is typically used for training a model or batch prediction. In this section, you learn how to prepare training examples by using the Feature Store's batch serve function."
]
},
{
@@ -794,7 +804,7 @@
"id": "T5DW1MFt5OW7"
},
"source": [
"#### Load CSV file into a Pandas DataFrame"
"#### Load CSV file into a pandas dataFrame"
]
},
{
@@ -815,7 +825,7 @@
"id": "LsgNNH8G5OW8"
},
"source": [
"#### Change the Dtype of `Timestamp` to `Datetime64`"
"#### Change the dtype of `Timestamp` to `Datetime64`"
]
},
{
@@ -837,7 +847,7 @@
"id": "ao1dC5Pc5OW8"
},
"source": [
"#### Batch Serve Feature Values from Movie Predictions Feature Store"
"#### Batch serve feature values from Movie Predictions Feature Store"
]
},
{
@@ -864,7 +874,7 @@
"id": "29gLNORP5OW8"
},
"source": [
"## Read the Updated Feature Values"
"## Read the updated Feature Value resources"
]
},
{
@@ -894,7 +904,7 @@
"id": "feTUJjqG5OW9"
},
"source": [
"#### Ingest updated Feature Values"
"#### Ingest updated Feature Value resources"
]
},
{
@@ -934,7 +944,7 @@
"id": "s47WCIvL5OW9"
},
"source": [
"#### Latest Feature Values\n",
"#### Read the latest Feature Values\n",
"Read from the Entity Type shows updated Feature values from the latest ingestion."
]
},
@@ -1030,7 +1040,7 @@
"id": "WXb4JUhu5OW-"
},
"source": [
"#### Ingest backfilled/corrected point-in-time data from dataframe"
"#### Ingest backfilled/corrected point-in-time data from dataFrame"
]
},
{