Compare commits

...
Author SHA1 Message Date
Andrew Ferlitsch 0ba603596b fix: nits 2022-06-22 16:07:12 +00:00
Andrew Ferlitsch 7ede61adb6 fix: nits 2022-06-22 16:06:40 +00:00
Andrew FerlitschandGitHub dcbfbb40f0 fix: aip reference 2022-06-22 08:23:29 -07:00
Andrew FerlitschandGitHub 1e2a006b51 Update sdk-feature-store.ipynb 2022-06-21 18:34:36 -07:00
Andrew FerlitschandGitHub d12a3efda9 Update sdk-feature-store.ipynb 2022-06-21 17:19:44 -07:00
Andrew FerlitschandGitHub c694bcbf6e Update sdk-feature-store.ipynb 2022-06-21 17:12:23 -07:00
Andrew Ferlitsch c0965e2be9 upgrade: notebook standard 2022-06-21 22:04:11 +00:00
Andrew Ferlitsch 0f590ea596 upgrade: notebook standard 2022-06-21 22:02:50 +00:00
@@ -31,7 +31,7 @@
"source": [
"<table align=\"left\">\n",
" <td>\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/feature_store/sdk-feature-store.ipynb\"\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/feature_store/sdk-feature-store.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" </a>\n",
" </td>\n",
@@ -66,12 +66,20 @@
"\n",
"### Objective\n",
"\n",
"In this notebook, you will learn how to:\n",
"In this notebook, you will learn how to use `Vertex AI Feature Store` to import feature data, and to access the feature data for both online serving and offline tasks, such as training.\n",
"\n",
" * Create featurestore, entity type, and feature resources. \n",
" * Import your features into Vertex AI Feature Store.\n",
" * Serve online prediction requests using the imported features.\n",
" * Access imported features in offline jobs, such as training jobs.\n",
"This tutorial uses the following Google Cloud ML services:\n",
"\n",
"- `Vertex AI Feature Store`\n",
"\n",
"\n",
"\n",
"The steps performed include:\n",
"\n",
"- Create featurestore, entity type, and feature resources.\n",
"- Import feature data into `Vertex AI Feature Store` resource.\n",
"- Serve online prediction requests using the imported features.\n",
"- Access imported features in offline jobs, such as training jobs.\n",
"\n",
"### Costs \n",
"\n",
@@ -152,9 +160,9 @@
"id": "i7EUnXsZhAGF"
},
"source": [
"### Install additional packages\n",
"## Installation\n",
"\n",
"For this notebook, you need the Vertex SDK for Python."
"Install the packages required for executing this notebook."
]
},
{
@@ -167,24 +175,18 @@
"source": [
"import os\n",
"\n",
"# The Google Cloud Notebook product has specific requirements\n",
"IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n",
"# The Vertex AI Workbench Notebook product has specific requirements\n",
"IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\") and not os.getenv(\"VIRTUAL_ENV\")\n",
"IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
" \"/opt/deeplearning/metadata/env_version\"\n",
")\n",
"\n",
"# Google Cloud Notebook requires dependencies to be installed with '--user'\n",
"# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
"USER_FLAG = \"\"\n",
"if IS_GOOGLE_CLOUD_NOTEBOOK:\n",
" USER_FLAG = \"--user\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wyy5Lbnzg5fi"
},
"outputs": [],
"source": [
"! pip install {USER_FLAG} --upgrade google-cloud-aiplatform"
"if IS_WORKBENCH_NOTEBOOK:\n",
" USER_FLAG = \"--user\"\n",
"\n",
"! pip3 install --upgrade google-cloud-aiplatform $USER_FLAG -q"
]
},
{
@@ -260,24 +262,7 @@
},
"outputs": [],
"source": [
"import os\n",
"\n",
"PROJECT_ID = \"\"\n",
"\n",
"# Get your Google Cloud project ID from gcloud\n",
"if not os.getenv(\"IS_TESTING\"):\n",
" shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
" PROJECT_ID = shell_output[0]\n",
" print(\"Project ID: \", PROJECT_ID)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qJYoRfYng0XZ"
},
"source": [
"Otherwise, set your project ID here."
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}"
]
},
{
@@ -290,7 +275,76 @@
"source": [
"if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
" PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"print(\"Project ID: \", PROJECT_ID)"
" # Get your GCP project id from gcloud\n",
" shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null\n",
" PROJECT_ID = shell_output[0]\n",
" print(\"Project ID:\", PROJECT_ID)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "250cb8c648d5"
},
"outputs": [],
"source": [
"! gcloud config set project $PROJECT_ID"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "region"
},
"source": [
"#### Region\n",
"\n",
"You can also change the `REGION` variable, which is used for operations\n",
"throughout the rest of this notebook. Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n",
"\n",
"- Americas: `us-central1`\n",
"- Europe: `europe-west4`\n",
"- Asia Pacific: `asia-east1`\n",
"\n",
"You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
"\n",
"Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "region"
},
"outputs": [],
"source": [
"REGION = \"us-central1\" # @param {type: \"string\"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "timestamp"
},
"source": [
"#### Timestamp\n",
"\n",
"If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "timestamp"
},
"outputs": [],
"source": [
"from datetime import datetime\n",
"\n",
"TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
]
},
{
@@ -343,19 +397,19 @@
},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"\n",
"# If you are running this notebook in Colab, run this cell and follow the\n",
"# instructions to authenticate your GCP account. This provides access to your\n",
"# Cloud Storage bucket and lets you submit training jobs and prediction\n",
"# requests.\n",
"\n",
"# The Google Cloud Notebook product has specific requirements\n",
"IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n",
"import os\n",
"import sys\n",
"\n",
"# If on Google Cloud Notebooks, then don't execute this code\n",
"if not IS_GOOGLE_CLOUD_NOTEBOOK:\n",
"# If on Vertex AI Workbench, then don't execute this code\n",
"IS_COLAB = \"google.colab\" in sys.modules\n",
"if not os.path.exists(\"/opt/deeplearning/metadata/env_version\") and not os.getenv(\n",
" \"DL_ANACONDA_HOME\"\n",
"):\n",
" if \"google.colab\" in sys.modules:\n",
" from google.colab import auth as google_auth\n",
"\n",
@@ -385,18 +439,11 @@
},
"outputs": [],
"source": [
"from google.cloud import aiplatform\n",
"from google.cloud.aiplatform import Feature, Featurestore\n",
"\n",
"REGION = \"[your-region]\" # @param {type:\"string\"}\n",
"if REGION == \"[your-region]\":\n",
" REGION = \"us-central1\"\n",
"\n",
"FEATURESTORE_ID = \"movie_prediction\"\n",
"INPUT_CSV_FILE = \"gs://cloud-samples-data-us-central1/vertex-ai/feature-store/datasets/movie_prediction.csv\"\n",
"ONLINE_STORE_FIXED_NODE_COUNT = 1\n",
"\n",
"aiplatform.init(project=PROJECT_ID, location=REGION)"
"ONLINE_STORE_FIXED_NODE_COUNT = 1"
]
},
{