Added redis info

This commit is contained in:
ivanmkc@google.com
2023-03-15 14:54:42 -04:00
parent 9a409b9011
commit c3526504d8
@@ -150,6 +150,24 @@
" tensorflow-hub"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Install the latest version of redis for low-latency data retrieval"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Install the redis package\n",
"! pip install --upgrade redis"
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -1015,6 +1033,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "6LCGvBNvBd8D"
@@ -1022,7 +1041,9 @@
"source": [
"## Create Online Queries\n",
"\n",
"After you built your indexes, you may query against the deployed index to find nearest neighbors."
"After you built your indexes, you may query against the deployed index to find nearest neighbors.\n",
"\n",
"Note: For the DOT_PRODUCT_DISTANCE distance type, the \"distance\" property returned with each MatchNeighbor actually refers to the similarity."
]
},
{
@@ -1087,6 +1108,76 @@
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Storing and retrieving titles from a Redis data store\n",
"When you productionize this code into a service, you will need to conver the nearest nearest id's returned from Vertex AI Matching Engine into data usable by downstream services.\n",
"\n",
"In this case, you'll need to convert the id's to titles.\n",
"\n",
"You can use Google Cloud's Memorystore to deploy a managed Redis instance to save the id-title key-value pairs.\n",
"\n",
"See more information on [Memorystore](https://cloud.google.com/memorystore/docs/redis/create-manage-instances?hl=en)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"REDIS_INSTANCE_NAME = \"stackoverflow-questions-unique\"\n",
"\n",
"# Create a Redis instance\n",
"! gcloud redis instances create '{REDIS_INSTANCE_NAME}' --size=5 --region={REGION} --network={VPC_NETWORK_FULL} --connect-mode=private-service-access"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get host and port info\n",
"REDIS_HOST = ! gcloud redis instances list --region {REGION} --format='value(HOST)'\n",
"REDIS_PORT = ! gcloud redis instances list --region {REGION} --format='value(PORT)'"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Connect to the instance\n",
"import redis\n",
"\n",
"redis_client = redis.StrictRedis(host=REDIS_HOST, port=REDIS_PORT)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Convert the id -> title relationship into a dict and write to redis\n",
"redis_client.mset({str(id): str(title) for id, title in zip(df.id, df.title)})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Verify that redis can retrieve the correct information\n",
"[f\"Actual = {title}, Retrieved = {redis_client.get(str(id))}\" for id, title in list(zip(df.id, df.title))[:10]]"
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -1123,6 +1214,16 @@
"# Delete indexes\n",
"tree_ah_index.delete()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete redis instance\n",
"! gcloud redis instances delete '{REDIS_INSTANCE_NAME}' --region {REGION} --quiet"
]
}
],
"metadata": {