Compare commits

..
Author SHA1 Message Date
Andrew Ferlitsch 6693d5e589 debug: experiments 2023-01-19 19:00:01 +00:00
Andrew Ferlitsch 137cd5d74a migration: experiments 2023-01-13 22:34:40 +00:00
Andrew Ferlitsch ddfb287480 migration: experiments 2023-01-13 20:55:13 +00:00
5 changed files with 200 additions and 3513 deletions
File diff suppressed because it is too large Load Diff
@@ -40,18 +40,18 @@
"source": [
"<table align=\"left\">\n",
" <td>\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/t5x_base_optimized_online_prediction.ipynb\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/bert_optimized_online_prediction.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/t5x_base_optimized_online_prediction.ipynb\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/bert_optimized_online_prediction.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/t5x_base_optimized_online_prediction.ipynb\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/bert_optimized_online_prediction.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
File diff suppressed because it is too large Load Diff
@@ -31,7 +31,7 @@
"id": "fsv4jGuU89rX"
},
"source": [
"# Vertex AI Model Monitoring for AutoML tabular models\n",
"# E2E ML on GCP: MLOps stage 7 : monitoring: Vertex AI Model Monitoring for AutoML tabular models\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
@@ -51,7 +51,7 @@
" Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
"</table>\n"
"</table>"
]
},
{
@@ -91,6 +91,7 @@
"- Deploy the `Model` resource to the `Endpoint` resource.\n",
"- Configure the `Endpoint` resource for model monitoring.\n",
"- Generate synthetic prediction requests for skew.\n",
"- Wait for email alert notification.\n",
"- Generate synthetic prediction requests for drift.\n",
"- Wait for email alert notification.\n",
"\n",
@@ -105,7 +106,7 @@
"source": [
"### Dataset\n",
"\n",
"The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). In this notebook, you use only the fields year, month and day from the dataset to predict the value of mean daily temperature (mean_temp)."
"The dataset used for this tutorial is the GSOD dataset from [BigQuery public datasets](https://cloud.google.com/bigquery/public-data). The version of this dataset you use only the fields year, month and day to predict the value of mean daily temperature (mean_temp)."
]
},
{
@@ -562,7 +563,7 @@
"source": [
"### Create BigQuery client\n",
"\n",
"In this tutorial, you explore the monitoring data stored in BigQuery. You create a client interface, which you subsequently use to access the data."
"In this tutorial, you use data from the same public BigQuery table that was used to train the pre-trained model. You create a client interface, which you subsequently use to access the data."
]
},
{
@@ -667,6 +668,23 @@
"print(dataset.resource_name)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "set_transformations:gsod"
},
"outputs": [],
"source": [
"TRANSFORMATIONS = [\n",
" {\"auto\": {\"column_name\": \"year\"}},\n",
" {\"auto\": {\"column_name\": \"month\"}},\n",
" {\"auto\": {\"column_name\": \"day\"}},\n",
"]\n",
"\n",
"label_column = \"mean_temp\""
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -685,7 +703,7 @@
"- `optimization_prediction_type`: The type task to train the model for.\n",
" - `classification`: A tabuar classification model.\n",
" - `regression`: A tabular regression model.\n",
"- `column_transformations`: (Optional): Transformations to apply to the input columns. In this example, you set the column transformations to use the default transformation based on their data type.\n",
"- `column_transformations`: (Optional): Transformations to apply to the input columns\n",
"- `optimization_objective`: The optimization objective to minimize or maximize.\n",
" - binary classification:\n",
" - `minimize-log-loss`\n",
@@ -701,23 +719,6 @@
" - `minimize-rmsle`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "set_transformations:gsod"
},
"outputs": [],
"source": [
"TRANSFORMATIONS = [\n",
" {\"auto\": {\"column_name\": \"year\"}},\n",
" {\"auto\": {\"column_name\": \"month\"}},\n",
" {\"auto\": {\"column_name\": \"day\"}},\n",
"]\n",
"\n",
"label_column = \"mean_temp\""
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -1064,9 +1065,7 @@
"You are receiving this mail because you are using the Vertex AI Model Monitoring service.\n",
"This mail is to inform you that we received your request to set up drift or skew detection for the Prediction Endpoint listed below. Starting from now, incoming prediction requests will be sampled and logged for analysis.\n",
"Raw requests and responses will be collected from prediction service and saved in bq://[your-project-id].model_deployment_monitoring_[endpoint-id].serving_predict .\n",
"</blockquote>\n",
"\n",
"*Note:* You do not need to wait for the email notification to continue to the next step."
"</blockquote>"
]
},
{
File diff suppressed because one or more lines are too long