Compare commits

...
Author SHA1 Message Date
Andrew FerlitschandGitHub 64f9fb115b fix: links after rename UJ9 2023-04-20 12:28:03 -07:00
Xiang XuandGitHub 0661fcc121 fix blip2 (#1771) 2023-04-20 19:20:47 +00:00
Andrew FerlitschandGitHub f4c0fc7781 fix: rename UJ1 (#1757)
* fix: rename UJ1

* fix: lint
2023-04-20 18:22:47 +00:00
Andrew FerlitschandGitHub 8e3db693cb fix: UJ9 (#1770) 2023-04-20 18:22:15 +00:00
Andrew FerlitschandGitHub 0349a9552d fix: UJ8 (#1769) 2023-04-20 18:21:45 +00:00
Andrew FerlitschandGitHub ac1f03dd71 fix: rename UJ7 (#1768) 2023-04-20 18:21:16 +00:00
Andrew FerlitschandGitHub cb18a37b0a fix: rename UJ6 (#1767) 2023-04-20 18:20:51 +00:00
Andrew FerlitschandGitHub 4c8ec05f22 fix: rename UJ5 (#1766) 2023-04-20 18:20:28 +00:00
Andrew FerlitschandGitHub c17df33c7c fix: rename UJ4 (#1765)
* fix: rename UJ4

* fix: lint
2023-04-20 18:20:03 +00:00
Andrew FerlitschandGitHub 543cd66c16 fix: rename UJ3 (#1764) 2023-04-20 18:19:36 +00:00
Andrew FerlitschandGitHub f50dac63eb fix: rename UJ2 (#1763) 2023-04-20 18:19:08 +00:00
Andrew FerlitschandGitHub df979c1c65 fix: rename UJ15 (#1762) 2023-04-20 18:18:21 +00:00
Andrew FerlitschandGitHub 7b49c23eb6 fix: rename UJ14 (#1761)
* fix: rename UJ14

* fix: lint
2023-04-20 18:17:51 +00:00
Andrew FerlitschandGitHub 965835122b fix: rename UJ11 (#1759) 2023-04-20 18:17:22 +00:00
Andrew FerlitschandGitHub 54a34a7122 fix: rename UJ10 (#1758) 2023-04-20 18:17:00 +00:00
Xiang XuandGitHub a701f3b08c fix controlnet (#1756) 2023-04-20 04:44:53 +00:00
59a220cb58 fix: pin kfp and gcpc < 2.0 in rapid_prototyping_bqml_automl (#1755)
* pin kfp in rapid_prototyping_bqml_automl

* downgrade component version

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@gmail.com>
2023-04-20 04:36:39 +00:00
26 changed files with 152 additions and 25 deletions
@@ -271,7 +271,7 @@
"\n",
"\n",
"def deploy_model(model_id, task):\n",
" model_name = \"blip-image-captioning\"\n",
" model_name = \"blip2\"\n",
" endpoint = aiplatform.Endpoint.create(display_name=f\"{model_name}-endpoint\")\n",
" serving_env = {\n",
" \"MODEL_ID\": model_id,\n",
@@ -348,7 +348,7 @@
"id": "12893aa2c5af"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. When the model is very large it could add 5~15mins additional time before the endpoint is ready for prediction."
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 10 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
@@ -412,7 +412,7 @@
"id": "0ac7f8d945e3"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. When the model is very large it could add 5~15mins additional time before the endpoint is ready for prediction."
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 10 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -278,6 +278,8 @@
"outputs": [],
"source": [
"import base64\n",
"import os\n",
"from datetime import datetime\n",
"from io import BytesIO\n",
"\n",
"import cv2\n",
@@ -410,6 +412,13 @@
"# for a full list of training arguments.\n",
"model = job.run(\n",
" args=[\n",
" f\"--num_machines={num_nodes}\",\n",
" f\"--num_processes={num_gpus}\",\n",
" \"--machine_rank=0\",\n",
" \"--mixed_precision=no\",\n",
" \"--gpu_ids=all\",\n",
" \"--same_network\",\n",
" \"--dynamo_backend=no\",\n",
" \"controlnet/train_controlnet.py\",\n",
" \"--tracker_project_name=train_controlnet\",\n",
" f\"--pretrained_model_name_or_path={stable_diffusion_model_id}\",\n",
@@ -477,6 +486,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "da804e9b99a8"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -520,11 +538,11 @@
},
"outputs": [],
"source": [
"# Delete models.\n",
"model.delete()\n",
"\n",
"# Undeploy model and delete endpoint.\n",
"endpoint.delete(force=True)"
"endpoint.delete(force=True)\n",
"\n",
"# Delete models.\n",
"model.delete()"
]
},
{
@@ -558,6 +576,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d5fa0300f00f"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -352,6 +352,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "48ff57f4f342"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -351,6 +351,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -510,6 +510,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9dc3c39db066"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -585,6 +594,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1139512c6fa9"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -510,6 +510,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9dc3c39db066"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -332,6 +332,15 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "80b3fd2ace09"
},
"source": [
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional 5 minutes of waiting time is needed before the endpoint is ready to use. Otherwise you might see an `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -1458,7 +1458,7 @@
],
"metadata": {
"colab": {
"name": "UJ1 Vertex SDK AutoML Image Classification.ipynb",
"name": "sdk-automl-image-classification-batch-online.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1454,7 +1454,7 @@
],
"metadata": {
"colab": {
"name": "UJ5 Vertex SDK AutoML Image Object Detection.ipynb",
"name": "sdk-automl-image-object-detection-batch-online.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -633,8 +633,9 @@
" gcs_source=[IMPORT_FILE],\n",
" import_schema_uri=aip.schema.dataset.ioformat.video.object_tracking,\n",
")\n",
"if os.getenv('IS_TESTING'):\n",
"if os.getenv(\"IS_TESTING\"):\n",
" import time\n",
"\n",
" time.sleep(30)\n",
"\n",
"print(dataset.resource_name)"
@@ -1251,7 +1252,7 @@
],
"metadata": {
"colab": {
"name": "UJ15 Vertex SDK AutoML Object Tracking.ipynb",
"name": "sdk-automl-object-tracking-batch-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1410,7 +1410,7 @@
],
"metadata": {
"colab": {
"name": "UJ4 Vertex SDK AutoML Tabular Binary Classification.ipynb",
"name": "sdk-automl-tabular-binary-classification-online-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1337,7 +1337,7 @@
],
"metadata": {
"colab": {
"name": "UJ6 Vertex SDK AutoML Text Classification.ipynb",
"name": "sdk-automl-text-classification-batch-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1428,7 +1428,7 @@
],
"metadata": {
"colab": {
"name": "UJ7 Vertex SDK AutoML Text Entity Extraction.ipynb",
"name": "sdk-automl-text-entity-extraction-batch-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1413,7 +1413,7 @@
],
"metadata": {
"colab": {
"name": "UJ8 Vertex SDK AutoML Text Sentiment Analysis.ipynb",
"name": "sdk-automl-text-sentiment-analysis-batch-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1297,7 +1297,7 @@
],
"metadata": {
"colab": {
"name": "UJ14 Vertex SDK AutoML Video Classification.ipynb",
"name": "sdk-automl-video-classification-batch-prediction.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1899,7 +1899,7 @@
],
"metadata": {
"colab": {
"name": "UJ3 Vertex SDK Custom Image Classification with custom training container.ipynb",
"name": "sdk-custom-image-classification-custom-container.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -2005,7 +2005,7 @@
],
"metadata": {
"colab": {
"name": "UJ2,12 Vertex SDK Custom Image Classification with pre-built training container.ipynb",
"name": "sdk-custom-image-classification-prebuilt-container.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1577,7 +1577,7 @@
],
"metadata": {
"colab": {
"name": "UJ10 Vertex SDK Custom Scikit-Learn with pre-built training container.ipynb",
"name": "sdk-custom-scikit-learn-prebuilt-container.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -33,18 +33,18 @@
"\n",
"<table align=\"left\">\n",
" <td>\n",
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
"<a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
"<a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" </a>\n",
" </td>\n",
" <td>\n",
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb\" target='_blank'>\n",
"<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/migration/sdk-custom-xgboost-prebuilt-container.ipynb\" target='_blank'>\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
@@ -1459,7 +1459,7 @@
],
"metadata": {
"colab": {
"name": "UJ9 Vertex SDK Custom XGBoost with pre-built training container.ipynb",
"name": "sdk-custom-xgboost-prebuilt-container.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -1414,7 +1414,7 @@
],
"metadata": {
"colab": {
"name": "UJ11 Vertex SDK Hyperparameter Tuning.ipynb",
"name": "sdk-hyperparameter-tuning.ipynb",
"toc_visible": true
},
"kernelspec": {
@@ -265,7 +265,7 @@
"source": [
"# Install Python package dependencies.\n",
"print(\"Installing libraries\")\n",
"! pip3 install {USER_FLAG} --quiet google-cloud-pipeline-components kfp\n",
"! pip3 install {USER_FLAG} --quiet 'google-cloud-pipeline-components==1.0.20' 'kfp<2'\n",
"! pip3 install {USER_FLAG} --quiet --upgrade google-cloud-aiplatform google-cloud-bigquery"
]
},