mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Add controlnet-canny to the Gradio playground, and some additional UX enhancements. (#2783)
* Add controlnet-canny to the Gradio playground, and some additional UX enhancement. * Minor fixes. * Minor fixes
This commit is contained in:
weigary
+263
-104
@@ -29,7 +29,7 @@
|
||||
"id": "99c1c3fc2ca5"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI Model Garden - Image Generation Playground based on Gradio UI\n",
|
||||
"# Vertex AI Model Garden Playground for Image Generation (inspired by Stable Diffusion WebUI)\n",
|
||||
"\n",
|
||||
"<table><tbody><tr>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
@@ -53,13 +53,13 @@
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook demonstrates starting a playground based on [Gradio UI](https://www.gradio.app/), which allows users to interact with the stable diffusion models more easily and intuitively. The playground now support `text-to-image`, `image-to-image` and `image-inpainting` tasks.\n",
|
||||
"This notebook demonstrates starting a playground based on [Gradio UI](https://www.gradio.app/), inspired by the famous [Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) project, which allows users to interact with the stable diffusion models more easily and intuitively. The playground now support `text-to-image`, `image-inpainting`, `controlnet-canny`, `instruct-pix2pix`, and `SD 4x upscaler` tasks\n",
|
||||
"\n",
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"- Deploy model to a [Vertex AI Endpoint resource](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
|
||||
"- Run online predictions for `text-to-image`, `image-inpainting`, `instruct-pix2pix`, and `SD 4x upscaler` tasks, from the UI.\n",
|
||||
"- Adjust the parameters, such as prompt, negative_prompt, num_inference_steps, and check out the generated images.\n",
|
||||
"- Run online predictions for `text-to-image`, `image-inpainting`, `controlnet-canny`, `instruct-pix2pix`, and `SD 4x upscaler` tasks, from the UI.\n",
|
||||
"- Adjust the parameters, such as prompt, negative_prompt, num_inference_steps, and check out the generated images for best image quality.\n",
|
||||
"\n",
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
@@ -74,10 +74,10 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xvlByex0zwU0"
|
||||
"id": "879fca33129c"
|
||||
},
|
||||
"source": [
|
||||
"## Start the Gradio UI playground"
|
||||
"## Run the playground"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -93,7 +93,7 @@
|
||||
"\n",
|
||||
"# @markdown [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"! pip3 install --upgrade gradio==3.48.0\n",
|
||||
"! pip3 install --upgrade gradio==3.48.0 opencv-python\n",
|
||||
"\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
@@ -120,7 +120,7 @@
|
||||
"print(\"Using this default Service Account:\", SERVICE_ACCOUNT)\n",
|
||||
"\n",
|
||||
"# The pre-built serving docker image. It contains serving scripts and models.\n",
|
||||
"SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-diffusers-serve-opt:20240223_1230_RC00\"\n",
|
||||
"SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-diffusers-serve-opt:20240315_1230_RC00\"\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
@@ -132,32 +132,43 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "1cc26e68d7b0"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# @title Run the playground\n",
|
||||
"# @title Start the playground\n",
|
||||
"\n",
|
||||
"# @markdown This is a simple playground similar to the popular [stable diffusion webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui).\n",
|
||||
"# @markdown This UI is avaible in a separate browser tab if you click the public URL after the cell runs.\n",
|
||||
"# @markdown The public URL is something similar to \"https://####.gradio.live\". Click the URL to open the playground.\n",
|
||||
"# @markdown This is a playground for image generation similar to the popular [Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui).\n",
|
||||
"# @markdown After the cell runs, this playground is avaible in a separate browser tab if you click the public URL.\n",
|
||||
"# @markdown Sometsomething similar to [\"https://####.gradio.live\"]() in the output of the cell.\n",
|
||||
"\n",
|
||||
"# @markdown Before you start, you need to select an existing Vertex prediction endpoint from the dropdown list\n",
|
||||
"# @markdown which has been deployed in the project and region; If no models were deployed in the past, you can\n",
|
||||
"# @markdown create a new Vertex prediction endpoint by selecting your favorite model and click \"Deploy\".\n",
|
||||
"# @markdown Five tasks `text-to-image`, `image-inpainting`, `controlnet-canny`, `instruct-pix2pix` and `SD 4x upscaler` are currently supported.\n",
|
||||
"\n",
|
||||
"# @markdown **How to use:**\n",
|
||||
"# @markdown 1. Before you start, you need to select a Vertex prediction endpoint, with a matching model deployed to the endpoint\n",
|
||||
"# @markdown from the endpoint dropdown list, that has been deployed in the project and region;\n",
|
||||
"# @markdown 2. Make sure the selected endpoint/model match with the chosen task. Mismatched task and model will produce unreliable results.\n",
|
||||
"# @markdown 3. If no models were deployed in the past, you can create a new Vertex prediction\n",
|
||||
"# @markdown endpoint by selecting your favorite model and click \"Deploy\".\n",
|
||||
"# @markdown 4. New model deployment takes ~20 minutes. You can check the progress at [Vertex Online Prediction](https://console.cloud.google.com/vertex-ai/online-prediction/endpoints).\n",
|
||||
"# @markdown 5. Adjust the prompt/negative-prompt, image-dimension, inference steps, guidance-scale to achieve the optimum image quality.\n",
|
||||
"# @markdown 6. Don't forget to undeploy the models after all the experiment to avoid continuous charges to the project.\n",
|
||||
"\n",
|
||||
"# @markdown Four tasks `text-to-image`, `image-inpainting`, `instruct-pix2pix` and `SD 4x upscaler` are currently supported.\n",
|
||||
"\n",
|
||||
"import base64\n",
|
||||
"from datetime import datetime\n",
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import cv2\n",
|
||||
"import gradio as gr\n",
|
||||
"import numpy as np\n",
|
||||
"from google.cloud import aiplatform\n",
|
||||
"from PIL import Image\n",
|
||||
"\n",
|
||||
"# The pre-built serving docker image. It contains serving scripts and models.\n",
|
||||
"SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-diffusers-serve-opt:20240306_1230_RC00\"\n",
|
||||
"def create_job_name(prefix):\n",
|
||||
" now = datetime.now().strftime(\"%y%m%d-%H%M%S\")\n",
|
||||
" job_name = f\"{prefix}-gradio-{now}\"\n",
|
||||
" return job_name\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def base64_to_image(image_str: str) -> Image:\n",
|
||||
@@ -173,12 +184,42 @@
|
||||
" return image_str\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def canny(image, low_threshold=100, high_threshold=200) -> Image.Image:\n",
|
||||
" image = np.array(image)\n",
|
||||
" image = cv2.Canny(image, low_threshold, high_threshold)\n",
|
||||
"\n",
|
||||
" image = image[:, :, None]\n",
|
||||
" image = np.concatenate([image, image, image], axis=2)\n",
|
||||
" image = Image.fromarray(image)\n",
|
||||
" return image\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def is_image_generation_endpoint(endpoint: aiplatform.Endpoint) -> bool:\n",
|
||||
" \"\"\"Returns True if the endpoint is an image generation endpoint.\"\"\"\n",
|
||||
" return (\n",
|
||||
" \"sd\" in endpoint.display_name.lower()\n",
|
||||
" or \"diffusion\" in endpoint.display_name.lower()\n",
|
||||
" or \"inpaint\" in endpoint.display_name.lower()\n",
|
||||
" or \"controlnet\" in endpoint.display_name.lower()\n",
|
||||
" or \"pix2pix\" in endpoint.display_name.lower()\n",
|
||||
" or \"upscaler\" in endpoint.display_name.lower()\n",
|
||||
" or \"canny\" in endpoint.display_name.lower()\n",
|
||||
" or \"gradio\" in endpoint.display_name.lower()\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def list_endpoints() -> list[str]:\n",
|
||||
" \"\"\"Returns all valid prediction endpoints for in the project and region.\"\"\"\n",
|
||||
" # Gets all the valid endpoints in the project and region.\n",
|
||||
" endpoints = aiplatform.Endpoint.list(order_by=\"create_time desc\")\n",
|
||||
" # Filters out the endpoints which do not have a deployed model\n",
|
||||
" endpoints = list(filter(lambda endpoint: endpoint.traffic_split, endpoints))\n",
|
||||
" # Filters out the endpoints which do not have a deployed model, and the endpoint is for image generation\n",
|
||||
" endpoints = list(\n",
|
||||
" filter(\n",
|
||||
" lambda endpoint: endpoint.traffic_split\n",
|
||||
" and is_image_generation_endpoint(endpoint),\n",
|
||||
" endpoints,\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" endpoint_names = list(\n",
|
||||
" map(\n",
|
||||
@@ -221,10 +262,12 @@
|
||||
" \"kandinsky-community/kandinsky-2-2-decoder-inpaint\": \"image-inpainting\",\n",
|
||||
" \"diffusers/stable-diffusion-xl-1.0-inpainting-0.1\": \"image-inpainting\",\n",
|
||||
" \"timbrooks/instruct-pix2pix\": \"instruct-pix2pix\",\n",
|
||||
" \"lllyasviel/sd-controlnet-canny\": \"controlnet\",\n",
|
||||
" \"stabilityai/stable-diffusion-x4-upscaler\": \"conditioned-super-res\",\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" if model_name not in model_to_task_dict.keys():\n",
|
||||
" print(model_name)\n",
|
||||
" raise gr.Error(\"Please select a valid model name for Endpoint creation.\")\n",
|
||||
"\n",
|
||||
" return model_to_task_dict[model_name]\n",
|
||||
@@ -238,8 +281,7 @@
|
||||
" raise gr.Error(\"Please select a valid model name for model list.\")\n",
|
||||
" return\n",
|
||||
"\n",
|
||||
" gr.Info(\"Model is being deployed. It may take ~20 minutes to complete.\")\n",
|
||||
"\n",
|
||||
" model_name = model_name.split(\": \")[1]\n",
|
||||
" task_name = get_task_name(model_name)\n",
|
||||
" model_id = model_name\n",
|
||||
" if (\n",
|
||||
@@ -260,8 +302,9 @@
|
||||
" \"REFINER_MODEL_ID\": refiner_model_id,\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" display_name = create_job_name(model_name)\n",
|
||||
" model = aiplatform.Model.upload(\n",
|
||||
" display_name=model_name,\n",
|
||||
" display_name=display_name,\n",
|
||||
" serving_container_image_uri=SERVE_DOCKER_URI,\n",
|
||||
" serving_container_ports=[7080],\n",
|
||||
" serving_container_predict_route=\"/predictions/diffusers_serving\",\n",
|
||||
@@ -278,9 +321,12 @@
|
||||
" accelerator_count=1,\n",
|
||||
" deploy_request_timeout=1800,\n",
|
||||
" service_account=SERVICE_ACCOUNT,\n",
|
||||
" sync=False,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" gr.Info(\"Model have been deployed successfully.\")\n",
|
||||
" gr.Info(\n",
|
||||
" f\"Model {display_name} is being deployed. It may take ~20 minutes to complete.\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return endpoint\n",
|
||||
"\n",
|
||||
@@ -294,8 +340,10 @@
|
||||
"\n",
|
||||
" if \"stable-diffusion-xl\" in model_name or \"sdxl\" in model_name:\n",
|
||||
" dimension = 1024\n",
|
||||
" elif \"stable-diffusion-2-1\" in model_name:\n",
|
||||
" if \"stable-diffusion-2-1\" in model_name:\n",
|
||||
" dimension = 768\n",
|
||||
" if \"sdxl-turbo\" in model_name:\n",
|
||||
" dimension = 512\n",
|
||||
"\n",
|
||||
" return dimension\n",
|
||||
"\n",
|
||||
@@ -335,8 +383,7 @@
|
||||
" num_samples=1,\n",
|
||||
" guidance_scale=7.5,\n",
|
||||
" num_inference_steps=25,\n",
|
||||
" height=512,\n",
|
||||
" width=512,\n",
|
||||
" image_dimension=512,\n",
|
||||
") -> list[Image.Image]:\n",
|
||||
" if not endpoint_name:\n",
|
||||
" raise gr.Error(\"Please select (or deploy) a model first!\")\n",
|
||||
@@ -345,8 +392,8 @@
|
||||
" {\n",
|
||||
" \"prompt\": prompt,\n",
|
||||
" \"negative_prompt\": negative_prompt,\n",
|
||||
" \"height\": height,\n",
|
||||
" \"width\": width,\n",
|
||||
" \"height\": image_dimension,\n",
|
||||
" \"width\": image_dimension,\n",
|
||||
" \"guidance_scale\": guidance_scale,\n",
|
||||
" \"num_inference_steps\": num_inference_steps,\n",
|
||||
" },\n",
|
||||
@@ -369,8 +416,7 @@
|
||||
" guidance_scale=7.5,\n",
|
||||
" num_inference_steps=25,\n",
|
||||
" dict=None,\n",
|
||||
" height=512,\n",
|
||||
" width=512,\n",
|
||||
" image_dimension=512,\n",
|
||||
") -> list[Image.Image]:\n",
|
||||
" if not endpoint_name:\n",
|
||||
" raise gr.Error(\"Please select (or deploy) a model first!\")\n",
|
||||
@@ -391,8 +437,8 @@
|
||||
" \"negative_prompt\": negative_prompt,\n",
|
||||
" \"image\": image_to_base64(init_image),\n",
|
||||
" \"mask_image\": image_to_base64(mask),\n",
|
||||
" \"height\": height,\n",
|
||||
" \"width\": width,\n",
|
||||
" \"height\": image_dimension,\n",
|
||||
" \"width\": image_dimension,\n",
|
||||
" \"guidance_scale\": guidance_scale,\n",
|
||||
" \"num_inference_steps\": num_inference_steps,\n",
|
||||
" },\n",
|
||||
@@ -415,8 +461,7 @@
|
||||
" guidance_scale=7.5,\n",
|
||||
" num_inference_steps=25,\n",
|
||||
" init_image=None,\n",
|
||||
" height=512,\n",
|
||||
" width=512,\n",
|
||||
" image_dimension=512,\n",
|
||||
") -> list[Image.Image]:\n",
|
||||
" if not endpoint_name:\n",
|
||||
" raise gr.Error(\"Please select (or deploy) a model first!\")\n",
|
||||
@@ -426,8 +471,8 @@
|
||||
" \"prompt\": prompt,\n",
|
||||
" \"negative_prompt\": negative_prompt,\n",
|
||||
" \"image\": image_to_base64(init_image),\n",
|
||||
" \"height\": height,\n",
|
||||
" \"width\": width,\n",
|
||||
" \"height\": image_dimension,\n",
|
||||
" \"width\": image_dimension,\n",
|
||||
" \"guidance_scale\": guidance_scale,\n",
|
||||
" \"num_inference_steps\": num_inference_steps,\n",
|
||||
" },\n",
|
||||
@@ -450,9 +495,9 @@
|
||||
" guidance_scale=7.5,\n",
|
||||
" num_inference_steps=25,\n",
|
||||
" init_image=None,\n",
|
||||
" height=512,\n",
|
||||
" width=512,\n",
|
||||
" image_dimension=512,\n",
|
||||
") -> list[Image.Image]:\n",
|
||||
"\n",
|
||||
" if not endpoint_name:\n",
|
||||
" raise gr.Error(\"Please select (or deploy) a model first!\")\n",
|
||||
"\n",
|
||||
@@ -466,8 +511,8 @@
|
||||
" \"prompt\": prompt,\n",
|
||||
" \"negative_prompt\": negative_prompt,\n",
|
||||
" \"image\": image_to_base64(init_image),\n",
|
||||
" \"height\": height,\n",
|
||||
" \"width\": width,\n",
|
||||
" \"height\": image_dimension,\n",
|
||||
" \"width\": image_dimension,\n",
|
||||
" \"guidance_scale\": guidance_scale,\n",
|
||||
" \"num_inference_steps\": num_inference_steps,\n",
|
||||
" },\n",
|
||||
@@ -482,53 +527,120 @@
|
||||
" return images\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def controlnet_generate_images(\n",
|
||||
" endpoint_name: str,\n",
|
||||
" prompt=\"\",\n",
|
||||
" negative_prompt=\"\",\n",
|
||||
" num_samples=1,\n",
|
||||
" guidance_scale=7.5,\n",
|
||||
" num_inference_steps=25,\n",
|
||||
" init_image=None,\n",
|
||||
" image_dimension=512,\n",
|
||||
" canny_low_threshold=100,\n",
|
||||
" canny_high_threshold=200,\n",
|
||||
") -> list[Image.Image]:\n",
|
||||
" if not endpoint_name:\n",
|
||||
" raise gr.Error(\"Please select (or deploy) a model first!\")\n",
|
||||
"\n",
|
||||
" init_image = init_image.convert(\"RGB\").resize((image_dimension, image_dimension))\n",
|
||||
"\n",
|
||||
" canny_image = canny(init_image, canny_low_threshold, canny_high_threshold)\n",
|
||||
"\n",
|
||||
" instances = [\n",
|
||||
" {\n",
|
||||
" \"prompt\": prompt,\n",
|
||||
" \"negative_prompt\": negative_prompt,\n",
|
||||
" \"image\": image_to_base64(canny_image),\n",
|
||||
" \"height\": image_dimension,\n",
|
||||
" \"width\": image_dimension,\n",
|
||||
" \"guidance_scale\": guidance_scale,\n",
|
||||
" \"num_inference_steps\": num_inference_steps,\n",
|
||||
" },\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" if len(instances) == 1 and num_samples > 1:\n",
|
||||
" instances = instances * num_samples\n",
|
||||
"\n",
|
||||
" response = get_endpoint(endpoint_name).predict(instances=instances)\n",
|
||||
" images = [base64_to_image(image) for image in response.predictions]\n",
|
||||
"\n",
|
||||
" images.insert(0, canny_image)\n",
|
||||
"\n",
|
||||
" return images\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def select_interface(interface_name: str):\n",
|
||||
" if interface_name == \"Text2Image pipeline\":\n",
|
||||
" if interface_name == \"Text2Image\":\n",
|
||||
" return {\n",
|
||||
" endpoint_name: gr.update(visible=True, value=None),\n",
|
||||
" prompt: gr.update(visible=True, value=None),\n",
|
||||
" negative_prompt: gr.update(visible=True, value=None),\n",
|
||||
" image_input: gr.update(visible=False, value=None),\n",
|
||||
" image_input: gr.update(visible=False, value=None, label=\"Upload\"),\n",
|
||||
" image_output: gr.update(value=None),\n",
|
||||
" generate_button: gr.update(visible=True),\n",
|
||||
" inpaint_generate_button: gr.update(visible=False),\n",
|
||||
" instruct_pix2pix_generate_button: gr.update(visible=False),\n",
|
||||
" controlnet_generate_button: gr.update(visible=False),\n",
|
||||
" upscaler_generate_button: gr.update(visible=False),\n",
|
||||
" canny_low_threshold: gr.update(visible=False),\n",
|
||||
" canny_high_threshold: gr.update(visible=False),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" elif interface_name == \"Inpaint Pipeline\":\n",
|
||||
" elif interface_name == \"Inpainting\":\n",
|
||||
" return {\n",
|
||||
" endpoint_name: gr.update(visible=True, value=None),\n",
|
||||
" prompt: gr.update(visible=True, value=None),\n",
|
||||
" negative_prompt: gr.update(visible=True, value=None),\n",
|
||||
" image_input: gr.update(visible=True, value=None, tool=\"sketch\"),\n",
|
||||
" image_input: gr.update(\n",
|
||||
" visible=True, value=None, tool=\"sketch\", label=\"Upload\"\n",
|
||||
" ),\n",
|
||||
" image_output: gr.update(value=None),\n",
|
||||
" generate_button: gr.update(visible=False),\n",
|
||||
" inpaint_generate_button: gr.update(visible=True),\n",
|
||||
" instruct_pix2pix_generate_button: gr.update(visible=False),\n",
|
||||
" controlnet_generate_button: gr.update(visible=False),\n",
|
||||
" upscaler_generate_button: gr.update(visible=False),\n",
|
||||
" canny_low_threshold: gr.update(visible=False),\n",
|
||||
" canny_high_threshold: gr.update(visible=False),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" elif interface_name == \"Instruct pix2pix Pipeline\":\n",
|
||||
" elif interface_name == \"Instruct pix2pix\":\n",
|
||||
" return {\n",
|
||||
" endpoint_name: gr.update(visible=True, value=None),\n",
|
||||
" prompt: gr.update(visible=True, value=None),\n",
|
||||
" negative_prompt: gr.update(visible=True, value=None),\n",
|
||||
" image_input: gr.update(visible=True, value=None, tool=\"None\"),\n",
|
||||
" image_input: gr.update(\n",
|
||||
" visible=True, value=None, tool=\"None\", label=\"Upload\"\n",
|
||||
" ),\n",
|
||||
" image_output: gr.update(value=None),\n",
|
||||
" generate_button: gr.update(visible=False),\n",
|
||||
" inpaint_generate_button: gr.update(visible=False),\n",
|
||||
" instruct_pix2pix_generate_button: gr.update(visible=True),\n",
|
||||
" controlnet_generate_button: gr.update(visible=False),\n",
|
||||
" upscaler_generate_button: gr.update(visible=False),\n",
|
||||
" canny_low_threshold: gr.update(visible=False),\n",
|
||||
" canny_high_threshold: gr.update(visible=False),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" elif interface_name == \"SD 4x Upscaler Pipeline\":\n",
|
||||
" elif interface_name == \"ControlNet Canny\":\n",
|
||||
" return {\n",
|
||||
" endpoint_name: gr.update(visible=True, value=None),\n",
|
||||
" prompt: gr.update(visible=True, value=None),\n",
|
||||
" negative_prompt: gr.update(visible=True, value=None),\n",
|
||||
" image_input: gr.update(visible=True, value=None, tool=\"None\"),\n",
|
||||
" image_input: gr.update(\n",
|
||||
" visible=True, value=None, tool=\"None\", label=\"Upload a reference image\"\n",
|
||||
" ),\n",
|
||||
" image_output: gr.update(value=None),\n",
|
||||
" generate_button: gr.update(visible=False),\n",
|
||||
" inpaint_generate_button: gr.update(visible=False),\n",
|
||||
" instruct_pix2pix_generate_button: gr.update(visible=False),\n",
|
||||
" controlnet_generate_button: gr.update(visible=True),\n",
|
||||
" upscaler_generate_button: gr.update(visible=False),\n",
|
||||
" canny_low_threshold: gr.update(visible=True),\n",
|
||||
" canny_high_threshold: gr.update(visible=True),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" elif interface_name == \"SD 4x Upscaler\":\n",
|
||||
" return {\n",
|
||||
" image_input: gr.update(\n",
|
||||
" visible=True, value=None, tool=\"None\", label=\"Upload\"\n",
|
||||
" ),\n",
|
||||
" image_output: gr.update(value=None),\n",
|
||||
" generate_button: gr.update(visible=False),\n",
|
||||
" inpaint_generate_button: gr.update(visible=False),\n",
|
||||
" instruct_pix2pix_generate_button: gr.update(visible=False),\n",
|
||||
" controlnet_generate_button: gr.update(visible=False),\n",
|
||||
" upscaler_generate_button: gr.update(visible=True),\n",
|
||||
" canny_low_threshold: gr.update(visible=False),\n",
|
||||
" canny_high_threshold: gr.update(visible=False),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
"\n",
|
||||
@@ -539,27 +651,40 @@
|
||||
" num_inference_steps: gr.update(\n",
|
||||
" value=get_default_num_inference_steps(model_name)\n",
|
||||
" ),\n",
|
||||
" height: gr.update(value=get_default_dimension(model_name)),\n",
|
||||
" width: gr.update(value=get_default_dimension(model_name)),\n",
|
||||
" image_dimension: gr.update(value=get_default_dimension(model_name)),\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"tip_text = r\"\"\"\n",
|
||||
"<b> Tips: </b>\n",
|
||||
"1. Select a Vertex prediction endpoint with a model deployed for your chosen task. Mismatched models can lead to unreliable outcomes.\n",
|
||||
"2. New model deployment takes ~20 minutes. You can check the progress at [Vertex Online Prediction](https://console.cloud.google.com/vertex-ai/online-prediction/endpoints).\n",
|
||||
"3. After the model deployment is complete, restart the playground in Colab to see the updated endpoint list.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"css = \"\"\"\n",
|
||||
".gradio-container {\n",
|
||||
" width: 85% !important\n",
|
||||
"}\n",
|
||||
"\"\"\"\n",
|
||||
"with gr.Blocks(\n",
|
||||
" theme=gr.themes.Default(primary_hue=\"orange\", secondary_hue=\"blue\")\n",
|
||||
" css=css, theme=gr.themes.Default(primary_hue=\"orange\", secondary_hue=\"blue\")\n",
|
||||
") as demo:\n",
|
||||
" gr.Markdown(\"# Stable Diffusion Playground \")\n",
|
||||
" gr.Markdown(\"# Model Garden Playground for Image Generation\")\n",
|
||||
"\n",
|
||||
" with gr.Tab(\"Tasks\"):\n",
|
||||
" interfaces_box = gr.Radio(\n",
|
||||
" show_label=False,\n",
|
||||
" choices=[\n",
|
||||
" \"Text2Image pipeline\",\n",
|
||||
" \"Inpaint Pipeline\",\n",
|
||||
" \"Instruct pix2pix Pipeline\",\n",
|
||||
" \"SD 4x Upscaler Pipeline\",\n",
|
||||
" \"Text2Image\",\n",
|
||||
" \"Inpainting\",\n",
|
||||
" \"Instruct pix2pix\",\n",
|
||||
" \"ControlNet Canny\",\n",
|
||||
" \"SD 4x Upscaler\",\n",
|
||||
" ],\n",
|
||||
" value=\"Text2Image pipeline\",\n",
|
||||
" value=\"Text2Image\",\n",
|
||||
" )\n",
|
||||
" tip = gr.Markdown(tip_text)\n",
|
||||
"\n",
|
||||
" with gr.Row(equal_height=True):\n",
|
||||
" with gr.Column(scale=3):\n",
|
||||
@@ -576,18 +701,19 @@
|
||||
" scale=7,\n",
|
||||
" label=\"Deploy a new model to Vertex\",\n",
|
||||
" choices=[\n",
|
||||
" \"runwayml/stable-diffusion-v1-5\",\n",
|
||||
" \"stabilityai/stable-diffusion-2-1\",\n",
|
||||
" \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" \"stabilityai/stable-diffusion-xl-base-1.0 - refiner\",\n",
|
||||
" \"latent-consistency/lcm-sdxl\",\n",
|
||||
" \"latent-consistency/lcm-lora-sdxl\",\n",
|
||||
" \"stabilityai/sdxl-turbo\",\n",
|
||||
" \"runwayml/stable-diffusion-inpainting\",\n",
|
||||
" \"kandinsky-community/kandinsky-2-2-decoder-inpaint\",\n",
|
||||
" \"diffusers/stable-diffusion-xl-1.0-inpainting-0.1\",\n",
|
||||
" \"timbrooks/instruct-pix2pix\",\n",
|
||||
" \"stabilityai/stable-diffusion-x4-upscaler\",\n",
|
||||
" \"txt2img: runwayml/stable-diffusion-v1-5\",\n",
|
||||
" \"txt2img: stabilityai/stable-diffusion-2-1\",\n",
|
||||
" \"txt2img: stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" \"txt2img: stabilityai/stable-diffusion-xl-base-1.0 - refiner\",\n",
|
||||
" \"txt2img: latent-consistency/lcm-sdxl\",\n",
|
||||
" \"txt2img: latent-consistency/lcm-lora-sdxl\",\n",
|
||||
" \"txt2img: stabilityai/sdxl-turbo\",\n",
|
||||
" \"inpainting: runwayml/stable-diffusion-inpainting\",\n",
|
||||
" \"inpainting: kandinsky-community/kandinsky-2-2-decoder-inpaint\",\n",
|
||||
" \"inpainting: diffusers/stable-diffusion-xl-1.0-inpainting-0.1\",\n",
|
||||
" \"instruct-pix2pix: timbrooks/instruct-pix2pix\",\n",
|
||||
" \"controlnet: lllyasviel/sd-controlnet-canny\",\n",
|
||||
" \"upscaler: stabilityai/stable-diffusion-x4-upscaler\",\n",
|
||||
" ],\n",
|
||||
" value=None,\n",
|
||||
" )\n",
|
||||
@@ -604,6 +730,9 @@
|
||||
" instruct_pix2pix_generate_button = gr.Button(\n",
|
||||
" \"Generate\", variant=\"primary\", visible=False\n",
|
||||
" )\n",
|
||||
" controlnet_generate_button = gr.Button(\n",
|
||||
" \"Generate\", variant=\"primary\", visible=False\n",
|
||||
" )\n",
|
||||
" upscaler_generate_button = gr.Button(\n",
|
||||
" \"Generate\", variant=\"primary\", visible=False\n",
|
||||
" )\n",
|
||||
@@ -611,11 +740,8 @@
|
||||
" num_samples = gr.Slider(\n",
|
||||
" label=\"Number of samples\", value=1, step=1, minimum=1, maximum=4\n",
|
||||
" )\n",
|
||||
" height = gr.Slider(\n",
|
||||
" label=\"Height\", value=768, step=256, minimum=512, maximum=1024\n",
|
||||
" )\n",
|
||||
" width = gr.Slider(\n",
|
||||
" label=\"Width\", value=768, step=256, minimum=512, maximum=1024\n",
|
||||
" image_dimension = gr.Slider(\n",
|
||||
" label=\"Image Dimension\", value=768, step=256, minimum=512, maximum=1024\n",
|
||||
" )\n",
|
||||
" num_inference_steps = gr.Slider(\n",
|
||||
" label=\"Sampling steps\", value=25, step=1, minimum=1, maximum=100\n",
|
||||
@@ -623,33 +749,53 @@
|
||||
" guidance_scale = gr.Slider(\n",
|
||||
" label=\"Guidance scale\", value=7.5, step=0.5, minimum=0, maximum=20.0\n",
|
||||
" )\n",
|
||||
" canny_low_threshold = gr.Slider(\n",
|
||||
" label=\"Canny low threshold\",\n",
|
||||
" value=100,\n",
|
||||
" step=5,\n",
|
||||
" minimum=1,\n",
|
||||
" maximum=255,\n",
|
||||
" visible=False,\n",
|
||||
" )\n",
|
||||
" canny_high_threshold = gr.Slider(\n",
|
||||
" label=\"Canny high threshold\",\n",
|
||||
" value=200,\n",
|
||||
" step=5,\n",
|
||||
" minimum=1,\n",
|
||||
" maximum=255,\n",
|
||||
" visible=False,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" with gr.Column(scale=3):\n",
|
||||
" with gr.Row(equal_height=True):\n",
|
||||
" image_input = gr.Image(\n",
|
||||
" source=\"upload\",\n",
|
||||
" show_label=True,\n",
|
||||
" tool=\"sketch\",\n",
|
||||
" type=\"pil\",\n",
|
||||
" label=\"Upload\",\n",
|
||||
" visible=False,\n",
|
||||
" height=400,\n",
|
||||
" height=500,\n",
|
||||
" )\n",
|
||||
" image_output = gr.Gallery(\n",
|
||||
" show_label=False, rows=1, height=400, preview=True\n",
|
||||
" label=\"Generated Images\",\n",
|
||||
" rows=1,\n",
|
||||
" height=500,\n",
|
||||
" preview=True,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" interfaces_box.change(\n",
|
||||
" select_interface,\n",
|
||||
" interfaces_box,\n",
|
||||
" [\n",
|
||||
" endpoint_name,\n",
|
||||
" prompt,\n",
|
||||
" negative_prompt,\n",
|
||||
" image_input,\n",
|
||||
" image_output,\n",
|
||||
" generate_button,\n",
|
||||
" inpaint_generate_button,\n",
|
||||
" instruct_pix2pix_generate_button,\n",
|
||||
" controlnet_generate_button,\n",
|
||||
" upscaler_generate_button,\n",
|
||||
" canny_low_threshold,\n",
|
||||
" canny_high_threshold,\n",
|
||||
" ],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
@@ -659,8 +805,7 @@
|
||||
" [\n",
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" height,\n",
|
||||
" width,\n",
|
||||
" image_dimension,\n",
|
||||
" ],\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
@@ -679,8 +824,7 @@
|
||||
" num_samples,\n",
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" height,\n",
|
||||
" width,\n",
|
||||
" image_dimension,\n",
|
||||
" ],\n",
|
||||
" outputs=image_output,\n",
|
||||
" )\n",
|
||||
@@ -695,8 +839,7 @@
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" image_input,\n",
|
||||
" height,\n",
|
||||
" width,\n",
|
||||
" image_dimension,\n",
|
||||
" ],\n",
|
||||
" outputs=image_output,\n",
|
||||
" )\n",
|
||||
@@ -711,8 +854,24 @@
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" image_input,\n",
|
||||
" height,\n",
|
||||
" width,\n",
|
||||
" image_dimension,\n",
|
||||
" ],\n",
|
||||
" outputs=image_output,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" controlnet_generate_button.click(\n",
|
||||
" controlnet_generate_images,\n",
|
||||
" inputs=[\n",
|
||||
" endpoint_name,\n",
|
||||
" prompt,\n",
|
||||
" negative_prompt,\n",
|
||||
" num_samples,\n",
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" image_input,\n",
|
||||
" image_dimension,\n",
|
||||
" canny_low_threshold,\n",
|
||||
" canny_high_threshold,\n",
|
||||
" ],\n",
|
||||
" outputs=image_output,\n",
|
||||
" )\n",
|
||||
@@ -727,12 +886,12 @@
|
||||
" guidance_scale,\n",
|
||||
" num_inference_steps,\n",
|
||||
" image_input,\n",
|
||||
" height,\n",
|
||||
" width,\n",
|
||||
" image_dimension,\n",
|
||||
" ],\n",
|
||||
" outputs=image_output,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"demo.queue(concurrency_count=5, max_size=10)\n",
|
||||
"demo.launch(share=True, inline=False, inbrowser=True, debug=True, show_error=True)"
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user