mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Add a new notebook for sdxl LCM (#2582)
* Add SDXL-turbo as a separate section in the SDXL notebook. * Add SDXL-turbo as a separate section in the SDXL notebook. * Add a new notebook for sdxl LCM
This commit is contained in:
weigary
@@ -0,0 +1,638 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "7d9bbf86da5e"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2023 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "99c1c3fc2ca5"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI Model Garden - Stable Diffusion XL 1.0 - Latent Consistent Model\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_xl_lcm.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_xl_lcm.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
|
||||
" View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td>\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/notebooks/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/model_garden/model_garden_pytorch_stable_diffusion_xl_lcm.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
|
||||
"Open in Vertex AI Workbench\n",
|
||||
" </a>\n",
|
||||
" (a Python-3 GPU notebook with preinstalled HuggingFace/transformer libraries is recommended)\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3de7470326a2"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook demonstrates running local inference for both the [Latent Consistent Model (LCM)](https://huggingface.co/docs/diffusers/main/en/using-diffusers/inference_with_lcm) and [LCM-LoRA](https://huggingface.co/docs/diffusers/main/en/using-diffusers/inference_with_lcm_lora) based on the [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) models in Colab and deploying the model on Vertex AI for online prediction.\n",
|
||||
"\n",
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"- Run local predictions for `text-to-image` with LCM and LCM-LoRA.\n",
|
||||
"- Deploy the model to a [Vertex AI Endpoint resource](https://cloud.google.com/vertex-ai/docs/predictions/using-private-endpoints).\n",
|
||||
"- Run online predictions for text-to-image.\n",
|
||||
"\n",
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"* Cloud Storage\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "264c07757582"
|
||||
},
|
||||
"source": [
|
||||
"## Before you begin\n",
|
||||
"\n",
|
||||
"**NOTE**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ioensNKM8ned"
|
||||
},
|
||||
"source": [
|
||||
"### Setup notebook"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d73ffa0c0b83"
|
||||
},
|
||||
"source": [
|
||||
"#### Colab\n",
|
||||
"Run the following commands for Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "2707b02ef5df"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if \"google.colab\" in str(get_ipython()):\n",
|
||||
" ! pip3 install --upgrade google-cloud-aiplatform\n",
|
||||
" from google.colab import auth as google_auth\n",
|
||||
"\n",
|
||||
" google_auth.authenticate_user()\n",
|
||||
"\n",
|
||||
"! pip3 install --upgrade pip\n",
|
||||
"! pip3 install --upgrade torch==2.1.0+cu121 -f https://download.pytorch.org/whl/torch_stable.html\n",
|
||||
"! pip3 install transformers==4.35.0\n",
|
||||
"! pip3 install peft==0.6.2\n",
|
||||
"! pip3 install diffusers==0.23.1\n",
|
||||
"! pip3 install accelerate==0.24.1\n",
|
||||
"! pip3 install safetensors\n",
|
||||
"# Remove wrong cublas version.\n",
|
||||
"! pip3 uninstall nvidia_cublas_cu11 --yes\n",
|
||||
"\n",
|
||||
"# Restart the notebook kernel after installs.\n",
|
||||
"import IPython\n",
|
||||
"\n",
|
||||
"app = IPython.Application.instance()\n",
|
||||
"app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "bb7adab99e41"
|
||||
},
|
||||
"source": [
|
||||
"### Setup Google Cloud project\n",
|
||||
"\n",
|
||||
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
||||
"\n",
|
||||
"1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
||||
"\n",
|
||||
"1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n",
|
||||
"\n",
|
||||
"1. [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs.\n",
|
||||
"\n",
|
||||
"1. [Create a service account](https://cloud.google.com/iam/docs/service-accounts-create#iam-service-accounts-create-console) with `Vertex AI User` and `Storage Object Admin` roles for deploying models to Vertex AI endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6c460088b873"
|
||||
},
|
||||
"source": [
|
||||
"Set following variables for experiments environment:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "855d6b96f291"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Cloud project id.\n",
|
||||
"PROJECT_ID = \"\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# The region you want to launch jobs in, such as \"us-central1\"\n",
|
||||
"REGION = \"\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# The Cloud Storage bucket for storing experiments output. Fill it without the 'gs://' prefix.\n",
|
||||
"GCS_BUCKET = \"\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"# The service account for deploying fine tuned model. Something like: \"your-name-test@{PROJECT_ID}.iam.gserviceaccount.com\"\n",
|
||||
"SERVICE_ACCOUNT = \"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e828eb320337"
|
||||
},
|
||||
"source": [
|
||||
"Initialize Vertex AI API:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "12cd25839741"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from google.cloud import aiplatform\n",
|
||||
"\n",
|
||||
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=GCS_BUCKET)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2cc825514deb"
|
||||
},
|
||||
"source": [
|
||||
"### Define constants"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "b42bd4fa2b2d"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The pre-built serving docker image. It contains serving scripts and models.\n",
|
||||
"SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-diffusers-serve-opt:latest\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0c250872074f"
|
||||
},
|
||||
"source": [
|
||||
"### Define common functions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "354da31189dc"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import base64\n",
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"from google.cloud import aiplatform\n",
|
||||
"from PIL import Image\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def base64_to_image(image_str):\n",
|
||||
" \"\"\"Convert base64 encoded string to an image.\"\"\"\n",
|
||||
" image = Image.open(BytesIO(base64.b64decode(image_str)))\n",
|
||||
" return image\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def image_grid(imgs, rows=2, cols=2):\n",
|
||||
" \"\"\"Arrange and display multiple images in a grid.\"\"\"\n",
|
||||
" w, h = imgs[0].size\n",
|
||||
" grid = Image.new(\"RGB\", size=(cols * w, rows * h))\n",
|
||||
" for i, img in enumerate(imgs):\n",
|
||||
" grid.paste(img, box=(i % cols * w, i // cols * h))\n",
|
||||
" return grid\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def deploy_model(model_id, task, refiner_model_id=\"none\"):\n",
|
||||
" \"\"\"Create a Vertex AI Endpoint and deploy the specified model to the endpoint.\"\"\"\n",
|
||||
" model_name = \"stable-diffusion\"\n",
|
||||
" endpoint = aiplatform.Endpoint.create(display_name=f\"{model_name}-{task}-endpoint\")\n",
|
||||
" serving_env = {\n",
|
||||
" \"MODEL_ID\": model_id,\n",
|
||||
" \"TASK\": task,\n",
|
||||
" }\n",
|
||||
" model = aiplatform.Model.upload(\n",
|
||||
" display_name=model_name,\n",
|
||||
" serving_container_image_uri=SERVE_DOCKER_URI,\n",
|
||||
" serving_container_ports=[7080],\n",
|
||||
" serving_container_predict_route=\"/predictions/diffusers_serving\",\n",
|
||||
" serving_container_health_route=\"/ping\",\n",
|
||||
" serving_container_environment_variables=serving_env,\n",
|
||||
" )\n",
|
||||
" machine_type = \"g2-standard-8\"\n",
|
||||
" accelerator_type = \"NVIDIA_L4\"\n",
|
||||
"\n",
|
||||
" model.deploy(\n",
|
||||
" endpoint=endpoint,\n",
|
||||
" machine_type=machine_type,\n",
|
||||
" accelerator_type=accelerator_type,\n",
|
||||
" accelerator_count=1,\n",
|
||||
" deploy_request_timeout=1800,\n",
|
||||
" service_account=SERVICE_ACCOUNT,\n",
|
||||
" )\n",
|
||||
" return model, endpoint"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e8a42fa49305"
|
||||
},
|
||||
"source": [
|
||||
"## Run inferences locally"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1d5ebc91c786"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image with LCM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "d39ed8c97cc5"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"from diffusers import (LCMScheduler, StableDiffusionXLPipeline,\n",
|
||||
" UNet2DConditionModel)\n",
|
||||
"from diffusers.models.attention_processor import AttnProcessor2_0\n",
|
||||
"\n",
|
||||
"torch.cuda.empty_cache()\n",
|
||||
"\n",
|
||||
"unet = UNet2DConditionModel.from_pretrained(\n",
|
||||
" \"latent-consistency/lcm-sdxl\",\n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
" variant=\"fp16\",\n",
|
||||
")\n",
|
||||
"pipe = StableDiffusionXLPipeline.from_pretrained(\n",
|
||||
" \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" unet=unet,\n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
" variant=\"fp16\",\n",
|
||||
").to(\"cuda\")\n",
|
||||
"pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)\n",
|
||||
"\n",
|
||||
"pipe.upcast_vae()\n",
|
||||
"pipe.unet.set_attn_processor(AttnProcessor2_0())\n",
|
||||
"pipe.unet.to(memory_format=torch.channels_last)\n",
|
||||
"\n",
|
||||
"prompt = \"a deer in Matisse style\"\n",
|
||||
"\n",
|
||||
"# We set `num_inference_steps` as 8 for a good image, feel free to use a different number of steps to match your needs.\n",
|
||||
"# `guidance_scale` needs to be set as 0.\n",
|
||||
"generator = torch.manual_seed(21)\n",
|
||||
"image = pipe(\n",
|
||||
" prompt=prompt,\n",
|
||||
" num_inference_steps=8,\n",
|
||||
" generator=generator,\n",
|
||||
" guidance_scale=0.0,\n",
|
||||
").images[0]\n",
|
||||
"display(image)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xoi3jgmfHFry"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image with LCM-LoRA"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "B9A1SC3YHWO-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"from diffusers import DiffusionPipeline, LCMScheduler\n",
|
||||
"from diffusers.models.attention_processor import AttnProcessor2_0\n",
|
||||
"\n",
|
||||
"torch.cuda.empty_cache()\n",
|
||||
"\n",
|
||||
"pipe = DiffusionPipeline.from_pretrained(\n",
|
||||
" \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
" use_safetensors=True,\n",
|
||||
" variant=\"fp16\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# set scheduler\n",
|
||||
"pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)\n",
|
||||
"pipe.to(\"cuda\")\n",
|
||||
"\n",
|
||||
"pipe.upcast_vae()\n",
|
||||
"pipe.unet.set_attn_processor(AttnProcessor2_0())\n",
|
||||
"pipe.unet.to(memory_format=torch.channels_last)\n",
|
||||
"\n",
|
||||
"# load LCM-LoRA\n",
|
||||
"pipe.load_lora_weights(\"latent-consistency/lcm-lora-sdxl\")\n",
|
||||
"\n",
|
||||
"prompt = \"a deer in Matisse style\"\n",
|
||||
"\n",
|
||||
"# We set `num_inference_steps` as 8 for a good image, feel free to use a different number of steps to match your needs.\n",
|
||||
"# `guidance_scale` needs to be set as 0.\n",
|
||||
"generator = torch.manual_seed(21)\n",
|
||||
"image = pipe(\n",
|
||||
" prompt=prompt,\n",
|
||||
" num_inference_steps=8,\n",
|
||||
" generator=generator,\n",
|
||||
" guidance_scale=0.0,\n",
|
||||
").images[0]\n",
|
||||
"display(image)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "bf7f82732e61"
|
||||
},
|
||||
"source": [
|
||||
"## Upload and Deploy models"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1cc26e68d7b0"
|
||||
},
|
||||
"source": [
|
||||
"This section uploads the model to Model Registry and deploys it to a Vertex AI Endpoint resource.\n",
|
||||
"\n",
|
||||
"The model deployment step will take ~20 minutes to complete."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cd7b56421392"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image with LCM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6d331b1ea337"
|
||||
},
|
||||
"source": [
|
||||
"Deploy the stable diffusion XL LCM model for the text-to-image task.\n",
|
||||
"\n",
|
||||
"Once deployed, you can send a batch of text prompts to the endpoint to generated images.\n",
|
||||
"\n",
|
||||
"When deployed on one L4 GPU, the averaged inference time of a request is ~2-3 seconds."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "bf55e38815dc"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Set the model_id to \"stabilityai/stable-diffusion-xl-base-1.0\" to load the OSS pre-trained model.\n",
|
||||
"model, endpoint = deploy_model(\n",
|
||||
" model_id=\"stabilityai/stable-diffusion-xl-base-1.0\", task=\"text-to-image-sdxl-lcm\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "80b3fd2ace09"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus upto 10 minutes of additional waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4ab04da3ec9a"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"instances = [\n",
|
||||
" {\n",
|
||||
" \"prompt\": \"Photorealistic whale swimming in abyss\",\n",
|
||||
" \"height\": 1024,\n",
|
||||
" \"width\": 1024,\n",
|
||||
" \"num_inference_steps\": 8,\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"prompt\": \"Photorealistic happy dog running\",\n",
|
||||
" \"height\": 1024,\n",
|
||||
" \"width\": 1024,\n",
|
||||
" \"num_inference_steps\": 8,\n",
|
||||
" },\n",
|
||||
"]\n",
|
||||
"response = endpoint.predict(instances=instances)\n",
|
||||
"images = [base64_to_image(image) for image in response.predictions]\n",
|
||||
"image_grid(images, rows=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "af21a3cff1e0"
|
||||
},
|
||||
"source": [
|
||||
"Clean up resources:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "911406c1561e"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Undeploy model and delete endpoint.\n",
|
||||
"endpoint.delete(force=True)\n",
|
||||
"\n",
|
||||
"# Delete models.\n",
|
||||
"model.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "c1e51f764a60"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image with LCM-LoRA"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "fa686a54047c"
|
||||
},
|
||||
"source": [
|
||||
"Deploy the base stable diffusion models, with LCM-LoRA.\n",
|
||||
"\n",
|
||||
"When deployed on one L4 GPU, the averaged inference time of a request is ~3 seconds."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "65e32356fbd1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Set the model_id to \"stabilityai/stable-diffusion-xl-base-1.0\".\n",
|
||||
"model, endpoint = deploy_model(\n",
|
||||
" model_id=\"stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" task=\"text-to-image-sdxl-lcm-lora\",\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "SsjYFLvNymc0"
|
||||
},
|
||||
"source": [
|
||||
"NOTE: The model weights will be downloaded after the deployment succeeds. Thus additional **5 minutes** of waiting time is needed **after** the above model deployment step succeeds and before you run the next step below. Otherwise you might see a `ServiceUnavailable: 503 502:Bad Gateway` error when you send requests to the endpoint.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "83a50fd4a1ed"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"instances = [\n",
|
||||
" {\"prompt\": \"a deer in Matisse style\"},\n",
|
||||
" {\"prompt\": \"a dog in Van Gogh style\"},\n",
|
||||
"]\n",
|
||||
"response = endpoint.predict(instances=instances)\n",
|
||||
"images = [base64_to_image(image) for image in response.predictions]\n",
|
||||
"image_grid(images, rows=1)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ed3795d474b9"
|
||||
},
|
||||
"source": [
|
||||
"Clean up resources:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "b53b883257b4"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Undeploy model and delete endpoint.\n",
|
||||
"endpoint.delete(force=True)\n",
|
||||
"\n",
|
||||
"# Delete models.\n",
|
||||
"model.delete()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"name": "model_garden_pytorch_stable_diffusion_xl_lcm.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Reference in New Issue
Block a user