xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated notebook comment for custom vllm container image (#4385)

Browse Source 
* updated comment for custom vllm container image

* Update notebooks/official/prediction/vertexai_serving_vllm/vertexai_serving_vllm_cpu_llama3_2_3B.ipynb

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
Ravi Dalal
2025-12-05 20:26:32 +00:00
committed by GitHub
co-authored by gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent 820c0f8db4
commit 6a83c4c695
4 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/official/prediction/vertexai_serving_vllm/vertexai_serving_vllm_cpu_llama3_2_3B.ipynb
+1 -1
View File
@@ -232,7 +232,7 @@
"source": [
"## Create vLLM Customer Container Image for Vertex AI\n",
"\n",
"Vertex AI requires [requests](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#inference) and [responses](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements) in specific formats. vLLM API server implements OpenAI API protocol and therefore, it does not support the Vertex AI request and response requirements. Therefore, the vLLM API server (vllm.entrypoints.openai.api_server.py) needs to be updated to support Vertex AI request and response formats."
"The vLLM custom container image requires gcloud SDK so that it can download the model from Google Cloud Storage when Hugging Face token is not provided. The `entrypoint` script has also been updated to enable this feature."
]
},
{
notebooks/official/prediction/vertexai_serving_vllm/vertexai_serving_vllm_gpu_llama3_2_3B.ipynb
+1 -1
View File
@@ -232,7 +232,7 @@
"source": [
"## Create vLLM Customer Container Image for Vertex AI\n",
"\n",
"Vertex AI requires [requests](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#inference) and [responses](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements) in specific formats. vLLM API server implements OpenAI API protocol and therefore, it does not support the Vertex AI request and response requirements. Therefore, the vLLM API server (vllm.entrypoints.openai.api_server.py) needs to be updated to support Vertex AI request and response formats."
"The vLLM custom container image requires gcloud SDK so that it can download the model from Google Cloud Storage when Hugging Face token is not provided. The `entrypoint` script has also been updated to enable this feature."
]
},
{
notebooks/official/prediction/vertexai_serving_vllm/vertexai_serving_vllm_tpu_gcs_llama3_2_3B.ipynb
+1 -1
View File
@@ -372,7 +372,7 @@
"source": [
"## Create vLLM Customer Container Image for Vertex AI\n",
"\n",
"Vertex AI requires [requests](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#inference) and [responses](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements) in specific formats. vLLM API server implements OpenAI API protocol and therefore, it does not support the Vertex AI request and response requirements. Therefore, the vLLM API server (vllm.entrypoints.openai.api_server.py) needs to be updated to support Vertex AI request and response formats."
"The vLLM custom container image requires gcloud SDK so that it can download the model from Google Cloud Storage when Hugging Face token is not provided. The `entrypoint` script has also been updated to enable this feature."
]
},
{
notebooks/official/prediction/vertexai_serving_vllm/vertexai_serving_vllm_tpu_llama3_2_3B.ipynb
+1 -1
View File
@@ -232,7 +232,7 @@
"source": [
"## Create vLLM Customer Container Image for Vertex AI\n",
"\n",
"Vertex AI requires [requests](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#inference) and [responses](https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements) in specific formats. vLLM API server implements OpenAI API protocol and therefore, it does not support the Vertex AI request and response requirements. Therefore, the vLLM API server (vllm.entrypoints.openai.api_server.py) needs to be updated to support Vertex AI request and response formats."
"The vLLM custom container image requires gcloud SDK so that it can download the model from Google Cloud Storage when Hugging Face token is not provided. The `entrypoint` script has also been updated to enable this feature."
]
},
{