xiaohei/model_garden
1
0
Fork 0
mirror of https://github.com/GoogleCloudPlatform/vertex-ai-samples.git synced 2026-09-26 14:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update Gemma deployment and finetuning notebook descriptions. (#2733)

Browse Source
This commit is contained in:
Kathy Yu
2024-02-21 14:11:52 +00:00
committed by GitHub
parent 8e7e5138a7
commit 97a677c08d
2 changed files with 7 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/community/model_garden/model_garden_gemma_deployment_on_vertex.ipynb
+6 -2
View File
@@ -60,7 +60,11 @@
"source": [
"## Overview\n",
"\n",
"This notebook demonstrates deploying Gemma models using Hex-LLM on TPU and using [vLLM](https://github.com/vllm-project/vllm) on GPU. This notebook also showcases how to use the [Text moderation API](https://cloud.google.com/natural-language/docs/moderating-text) to analyze model predictions against a predefined list of safety attributes.\n",
"This notebook demonstrates deploying Gemma models\n",
" * on TPU using **Hex-LLM**, a **H**igh-**E**fficiency **L**arge **L**anguage **M**odel serving solution built with **XLA** that is being developed by Google Cloud, and\n",
" * on GPU using [vLLM](https://github.com/vllm-project/vllm), the state-of-the-art open source LLM serving solution on GPU.\n",
"\n",
"This notebook also showcases how to use the [Text moderation API](https://cloud.google.com/natural-language/docs/moderating-text) to analyze model predictions against a predefined list of safety attributes.\n",
"\n",
"\n",
"### Objective\n",
@@ -488,7 +492,7 @@
"source": [
"## Deploy Gemma models with Hex-LLM on TPU and apply Google Cloud Text Moderation\n",
"\n",
"**Hex-LLM** is a **H**igh-**E**fficiency **L**arge **L**anguage **M**odel (LLM) TPU serving solution built with **XLA**.\n",
"**Hex-LLM** is a **H**igh-**E**fficiency **L**arge **L**anguage **M**odel (LLM) TPU serving solution built with **XLA**, which is being developed by Google Cloud.\n",
"\n",
"To request TPU quota, please follow the instructions at [Request a higher quota](https://cloud.google.com/docs/quota/view-manage#requesting_higher_quota)."
]
notebooks/community/model_garden/model_garden_gemma_finetuning_on_vertex.ipynb
+1 -1
View File
@@ -866,7 +866,7 @@
"source": [
"# @title Deploy\n",
"# @markdown This section uploads the model to Model Registry and deploys it on the Endpoint. It takes 15 minutes to 1 hour to finish.\n",
"# @markdown **Hex-LLM** is a **H**igh-**E**fficiency **L**arge **L**anguage **M**odel (LLM) TPU serving solution built with **XLA**. This notebook uses TPU v5e 8 cores. Click `Show code` to see more details.\n",
"# @markdown **Hex-LLM** is a **H**igh-**E**fficiency **L**arge **L**anguage **M**odel (LLM) TPU serving solution built with **XLA**, which is being developed by Google Cloud. This notebook uses TPU v5e machines. Click `Show code` to see more details.\n",
"\n",
"if \"2b\" in model_type:\n",
" # Sets ct5lp-hightpu-1t (1 TPU chip) to deploy Gemma 2B models.\n",