mirror of
https://github.com/GoogleCloudPlatform/vertex-ai-samples.git
synced 2026-09-26 14:42:04 +00:00
Add notebook for using Deepseek 3.2 model on Vertex AI.
PiperOrigin-RevId: 842763792
This commit is contained in:
Vertex MG Team
committed by
Copybara-Service
Copybara-Service
parent
b075990d88
commit
9bb8107110
@@ -0,0 +1,822 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "ur8xi4C7S06n"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2025 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JAPoU8Sm5E6e"
|
||||
},
|
||||
"source": [
|
||||
"# Vertex AI Model Garden - Get started with DeepSeek-V3.2 models\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_openai_api_deepseek3_2.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fcommunity%2Fmodel_garden%2Fmodel_garden_openai_api_deepseek3_2.ipynb\"\">\n",
|
||||
" <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td> \n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_openai_api_deepseek3_2.ipynb\">\n",
|
||||
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_openai_api_deepseek3_2.ipynb\">\n",
|
||||
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tvgnzT1CKxrO"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook demonstrates how to get started with using the OpenAI library and demonstrates how to use DeepSeek-V3.2 models as Model-as-service (MaaS) for building translation chain and document question-answer.\n",
|
||||
"\n",
|
||||
"### Objective\n",
|
||||
"\n",
|
||||
"- Configure OpenAI SDK for the DeepSeek-V3.2 Completions API\n",
|
||||
"- Chat with DeepSeek-V3.2 models with different prompts and model parameters, and apply Llama Guard for safeguarding\n",
|
||||
"- Build with DeepSeek-V3.2 models\n",
|
||||
" - Translation Chain.\n",
|
||||
"\n",
|
||||
"### Costs\n",
|
||||
"\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"* Vertex AI\n",
|
||||
"* Cloud Storage\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "61RBz8LLbxCR"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "No17Cw5hgx12"
|
||||
},
|
||||
"source": [
|
||||
"### Install Vertex AI SDK for Python and other required packages\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "tFy3H3aPgx12"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! pip3 install --upgrade --quiet google-cloud-aiplatform[langchain] openai\n",
|
||||
"! pip3 install --upgrade --quiet langchain-openai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R5Xep4W9lq-Z"
|
||||
},
|
||||
"source": [
|
||||
"### Restart runtime (Colab only)\n",
|
||||
"\n",
|
||||
"To use the newly installed packages, you must restart the runtime on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "XRvKdaPDTznN"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" import IPython\n",
|
||||
"\n",
|
||||
" app = IPython.Application.instance()\n",
|
||||
" app.kernel.do_shutdown(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "SbmM4z7FOBpM"
|
||||
},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-warning\">\n",
|
||||
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
|
||||
"</div>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dmWOrTJ3gx13"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"Authenticate your environment on Google Colab.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "NyKGtVQjgx13"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
"\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DF4l8DTdWgPY"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information\n",
|
||||
"\n",
|
||||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Nqwi-5ufWp_B"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PROJECT_ID = \"<YOUR PROJECT ID>\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\" # @param {type:\"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jVYoyDl165EE"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries\n",
|
||||
"\n",
|
||||
"Import libraries to use in this tutorial."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "c1tEW-U968h8"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Chat completions API\n",
|
||||
"import openai\n",
|
||||
"from google.auth import default, transport\n",
|
||||
"from langchain import PromptTemplate\n",
|
||||
"# Build\n",
|
||||
"from langchain_openai import ChatOpenAI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uqYCG2Fw7D3L"
|
||||
},
|
||||
"source": [
|
||||
"### Configure OpenAI SDK for the DeepSeek-V3.2 Chat Completions API\n",
|
||||
"\n",
|
||||
"To configure the OpenAI SDK for the DeepSeek-V3.2 Chat Completions API, you need to request the access token and initialize the client pointing to the DeepSeek-V3.2 endpoint.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "W0K6VSJRHhH2"
|
||||
},
|
||||
"source": [
|
||||
"#### Authentication\n",
|
||||
"\n",
|
||||
"You can request an access token from the default credentials for the current environment. Note that the access token lives for [1 hour by default](https://cloud.google.com/docs/authentication/token-types#at-lifetime); after expiration, it must be refreshed.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "i0qceuiQEPHv"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"credentials, _ = default()\n",
|
||||
"auth_request = transport.requests.Request()\n",
|
||||
"credentials.refresh(auth_request)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Q04wJmA0HT6X"
|
||||
},
|
||||
"source": [
|
||||
"Then configure the OpenAI SDK to point to the DeepSeek-V3.2 Chat Completions API endpoint.\n",
|
||||
"\n",
|
||||
"Notice, only `global` is supported region for DeepSeek-V3.2 models using Model-as-a-Service (MaaS)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "c-MRhsnlj6iw"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_LOCATION = \"global\"\n",
|
||||
"\n",
|
||||
"client = openai.OpenAI(\n",
|
||||
" base_url=f\"https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{MODEL_LOCATION}/endpoints/openapi/chat/completions?\",\n",
|
||||
" api_key=credentials.token,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "UGokrtdiIHrX"
|
||||
},
|
||||
"source": [
|
||||
"#### DeepSeek-V3.2 Model\n",
|
||||
"\n",
|
||||
"This tutorial uses DeepSeek-V3.2 using Model-as-a-Service (MaaS). Using Model-as-a-Service (MaaS), you can access DeepSeek-V3.2 model in just a few clicks without any setup or infrastructure hassles. Model-as-a-Service (MaaS) integrates [Llama Guard](https://huggingface.co/meta-llama/Llama-Guard-3-8B) as a safety filter. It is switched on by default and can be switched off. Llama Guard enables us to safeguard model inputs and outputs. If a response is filtered, it will be populated with a `finish_reason` field (with value `content_filtered`) and a `refusal` field (stating the filtering reason)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "r7OhyH46H2H5"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"deepseek-ai/deepseek-v3.2-maas\" # @param {type:\"string\"} [\"deepseek-ai/deepseek-v3.2-maas\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1xD62NTpqHXd"
|
||||
},
|
||||
"source": [
|
||||
"### Chat with DeepSeek-V3.2\n",
|
||||
"\n",
|
||||
"Use the Chat Completions API to send a request to the DeepSeek-V3.2model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tkyp9kZSuJGx"
|
||||
},
|
||||
"source": [
|
||||
"#### Hello, DeepSeek-V3.2!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "CKVOZ1HEqRbY"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"apply_llama_guard = True # @param {type:\"boolean\"}\n",
|
||||
"\n",
|
||||
"response = client.chat.completions.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" messages=[{\"role\": \"user\", \"content\": \"Hello, Deepseek!\"}],\n",
|
||||
" extra_body={\n",
|
||||
" \"extra_body\": {\n",
|
||||
" \"google\": {\n",
|
||||
" \"model_safety_settings\": {\n",
|
||||
" \"enabled\": apply_llama_guard,\n",
|
||||
" \"llama_guard_settings\": {},\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "LxpdxYCxH51u"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response.choices[0].message.content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "B1rKbHUQt605"
|
||||
},
|
||||
"source": [
|
||||
"#### Ask DeepSeek-V3.2 using different model configuration\n",
|
||||
"\n",
|
||||
"Use the following parameters to generate different answers:\n",
|
||||
"\n",
|
||||
"* `temperature` to control the randomness of the response\n",
|
||||
"* `max_tokens` to limit the response length\n",
|
||||
"* `top_p` to control the quality of the response\n",
|
||||
"* `stream` to stream the response back or not\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "owv-5Sz5rIEU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"temperature = 1.0 # @param {type:\"number\"}\n",
|
||||
"max_tokens = 256 # @param {type:\"integer\"}\n",
|
||||
"top_p = 1.0 # @param {type:\"number\"}\n",
|
||||
"stream = True # @param {type:\"boolean\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "a-qBuhcK-G1V"
|
||||
},
|
||||
"source": [
|
||||
"Get the answer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "O1YU8bSivH0B"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"apply_llama_guard = True # @param {type:\"boolean\"}\n",
|
||||
"\n",
|
||||
"response = client.chat.completions.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": \"What is Vertex AI?\"},\n",
|
||||
" {\"role\": \"assistant\", \"content\": \"Sure, Vertex AI is:\"},\n",
|
||||
" ],\n",
|
||||
" temperature=temperature,\n",
|
||||
" max_tokens=max_tokens,\n",
|
||||
" top_p=top_p,\n",
|
||||
" stream=stream,\n",
|
||||
" extra_body={\n",
|
||||
" \"extra_body\": {\n",
|
||||
" \"google\": {\n",
|
||||
" \"model_safety_settings\": {\n",
|
||||
" \"enabled\": apply_llama_guard,\n",
|
||||
" \"llama_guard_settings\": {},\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-o9-gF0U-Kba"
|
||||
},
|
||||
"source": [
|
||||
"Depending if `stream` parameter is enabled or not, you can print the response entirely or chunk by chunk."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "CoDHLGhyyt8d"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if stream:\n",
|
||||
" for chunk in response:\n",
|
||||
" if chunk.choices:\n",
|
||||
" print(chunk.choices[0].delta.content, end=\"\")\n",
|
||||
"else:\n",
|
||||
" print(response.choices[0].message.content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BkoaelaKxm1r"
|
||||
},
|
||||
"source": [
|
||||
"#### Use DeepSeek-V3.2 with different tasks\n",
|
||||
"\n",
|
||||
"In this section, you will use DeepSeek-V3.2 to perform different tasks including text generation, text summarization, and code generation.\n",
|
||||
"\n",
|
||||
"For each task, you'll define a different prompt and submit a request to the model as you did before."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-en7AYQDyONt"
|
||||
},
|
||||
"source": [
|
||||
"##### Text Generation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "QANInNvizWbi"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"Write a poem about a cat who loves to code\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "2x8ML1Y_yfom"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"apply_llama_guard = True # @param {type:\"boolean\"}\n",
|
||||
"\n",
|
||||
"response = client.chat.completions.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" ],\n",
|
||||
" extra_body={\n",
|
||||
" \"extra_body\": {\n",
|
||||
" \"google\": {\n",
|
||||
" \"model_safety_settings\": {\n",
|
||||
" \"enabled\": apply_llama_guard,\n",
|
||||
" \"llama_guard_settings\": {},\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "qQ6UUgpHztXZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response.choices[0].message.content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "kBLESIw4zhto"
|
||||
},
|
||||
"source": [
|
||||
"##### Text summarization"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "UrAybklfzhtz"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"article = \"\"\"\n",
|
||||
"Vertex AI: Google's Unified Platform for Machine Learning\n",
|
||||
"\n",
|
||||
"Google Cloud's Vertex AI is a comprehensive platform that simplifies the process of building, deploying, and managing machine learning (ML) models and AI applications. It provides a single environment for all your AI needs, from data preparation to model deployment and monitoring.\n",
|
||||
"\n",
|
||||
"Vertex AI offers a range of features to cater to various user levels, including:\n",
|
||||
"\n",
|
||||
"AutoML: This feature allows you to train models on tabular, image, text, or video data without writing code. It's ideal for users without extensive ML expertise.\n",
|
||||
"Custom Training: For advanced users, Vertex AI provides custom training options, allowing you to use your preferred ML framework and write your own code.\n",
|
||||
"Model Garden: This feature lets you discover, test, and deploy pre-trained models from Vertex AI and open-source sources.\n",
|
||||
"Generative AI: Access Google's powerful large language models (LLMs) to generate text, code, images, and speech, which can be customized and deployed for your applications.\n",
|
||||
"Vertex AI seamlessly integrates with other Google Cloud services like BigQuery for data warehousing, Cloud Storage for data management, and Cloud AI Platform for custom model training. It provides managed infrastructure that can be tailored to your performance and budget needs.\n",
|
||||
"\n",
|
||||
"Whether you're a seasoned data scientist or just starting out with AI, Vertex AI simplifies the entire ML lifecycle and empowers you to build and deploy AI solutions effectively.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"prompt = (\"Summarize the following article in one sentence: \" + article).replace(\n",
|
||||
" \"\\n\", \"\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "VJYIZbGyzhtz"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"apply_llama_guard = True # @param {type:\"boolean\"}\n",
|
||||
"\n",
|
||||
"response = client.chat.completions.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" ],\n",
|
||||
" extra_body={\n",
|
||||
" \"extra_body\": {\n",
|
||||
" \"google\": {\n",
|
||||
" \"model_safety_settings\": {\n",
|
||||
" \"enabled\": apply_llama_guard,\n",
|
||||
" \"llama_guard_settings\": {},\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "9vfWA2i9zwOZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response.choices[0].message.content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-_mNB0Fezh6G"
|
||||
},
|
||||
"source": [
|
||||
"##### Code generation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "ltbhrGiwzh6H"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"Write a Python function that takes a list of numbers and returns the average. Include error handling for empty lists.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "q66yE4Pszh6H"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"apply_llama_guard = True # @param {type:\"boolean\"}\n",
|
||||
"\n",
|
||||
"response = client.chat.completions.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" ],\n",
|
||||
" extra_body={\n",
|
||||
" \"extra_body\": {\n",
|
||||
" \"google\": {\n",
|
||||
" \"model_safety_settings\": {\n",
|
||||
" \"enabled\": apply_llama_guard,\n",
|
||||
" \"llama_guard_settings\": {},\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "mcAf5tXrtPIu"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response.choices[0].message.content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gnrXpv5Y3yFK"
|
||||
},
|
||||
"source": [
|
||||
"### Build with DeepSeek-V3.2\n",
|
||||
"\n",
|
||||
"In this section, you use DeepSeek-V3.2 to build a translation simple applications.\n",
|
||||
"\n",
|
||||
"**Translation Chain** to translate text across multiple languages using DeepSeek-V3.2 and LangChain Expression Language (LCEL).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "IowGcZq95HqZ"
|
||||
},
|
||||
"source": [
|
||||
"#### Translation chain\n",
|
||||
"\n",
|
||||
"In this scenario, you use LangChain Expression Language (LCEL) to build a simple chain which translates some `text_to_translate` to the specified `target_language`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yLAwaYPzFqDQ"
|
||||
},
|
||||
"source": [
|
||||
"##### Initialize the chat interface and the translation prompt template using LangChain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "CE2KxIrG5xKC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"llm = ChatOpenAI(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" base_url=f\"https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{MODEL_LOCATION}/endpoints/openapi/chat/completions?\",\n",
|
||||
" api_key=credentials.token,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"template = \"\"\"Translate the following {text} to {target_language}:\"\"\"\n",
|
||||
"\n",
|
||||
"prompt = PromptTemplate(input_variables=[\"text\", \"target_language\"], template=template)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "odTLqzLiF8h_"
|
||||
},
|
||||
"source": [
|
||||
"##### Initialize the chain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "74Mywe4W9MmE"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chain = prompt | llm"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QE3sizbzGFER"
|
||||
},
|
||||
"source": [
|
||||
"##### Translate a text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "SRJ-xuSI9ZQl"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"text_to_translate = \"Hello Deepseek!\" # @param {type:\"string\"}\n",
|
||||
"target_language = \"Italian\" # @param {type:\"string\"}\n",
|
||||
"\n",
|
||||
"response = chain.invoke({\"text\": text_to_translate, \"target_language\": target_language})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "form",
|
||||
"id": "yYzc1kCjEHGP"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response.content)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "model_garden_openai_api_deepseek3_2.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Reference in New Issue
Block a user