Compare commits

...
Author SHA1 Message Date
Andrew Ferlitsch 5a8ea9ca4a fix: remove obsoleted case 2022-09-12 19:36:52 +00:00
Andrew Ferlitsch 9919f2d4e1 fix: remove obsoleted case 2022-09-12 19:34:45 +00:00
@@ -81,7 +81,6 @@
"The steps performed include:\n",
"\n",
"- Define and compile a `Vertex AI` pipeline.\n",
"- Schedule a recurring pipeline run.\n",
"- Specify which service account to use for a pipeline run."
]
},
@@ -97,13 +96,9 @@
"\n",
"* Vertex AI\n",
"* Cloud Storage\n",
"* Cloud Functions\n",
"* Cloud Scheduler\n",
"\n",
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
"[Cloud Storage pricing](https://cloud.google.com/storage/pricing),\n",
"[Cloud Functions pricing](ttps://cloud.google.com/functions/pricing), and\n",
"[Clould Scheduler pricing]((https://cloud.google.com/scheduler/pricing)),\n",
"and use the [Pricing\n",
"Calculator](https://cloud.google.com/products/calculator/)\n",
"to generate a cost estimate based on your projected usage."
@@ -926,69 +921,6 @@
"job.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "schedule_pipeline_run"
},
"source": [
"## Recurring pipeline runs: create a scheduled pipeline job\n",
"\n",
"This section shows how to create a **scheduled pipeline job**. You do this using the pipeline you already defined.\n",
"\n",
"Under the hood, the scheduled jobs are supported by the Cloud Scheduler and a Cloud Functions function. Check first that the APIs for both of these services are enabled.\n",
"You will need to first enable the [enable the Cloud Scheduler API](http://console.cloud.google.com/apis/library/cloudscheduler.googleapis.com) and the [Cloud Functions and Cloud Build APIs](https://console.cloud.google.com/flows/enableapi?apiid=cloudfunctions,cloudbuild.googleapis.com) if you have not already done so.\n",
"Note:you need to [create an App Engine app for your project](https://cloud.google.com/scheduler/docs/quickstart) if one does not already exist.\n",
"\n",
"\n",
"See the [Cloud Scheduler](https://cloud.google.com/scheduler/docs/configuring/cron-job-schedules) documentation for more on the cron syntax.\n",
"\n",
"Create a scheduled pipeline job, passing as an argument the job specification file that you compiled above.\n",
"\n",
"*Note:* You can pass a `parameter_values` dict that specifies the pipeline input parameters you want to use."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ty5hDoNX2Ou8"
},
"outputs": [],
"source": [
"if not os.getenv(\"IS_TESTING\"):\n",
" from kfp.v2.google.client import AIPlatformClient # noqa: F811\n",
"\n",
" api_client = AIPlatformClient(project_id=PROJECT_ID, region=REGION)\n",
"\n",
" # adjust time zone and cron schedule as necessary\n",
" response = api_client.create_schedule_from_job_spec(\n",
" job_spec_path=\"intro_pipeline.json\",\n",
" schedule=\"2 * * * *\",\n",
" time_zone=\"America/Los_Angeles\", # change this as necessary\n",
" parameter_values={\"text\": \"Hello world!\"},\n",
" # pipeline_root=PIPELINE_ROOT # this argument is necessary if you did not specify PIPELINE_ROOT as part of the pipeline definition.\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J8AP1viy2Ou8"
},
"source": [
"Once the scheduled job is created, you can see it listed in the [Cloud Scheduler](https://console.cloud.google.com/cloudscheduler/) panel in the Console.\n",
"\n",
"<a href=\"https://storage.googleapis.com/amy-jo/images/kf-pls/pipelines_scheduler.png\" target=\"_blank\"><img src=\"https://storage.googleapis.com/amy-jo/images/kf-pls/pipelines_scheduler.png\" width=\"95%\"/></a>\n",
"\n",
"You can test the setup from the Cloud Scheduler panel by clicking 'RUN NOW'.\n",
"\n",
"> **Note**: The implementation is using a Cloud Functions function, which you can see listed in the [Cloud Functions](https://console.cloud.google.com/functions/list) panel in the console as `templated_http_request-v1`.\n",
"Don't delete this function, as it will prevent the Cloud Scheduler jobs from actually kicking off the pipeline run. If you do delete it, create a new scheduled job in order to recreate the function.\n",
"\n",
"When you're done experimenting, you probably want to **PAUSE** your scheduled job from the Cloud Scheduler panel, so that the recurrent jobs do not keep running."
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -1222,7 +1154,7 @@
"outputs": [],
"source": [
"delete_pipeline = True\n",
"delete_bucket = True\n",
"delete_bucket = False\n",
"\n",
"try:\n",
" if delete_pipeline and \"DISPLAY_NAME\" in globals():\n",