Commit Graph
1007 Commits
Author SHA1 Message Date
3a5eec64af Vertex AI Experiments - Compare pipeline runs - Notebook review (#854)
* experiments cuj1 review

* linter test passed

* typo

* linter test passed

* add correct library. add IAM roles

* linter test passed

* minor fix

* linter test passed

* add api

* linter test passed

* remove library

* linter test passed

* fix

* linter passed

* check

* linter passed

* add text

* linter passed

* minor changes

* andy reviews

* linter passed

* fix link. fix warnings

* linter passed

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-23 07:57:41 -07:00
8cdc7f1f79 Modified notebook multi_node_ddp_gloo_vertex_training_with_custom_container.ipynb (#886)
* modified notebook according to template

* ran linter

Co-authored-by: gericdong <itseric@google.com>
2022-08-22 10:57:24 -04:00
Ivan CheungandGitHub 84fd10f408 fix: Fixed kernel spec by forcing it to use python3 (#889) 2022-08-19 18:43:07 -07:00
Andrew FerlitschandGitHub 7804c860c5 fix: replace tabs with spaces in JSON template (#888)
* fix: replace tabs with spaces in JSON template

* fix: replace tabs with spaces in JSON template
2022-08-19 11:11:08 -07:00
Andrew FerlitschandGitHub ed0c1c74a6 fix: typo in serving_container_uri parameter (#887) 2022-08-19 08:49:12 -07:00
Andrew FerlitschandGitHub 5127a59e92 Model monitoring (#883)
* fix: notebook template tuning

* fix: notebook template tuning

* fix: possible confusion on when to wait for the email notification

* fix: possible confusion on when to wait for the email notification
2022-08-19 08:38:29 -07:00
d8df732d6b Replace GAPIC with SDK - Model Monitoring Notebook (#879)
* Replace GAPIC with SDK

* Run linter

* Remove unused variables

* Run linter

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-18 11:41:11 -07:00
9c10899db8 AutoML Video Classificaton (#871)
* changes according to andrew review comments

* changes according to andrew review comments

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-18 07:25:14 -07:00
Ivan NardiniandGitHub 80770188ec Vertex AI Experiments - Model training - Notebook review (#853)
* experiments cuj1 review

* linter test passed

* add IAM role

* linter test passed

* add rm local dir

* linter test passed

* add api

* linter test passed

* remove libraries

* linter test passed

* solve build error

* fix issue

* clean

* linter passed

* fix dependency

* linter passed

* add dependency

* linter passed

* add dependency

* linter passed

* delete bucket flag fix

* linter passed
2022-08-17 10:43:34 -04:00
fa91e45018 Adds the updated managed_notebooks/predictive_maintenance notebook to the official folder (#521)
* adds the updated predictive-maintenance (managed)notebook from community to official folder

* ran linter test

* resubmitting during phase2

* ran linter test

* addresses the review comments: updates based on the new template, sets delete_bucket to False

* ran linter test

* replaces timestamp with uuid

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-16 09:39:07 -07:00
fa27309134 Fixes the issues from the regression tests (#822)
* fixes tensorflow version, replaces timestamp with uuid, adds service-account and minor textual changes

* removes second defnition of random library

* ran linter test

* uncomments the user flag and updates the installation command

* ran linter test

* removes the extra backslash

* ran linter test

* updates the installation step to fix long running compatibility checks

* ran linter test

* updates the METADATA path during installation steps

* ran linter test

* adds google-api-core version in the installation

* ran linter test

* removes METADATA step during installation

* ran linter test

* updates google api-core & auth versions

* ran linter test

* fixes tensorflow version, replaces timestamp with uuid, adds service-account and minor textual changes

* removes second defnition of random library

* ran linter test

* uncomments the user flag and updates the installation command

* ran linter test

* removes the extra backslash

* ran linter test

* updates the installation step to fix long running compatibility checks

* ran linter test

* updates the METADATA path during installation steps

* ran linter test

* adds google-api-core version in the installation

* ran linter test

* removes METADATA step during installation

* ran linter test

* updates google api-core & auth versions

* ran linter test

* fixes the issues from review: cell descriptions, parameter definitions, tense changes, 3rd person --> 2nd person, list model after pipeline run

* ran linter test

* removes the METADATA hack and updates the installations

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-16 09:11:30 -07:00
udaypunnaandGitHub 7b85f76384 custom_model_training_and_batch_prediction (#873)
* changed based on andrew review comments

* changed based on andrew review comments

* import library issues

* import library issues

* import issues

* import issues
2022-08-16 08:40:33 -07:00
40a0477b08 modified notebook automl-text-classification.ipynb (#757)
* modified notebook

* modified notebook

* added new notebook

* added new notebook

* new auto_ml_text_classifiation

* new auto_ml_text_classifiation

* new automl text classification

* linter test

* linter test

* changes on andrew comments

* changes on andrew comments

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-15 17:53:52 -07:00
839c7dddb8 Vertex AI Experiments - Compare Models - Notebook review (#852)
* experiments cuj1 review

* linter test passed

* typo

* linter test passed

* add andy reviews

* linter test passed

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-15 09:49:50 -07:00
Andrew FerlitschandGitHub 578cfb7da7 fi: Issue 850 (#863)
* fix: add missing end_run() for experiment

* fix: add missing end_run() for experiment
2022-08-15 08:57:07 -07:00
Ivan CheungandGitHub b129c0bf43 Update CODEOWNERS (#864)
Use proper Github username.
2022-08-12 17:21:39 -04:00
07b4c37135 Automl links fix (#742)
* fixing links to open notebook - main and images

* linter test changes

* fixes the papermill execution error(hard-coded bucket link was the cause)

* ran linter test

* adds minor textual changes

* ran linter test

* fixes issues from review: future tense, copyright year, section placement, latest sdk methods, new updates from the template

* ran linter test

Co-authored-by: Manuel Amunategui <manuel.amunategui@springml.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-12 12:34:53 -07:00
f48fb1c650 Modified file churn_prediction_for_game_developers (#809)
* made changes

* made changes

* ran linter test

* made changes

* ran linter

* made changes

* ran linter

* changes suggested by andrew done

* ran linter

* replaced timestamp with uuid

* ran linter

* changed bucket creation command according to template

* ran linter

* changed text in overview

* changed region cell from markdown to code

* made changes

* replaced dataset from constant to a variable

* replaced constant dataset_id with a variable

* ran linter

* changed suggested by andrew done

* ran linter

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-12 08:40:38 -07:00
3bbd59311c Made UUID Changes to SDK_BigQuery_Custom_Container_Training.ipynb (#849)
* MAde UUID changes

* Ran Linter test

* made some minor changes

* Ran Linter Test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-12 08:08:32 -07:00
Krishna Chaitanya MovvaandGitHub 15bb4cea73 Fixes the issues(missing variable) from regression tests, adds minor updates from the template. (#823)
* adds the missing delete_bucket variable, adds steps to configure SERVICE_ACCOUNT

* removes unnecessary random import

* ran linter test

* removes the src folder dependency to run on Colab, adds the pipeline.wait step, updates the cleanup steps

* ran linter test

* fixed issues from review: section posistions, tense changes, section descriptions, template updates

* ran linter test
2022-08-12 08:07:11 -07:00
Andrew FerlitschandGitHub 1511cc9fd1 fix: branding updates from autoreview (#862)
* autoreview: branding fixes

* fix: improve installation detection

* cleanup: rm tmp file

* fix: lint issues

* fix: 2nd try at lint fixes
2022-08-11 18:32:47 -07:00
2885a7a70f Migrated matching engine notebook to official and added VPC network support to CI (#836)
* Added matching engine notebook official

* Ran linter

* Added matching engine to .cloud-build/test_notebook_vm.txt

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 18:30:22 -07:00
ea36f5c43e Vertex SDK Custom Image Classification with pre-built training container (#833)
* new notebook of custom image classification

* new notebook of custome image classification

* andrew commented changes

* andrew commented changes

* andrew commented changes

* andrew commented changes

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 14:58:49 -07:00
a905a6305f fixed cleanup cell, cosmetic and doc improvements, and removed load test (to avoid flakiness), (#859)
* fix cleanup cell and print out more info on xai prediction test

* lint fix

* remove load test

* lint fixes

* lint fix

* fix andrews comments

* lint fix

* missing newline

* lint fixes

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 14:04:28 -07:00
Ivan CheungandGitHub aeaeddcc75 Update PULL_REQUEST_TEMPLATE.md (#860)
Updated PR template to emphasize need for a summary and checklist. Otherwise, people tend to skip this standard practice.
2022-08-11 16:18:54 -04:00
161965cfb0 Fixes the exception in batch-explain notebook in the official folder (#810)
* fixes exception(reg-test), replaces timestamp with uuid, minor changes

* ran linter test

* resolved review comments: license year, Vertex AI SDK, dataset after objective and future tense

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 10:25:51 -07:00
b9d4457474 Modified notebook forecasting-retail-demand (#827)
* deleted file in community and added file in official folder

* renamed file

* ran linter test

* renamed file

* ran linter

* made changes

* ran linter test

* made changes

* ran linter test

* made changes

* ran linter test

* made changes

* ran linter

* made change

* ran linter test

* changes suggested by andrew done

* ran linter

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 09:31:27 -07:00
a5b6bcfab1 those two files are moved to official folder.Deleting them in community content (#855)
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 09:14:02 -07:00
5d55f5b0d2 Modified notebook pricing-optimization.ipynb (#834)
* modified notebook according to notebook_template.

* ran linter

* Added create dataset step

* ran linter

* replaced hardcoded dataset name with a variable

* ran linter

* changes suggested by nadrew done

* ran linter

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 09:00:13 -07:00
36ace6f4a6 deleted inventory_prediction_folder,folder moved to official (#856)
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 08:24:39 -07:00
32d9b416c1 UUID changes done comparing_pipeline_runs.ipynb (#759)
* done UUID changes

* ran lintertest

* made changes in cleanup section

* ran lintertest

* done UUID changes

* ran lintertest

* made changes in cleanup section

* ran lintertest

* made changes in cleanup section

* Ran linter test

* made UUID changes

* RAN linter test

* Made Some minor Chanages notebook

* Ran Linter Test

* small changes made

* Ran Linter Test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 08:07:17 -07:00
1c6309f401 Modified notebook SDK_AutoML_Video_Classification.ipynb (#839)
* modified notebook according to template, tensorflow library is used only for file opening so instead of tf we used bucket.blob.download_as_string()

* ran linter

* all changes requested by andrew are done

* cleared all outputs

* making changes to run linter test

* making changes to run linter test

* ran linter

* removed region text in create bucket step

* ran linter

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-10 19:05:41 -07:00
728dec8526 Updates the custom-tabular-bq-managed-dataset notebook according to the new template (#780)
* updates the configuring steps, replaces timestamp with uuid, expands the imports

* ran linter test

* separates the vertex-ai and bigquery initialization steps

* adds comment to cell_24

* adds blank line to cell_24:7:1

* adds blank line to cell_24:7:1

* ran linter test

* fixes aiplatform+bigquery installation compatibility issue

* ran linter test

* fixes installation dependencies

* ran linter test

* fixes the issues from the review: future tense, section positions, updates from the latest template

* ran linter test

* fixes the issues from the review: Code formatting, delete redundant cells, resource name changes

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-10 08:41:50 -07:00
sudarshan-SpringMLandGitHub 7692f902dd Added minor changes to training-multi-class-classification-model-for-ads-targeting-usecase notebook (#735)
* added new file

* ran linter test

* made small changes

* ran linter

* added file

* ran linter

* changes requested by andrew done

* ran linter
2022-08-10 08:03:42 -07:00
9281403198 fix cleanup cell to undeploy models before deleting endpoint and remove BQ table and dataset (#846)
* fix artifact cleanup

* lint fixes

* remove extraneous echo

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-09 20:09:25 -07:00
2be602fbef Update comparing_local_trained_models.ipynb (#704)
Install from Pypi

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-09 15:39:54 -07:00
b140077eb1 Change REGION_NAME to REGION to match flag set earlier in notebook (#734)
Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-09 14:54:22 -07:00
d3139f7df8 Added colab , installed some packages, added some code based on standard template and made changes in cleanup section for the file inventory_prediction.ipynb (#533)
* Added notebook

* Ran linter test

* added pyarrow to install

Co-authored-by: Andrew Ferlitsch <aferlitsch@gmail.com>
Co-authored-by: sudarshan-SpringML <82567512+sudarshan-SpringML@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-09 14:49:40 -07:00
72baced988 model monitoring notebook fixes (#841)
* fix model monitoring notebook

* lint fixes

* fixed Soheila's review comments

* fix Andrew's comments

* lint fixes

* clean up install packages per review request

* lint fixes

* fix more comments from Andrew

* lint fixes

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-09 14:16:54 -07:00
Soheila ZangenehandGitHub 5232654705 Update folder name to training (#845) 2022-08-09 12:29:34 -07:00
Andrew FerlitschandGitHub 125fe32eb2 feat: notebook for DASK training (#837)
* feat: tune template

* feat: tune template

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review

* feat: auto review

* feat: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review script

* tune: project ID and Region

* tune: project ID and Region

* fix: set MODEL_DIR for HPT

* fix: set MODEL_DIR for HPT

* feat: add batch example

* feat: add batch example

* feat: add batch example

* feat: add batch example

* feat: add notebook for DASK training

* feat: add notebook for DASK training
2022-08-08 16:17:28 -07:00
Andrew FerlitschandGitHub 614f4154dc fix: owner names 2022-08-08 15:50:33 -07:00
Andrew FerlitschandGitHub 970415398b fix: owner username 2022-08-08 15:46:57 -07:00
Hyunuk LimandGitHub a052a3d361 A new notebook for Spark Sample (#720)
* Created spark notebook with test code

* Implemented experimental code for poly_view

* implemented the table

* WIP-notebook

* moved experimental to tutorial

* delete experimental code and rename the notebook

* clear all outputs

* fix: delete outputs again

* fix: changed template to the newer one

* fix: modify link on workbench

* add creating a cluster

* Completed Before you begin part

* Change execution sequence

* completed write back process

* change order that switching kernel goes top

* modify pie chart to bar chart

* Completed write up part

* WIP: adding description and comments.

* WIP: delete outputs

* fix: nbqa done

* Completed the first draft

* Delete %%time from cells

* Apply changes as per the code review from Brad except SparkSql

* delete outputs

* Change SparkSQL to Spark API

* change label to xlabel

* fix: description in Dataset

* fix: change BUCKET_NAME to DATASET_NAME, link for the region, and add descriptions and examples for frequency table

* fix: move normalize_name to top of the cell

* fix: refactor udf functions and descriptions

* fix: add link for udf

* fix: description in Dataset

* fix: grammer

* fix: add declared in the sentence

* fix: small changes on grammar

* fix: delete string

* fix: change UserDefinedFunction to udf

* fix: as per TW's code review

* fix: reorder REGION and TIMESTAMP under Creating a GCS bucket

* fix: lint

* chore: add bmiro@ as a codeowner of this doc

* fix: change the variable to fix a bug

* fix: as per TW's second review

* fix: add installation part to pass the ci test

* fix: url for links to main

* fix: delete disabling API since it doesn't affect to the pricing

* fix: add conditions for CI test

* fix: changed jar for testing

* fix: add gcs connector

* fix: change writing method to direct

* fix: delete gcs connector

* fix: specify java folder

* fix: change java_home location

* fix: change unzip instruction

* fix: delete mono_ranking_avg_bytes from testing env

* fix: delete frequency_table from testing env

* fix: delete GCS bucket part

* fix: as per Brad's review

* fix: lint

* fix: add version

* fix: change comment

* fix: add package due to switching the kernel

* fix: delete dataproc cluster command

* fix: change link

* fix: change link

* fix: revert cluster deletion command

* fix: change parenthesis to encoded character

* fix: change the name of the notebook

* fix: change timestamp to UUID

* fix: change the link and add description

* fix: change metadata
2022-08-08 16:23:16 -04:00
Krishna Chaitanya MovvaandGitHub f8203b9f46 UUID replaces Timestamp in the notebook template (#745)
* replaces timestamp with uuid #create *task #tag1 replace the TIMESTAMP with uuid in other official notebooks

* ran linter test

* updates the uuid code

* fixes the comment style highlighted through linter-test

* ran linter test

* adds length argument to uuid function defaulted to 8

* ran linter test
2022-08-08 11:20:33 -04:00
Andrew FerlitschandGitHub fac8c4c9b1 fix: improve batch description (#830)
* feat: tune template

* feat: tune template

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review

* feat: auto review

* feat: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review script

* tune: project ID and Region

* tune: project ID and Region

* fix: set MODEL_DIR for HPT

* fix: set MODEL_DIR for HPT

* feat: add batch example

* feat: add batch example

* feat: add batch example

* feat: add batch example
2022-08-05 13:50:14 -07:00
Andrew FerlitschandGitHub d6c13c201e Ml ops v9v4 (#817)
* feat: tune template

* feat: tune template

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review

* feat: auto review

* feat: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review script

* tune: project ID and Region

* tune: project ID and Region

* fix: set MODEL_DIR for HPT

* fix: set MODEL_DIR for HPT

* feat: add batch example

* feat: add batch example
2022-08-05 09:40:11 -07:00
Soheila ZangenehandGitHub 8db9ce0415 Move download link up (#813) 2022-08-04 15:05:18 -04:00
Ivan CheungandGitHub 8c7fb38136 Added missing dependency (#814) 2022-08-04 14:53:17 -04:00
Andrew FerlitschandGitHub acec861ad5 tune: notebook template (#812)
* feat: tune template

* feat: tune template

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auti review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review

* feat: auto review

* feat: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* fix: auto review

* feat: auto review script

* tune: project ID and Region

* tune: project ID and Region
2022-08-04 10:51:08 -07:00