Commit Graph
6 Commits
Author SHA1 Message Date
7577c0b1fc Migrate gsutil usage to gcloud storage (#4292)
* Migrate gsutil usage to gcloud storage

* Manual change

* update

* Update NotebookProcessors.py

* Update NotebookProcessors.py

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-19 15:07:50 +00:00
Ivan CheungandGitHub 6ccad69a18 Reduce boilerplate for custom tabular bq notebook (#1207)
* Reduced boilerplate

Ran linter and fixed install

Added pyarrow

Fixed pip install

Reverted unneeded changes

Fixed pip install

Linted code and added missing preprocessor call

Fixed preprocessor

Default to us-central1

* Ran linter

* Simplified service account section
2022-11-10 17:47:13 -05:00
f184411666 Create a function to get notebook python version for execution (#828)
* Create a function to get notebook python version for execution

* Inject python version to yaml file

* Fix python version references

* Add python string to python version variable

* add python version extraction script (untested)

* Create a function to get notebook python version for execution

* Inject python version to yaml file

* Fix python version references

* Add python string to python version variable

* Remove one notebook condition

* Remove extra check and use python 3 as default version

* Use python3.9 as default version

* Update python version notebook parser

* Add python version to the notebook template

* Fix bug

* Update python version parser function

* Add python version to a notebook for testing

* Run linter

* Use regex in python version parser function

* Add new notebook for testing

* Use better variable name

* fix typo

* Use f string instead +

* Use python from env instead of using _PYTHON_VERSION

* Use simpler regex

* Add python version test notebook

* Fixed a mistake

* Updated notebook template with python version

* Fixed python version format

* Print log contents to stdout

* Remove failing notebook

* Run linter

* Remove extra steps in the printed log

* Edit comments

* Run linter

* Run linter

* Revert test changes

Co-authored-by: AG Sol <aarongabriel@google.com>
2022-09-26 14:32:53 -07:00
Ivan CheungandGitHub 4fca7d98b2 Increases notebook concurrency and linted CI files (#512)
* Fixed private pool issues

* Ran linter

* Added worker timeouts

* Tweaked timeout

* Removed gcloud requirement

* Removed unneeded file
2022-05-18 18:58:54 -04:00
Ivan CheungandGitHub 92bb40349f Cleaned up cloud build by renaming files and moving methods around (#219)
* Cleaned up cloud build

* Fixed bug

* More renaming and comment cleanup

* Added helper file

* Fixed missing return
2022-01-20 15:55:38 -05:00
Ivan CheungandGitHub 740958005a Parallel custom job notebook execution (#120)
* Initial commit

* Added parallel execution
2021-11-12 18:31:36 -05:00