Commit Graph
247 Commits
Author SHA1 Message Date
Eric DongandGitHub 1f2db2903c refactor: Updated Github icons (#4413) 2026-01-12 10:10:34 -05:00
9c9f7a6e2a Migrate gsutil usage to gcloud storage (#4334)
* Migrate gsutil usage to gcloud storage

* Manual Changes

* changes for 4334

* fix linter issue\ for 4334

* manual changes

* Restore gcloud migration code

* Remove changes for model garden

* update

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-31 08:36:21 -05:00
ca53786c04 Migrate gsutil usage to gcloud storage (#4322)
* Migrate gsutil usage to gcloud storage

* Changes for 4322

* fix linter issue for 4322

* removed changes for model garden

* Update model_garden_pytorch_gemma_peft_finetuning_hf.ipynb

* Update model_garden_pytorch_gemma_peft_finetuning_hf.ipynb

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-25 10:49:46 -05:00
419f8310c9 Migrate gsutil usage to gcloud storage (#4316)
* Migrate gsutil usage to gcloud storage

* Changes for 4316

* Changes for 4316

* fix linter issue for 4316

* removed changes for model garden

* removed changes for model garden

* Update model_garden_tfvision_image_classification.ipynb

* Update model_garden_tfvision_image_classification.ipynb

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-25 10:49:33 -05:00
5b9d04d63a Migrate gsutil usage to gcloud storage (#4318)
* Migrate gsutil usage to gcloud storage

* Changes for 4318

* fix linter issue for 4318

* removed changes for model garden:

* Update model_garden_gemma2_deployment_on_vertex.ipynb

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-25 10:47:49 -05:00
9ed3c2f83d Migrate gsutil usage to gcloud storage (#4320)
* Migrate gsutil usage to gcloud storage

* Changes for 4320

* fix linter issue for 4320

* removed chnages for model garden

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-25 10:47:17 -05:00
633cf6a799 Migrate gsutil usage to gcloud storage (#4327)
* Migrate gsutil usage to gcloud storage

* Manual Changes-Migrate gsutil usage to gcloud storage

* Changes for 4327

* fixing linting erro

* removed changes of model garden

---------

Co-authored-by: bhandarivijay <bhandarivijay@google.com>
Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-25 10:44:05 -05:00
3239b301f2 Migrate gsutil usage to gcloud storage (#4315)
* Migrate gsutil usage to gcloud storage

* changes for 4315

* Apply automated linter fixes

* removed model_garden folder changes

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-24 09:41:28 -05:00
4d00356f4b Migrate gsutil usage to gcloud storage (#4312)
* Migrate gsutil usage to gcloud storage

* changes for 4312

* Apply automated linter fixes

* remove model_garden folder changes

* remove model_garden folder changes

* Update model_garden_pytorch_llama2_peft_hyperparameter_tuning.ipynb

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-24 09:40:01 -05:00
0654305994 Migrate gsutil usage to gcloud storage (#4311)
* Migrate gsutil usage to gcloud storage

* changes for 4311

* Apply automated linter fixes

* removed unused imports

* remove model_garden changes

* Update model_garden_pytorch_zipnerf.ipynb

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-24 09:39:27 -05:00
ca7d4e153e Migrate gsutil usage to gcloud storage (#4297)
* Migrate gsutil usage to gcloud storage

* changes for 4297

* Apply automated linter fixes

* removed changes from model_garden

* removed changes from model_garden

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-19 11:56:49 -05:00
5b6c766629 Migrate gsutil usage to gcloud storage (#4298)
* Migrate gsutil usage to gcloud storage

* update

* linter changes for 4298

* Revert "linter changes for 4298"

This reverts commit c7a00a9710.

* Linter fixes

* update

* update

* Update distributed_hyperparameter_tuning.ipynb

* removed changes in model_garden folder

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-19 11:56:15 -05:00
23af5373ec Migrate gsutil usage to gcloud storage (#4295)
* Migrate gsutil usage to gcloud storage

* removed notes

* Apply automated linter fixes

* Revert "Apply automated linter fixes"

This reverts commit 3f6c4c0d36.

* linter changes

* update

* update

* update

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-19 10:09:05 -05:00
70770a50c7 Migrate gsutil usage to gcloud storage (#4302)
* Migrate gsutil usage to gcloud storage

* changes for 4302

* changes for 4302

* removed note

* linter changes

* Revert "linter changes"

This reverts commit a9544e8251.

* Apply automated linter fixes

* Update lightweight_functions_component_io_kfp.ipynb

* Update lightweight_functions_component_io_kfp.ipynb

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-18 13:48:39 -05:00
0103299084 Migrate gsutil usage to gcloud storage (#4304)
* Migrate gsutil usage to gcloud storage

* changes for 4304

* Apply automated linter fixes

* remove unused import

---------

Co-authored-by: gurusai-voleti <gvoleti@google.com>
2025-12-18 14:00:23 +00:00
Holt SkinnerandGitHub 828f1a26fa Delete notebooks/official/pipelines/google_cloud_pipeline_components_automl_text.ipynb (#4254)
b/442906902
2025-09-09 20:43:03 +00:00
Eric DongandGitHub 7be7d5be44 refactor: Batch update icon links for notebooks/official (#4187) 2025-07-31 00:08:25 +00:00
Eric DongandGitHub 0bfe70afcd refactor: remove runtime reboot (#4026)
* refactor: remove runtime reboot

* Remove spaces

* Use %pip instead

* Update model path

* Update BQ path

* Downgrade numpy for backfoward compability
2025-05-05 21:12:55 +00:00
Franklin WhaiteandGitHub b5f634dd19 Update pandas version on xgboost_training component in kfp2_pipeline.ipynb (#3866)
The current version was causing a the below error when executing on xgboost_training component on vertex.

```
numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
```

Updating the pandas version as shown [here](https://github.com/numpy/numpy/issues/26710) resolved the issue
2025-03-08 01:49:52 +00:00
cf80c17db3 <Fix> Fixed the colab enterprise link. (#3468)
Co-authored-by: UBhavani <bhavani.ummadi@egen.ai>
2024-08-23 21:18:48 +00:00
Katie NguyenandGitHub ba935bf993 fix: title modifications (#3424) 2024-08-14 07:21:46 +00:00
85522dfe48 fix, chore, feat, refactor(Egen): Replace K80 with T4, add cleanup steps, refactors (#3272)
* fix, refactor, chore: follows new template, replaces K80 with T4, replace docker steps with cloud build, reorganize the sections, heading corrections

* chore: remove will and contracts 'do not'

* fix: replace K80 with T4

* feat: adds step to remove the training folder in the cleaning up section

* fix, chore: adds worker-pool-specs back in the pipeline as a global var, adds comments

* feat: adds a cleaning up step for artifact registry

* fix, chore: addresses the review comments, removes the mention of experimental feature in the markdown, replaces TPU_V3 with TPU_V2(not found error)

* chore: addresses review comments

* fix, chore, refactor: updates tensorflow version to 2.13, updates TPU driver libs, elaborates some steps, refactors the pipeline creation and run step to parameterize the arguments instead of using global vars

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-08-13 17:57:56 +00:00
4c7180bd10 refactor, chore(egen): Replaces K80 GPU with T4, package version updates, pre-built Docker container image for prediction update, other corrections from template (#3241)
* <refactor, chore> updates gcr to artifact registry, package version upgrades, updates prebuilt docker container image to 2.13, refactores notebook according to the template

* <refactor, chore> updates gcr to artifact registry, package version upgrades, updates prebuilt docker container image to 2.13, refactores notebook according to the template

* updated pip install statements

* Colab enterprise link fix

* Colab enterprise link fix

* Colab enterprise link fix

* markdown edits

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-08-13 17:44:07 +00:00
Aaron DietzandGitHub 26a25d0cf7 Update google_cloud_pipeline_components_automl_tabular.ipynb (#3418)
Restructured bullet points to remove nested bullets. The nested bullets don't render properly when we auto-generate our notebook list for our docs.
2024-08-12 21:26:15 +00:00
Aaron DietzandGitHub 3ece113c7a Minor update to text in automl_tabular_classification_beans.ipynb (#3413)
Updates objectives to eliminate nested bullets (those didn't get rendered in our docs) and removes some bolding that wasn't rendering properly in our docs
2024-08-09 20:52:07 +00:00
811d321451 refactor,chore(egen): added colab enterprise for custom_tabular_train_batch_pred_bq_pipeline notebook (#3387)
* refactor,chore(egen): added colab enterprise link for custom_tabular_train_batch_pred_bq_pipeline notebook

* refactor,chore(egen): added colab enterprise link for custom_tabular_train_batch_pred_bq_pipeline notebook

---------

Co-authored-by: Jayakrishna2801 <jayakrishna.rajaboina@egen.ai>
2024-08-06 03:38:12 +00:00
eb8a0a93a2 refactor,chore(egen): added colab enterprise for lightweight_functions_component_io_kfp notebook (#3386)
Co-authored-by: Jayakrishna2801 <jayakrishna.rajaboina@egen.ai>
2024-08-06 00:01:16 +00:00
cb8ec73e0a refactor(egen): Colab enterprise link fix (#3358)
* <fix, refactor> fixed and refactored notebook according to the template

* Apply suggested edits from @kittyabs

* Apply suggested edits from @kittyabs

* Colab enterprise link fix

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-07-31 05:15:22 +00:00
510e3d08ce refactor(egen): Colab enterprise link fix (#3359)
* <refactor, chore, fix> package version updates, pipeline components documentation link update, importer_node import fix, machineSpec update

* <refactor, chore, fix> package version updates, pipeline components documentation link update, importer_node import fix, machineSpec update

* Colab enterprise link fix

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-07-31 05:14:41 +00:00
9e3edfc8e4 refactor(egen): Colab enterprise link fix (#3360)
* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* Grammar fix

* lint fix

* Colab enterprise link fix

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-07-31 05:13:43 +00:00
66563cb700 refactor, chore(egen): refactored the notebook according to the template, updated and added new package (#3289)
* <Refactor, Chore> Refactored the notebook according to the template, updated and added new package.

* Applied suggested edits.

* Applied suggested edits.

---------

Co-authored-by: UBhavani <bhavani.ummadi@egen.ai>
2024-07-21 17:35:15 +00:00
16f117a7eb chore, refactor, feat(egen): Follows new template, markdown fixes, refactors (#3227)
* chore, refactor, feat: follows new template, fixes typos, updates dsl.Condition to ds.If, rewords the headings and organizes them as per the tutorial, adds a cleanup step for the pipeline file

* chore, fix: minor sentence corrections, removes undefined UUID parameter

* chore: addresses the review comments

* fix: fixes the var name pipeline --> pipeline_job

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-12 05:22:54 +00:00
f5d57558fe refactor, chore(egen): Replaces K80 GPU with T4, kfp and tensorflow versoin updates, pre-built Docker container image for training and prediction update, other corrections from template (#3219)
* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* <refactor, chore> replaces K80 GPU with T4, updates pre-built Docker container image for training and prediction to 2.13, cleansup intermediate files, updates kfp and tensorflow versions, fixes minor spelling mistakes and contracts words, removes future tense

* Grammar fix

* lint fix

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-07-12 00:34:20 +00:00
0f01fd7c18 chore(egen): follows new template, fixes typos, K80-->T4, REGION-->LOCATION etc. (#3214)
* chore: follows new template, fixes typos, removes unnecessary code-highlights, replaces K80 with T4, replaces REGION with LOCATION, removes IS_TESTING in cleaning up section, makes sentence/heading corrections and re-organizes some subsections as per the tutorial

* chore, refactor: minor markdown corrections, updates machine_type description and code to suit the explanation

* chore: addresses review comments and corrects 'uploading to a Vertex AI model resource' to 'uploading to Vertex AI Model Registry'

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-12 00:31:39 +00:00
4f7b63f3af refactor,chore(egen) : refcatored code according to template guidelines and added cleanup code (#3211)
* refactor,chore(egen) : refcatored code according to template guidelines and added cleanup code

* refactored code according to template guidelines

* refcatord code according to template guidelines

* refactor(egen) : refactored code according to template guidelines

* refactor(egen) : refactored code according to template guidelines

---------

Co-authored-by: Jayakrishna2801 <jayakrishna.rajaboina@egen.ai>
2024-07-12 00:23:53 +00:00
Kaushik KoiladaandGitHub 99fa616a14 refactor, chore, fix(egen): edits google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb notebook (#3229)
* chore: removes boiler plate and reformats according to the template

* chore: adds colab enterprise link to the notebook

* chore: testing end to end

* chore:end to end test with reformatting

* chore: lint

* chore: addresses comment on the markups
2024-07-10 22:02:12 +00:00
650272e370 chore, fix(egen): follows new template, markdown updates and fixes (#3221)
* chore: follows new template, remove IS_TESTING, replace REGION with LOCATION, organizes headings and styles, removes unnecessary code highlights

* fix: removes USER var, adds IS_COLAB var

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-10 21:36:46 +00:00
8429bba776 chore,refactor(egen):changed the versions of tensorflow, tensorflow-hub, apache_beam[gcp] and bs4 in requirements.txt and setup.py files, refactored code according to template guidelines (#3233)
Co-authored-by: sriramya2610 <sriramya.peddapally@egen.ai>
2024-07-10 21:05:47 +00:00
1a4db478de chore(egen) : added note to deprecated notebook and performed linter test (#3210)
* chore(egen) : added notes to deprecated notebook

* chore(egen) : performed linter test

* chore(egen) : added notes to deperecated notebook and performed linter test

* linter test

---------

Co-authored-by: Jayakrishna2801 <jayakrishna.rajaboina@egen.ai>
2024-07-09 21:03:53 +00:00
2e330ab7ab fix,chore,refactor(Egen): replaced aip with aiplatform and k80 GPU with T4 GPU, changed the versions of images,.keras extension added when saving and uploading the model, removed import statement of os module, refactored code according to template guidelines and performed linter test. (#3213)
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-09 03:34:01 +00:00
141e77caec refactor, chore, fix(egen): Pipeline documentation link update, removes GPU from machineSpec in the pipeline, package version updates, fixes import errors (#3206)
* <refactor, chore, fix> package version updates, pipeline components documentation link update, importer_node import fix, machineSpec update

* <refactor, chore, fix> package version updates, pipeline components documentation link update, importer_node import fix, machineSpec update

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-07-09 03:04:57 +00:00
8eb8db2ef8 chore , refactor : removed ! rm lightweight_pipeline.json , Added cleanup code for deletion of pipeline and refactored code according to template guidelines , performed linter test (#3149)
* refactor : removed ! rm lightweight_pipeline.json

* chore,refactor : Added cleanup code for deletion of pipeline and refactored code according to template guidelines , performed linter test

* chore,refactor : Added cleanup code for deletion of pipeline and refactored code according to template guidelines , performed linter test

* refactor,chore : Added code for deletion of pipeline and refactored code according to template guidelines , performed linter test

* refactor,chore : refactored code according to template guidelines and downgraded numpy version

* refactor : removed project name and bucket name used for testing in local

* refactor : refactored code according to template guidelines

* performed linter test

---------

Co-authored-by: Jayakrishna2801 <jayakrishna.rajaboina@egen.ai>
2024-07-03 23:05:12 +00:00
a749fa3377 chore, refactor(egen): replace dsl.Condition with dsl.If, sentence corrections (#3199)
* refactor(egen): refacted markdown as per notebook template

* chore, refactor: removes future tense, minor markdown fixes, removes UUID and uses -unique

* chore, fix: replaces dsl.Condition with dsl.If, fixes some sentences and explanation

---------

Co-authored-by: sumanvita-springml <sumanvita.kandregula@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-03 13:45:04 +00:00
df0e5a09cc chore,refactor(Egen): replaced aip with aiplatform and kfp.v2 with kfp, removed import statement of os module, refactored code according to template guidelines and performed linter test. (#3170)
* refractor,chore(egen): replaced kfp.v2 with kfp in compile step, refracted code according to template guidelines, performed linter test

* chore,refactor(Egen): replaced aip with aiplatform, removed import statement of os module, refactored code according to template guidelines and performed linter test.

* refactor(egen):refactored code by renaming REGION variable to LOCATION and performed linter test

---------

Co-authored-by: sriramya2610 <sriramya.peddapally@egen.ai>
2024-07-02 01:49:49 +00:00
bb3d3bde21 fix, chore, refactor(egen): REST api fixes, KFP v2 refactor, new template (#3180)
* fix, chore, refactor: fixes the issue with job creation request, refactors to use the latest SDKs, follows the new template, sentence corrections

* chore: removes future tense, minor sentence corrections

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-01 20:40:05 +00:00
a8f9cbbaa6 fix, refactor(egen): fixes data import issue, includes pipeline job deletion, corrections from template. (#3150)
* <fix, refactor> fixed and refactored notebook according to the template

* Apply suggested edits from @kittyabs

* Apply suggested edits from @kittyabs

---------

Co-authored-by: SumanthKasula99 <sumanth.kasula@egen.ai>
2024-06-27 20:15:29 +00:00
b96dd105d9 chore,refactor(egen) : Added clean up code for deletion of model endpoint and local files, refactored code according to template guidelines, performed linter test (#3147)
* refractor(egen): refracted code according to template guidelines, performed linter test

* refractor(egen):  refracted code by removing os.getenv(IS_TESTING) in clean up code of bucket

* chore,refactor(egen):grammar check according to template guidelines, refactored the code by adding clean up code for deletion of local files created and model  endpoint and performed linter test.

* refactor(egen):refactored code by modifying clean up code and performed linter test

---------

Co-authored-by: sriramya2610 <sriramya.peddapally@egen.ai>
2024-06-27 20:07:57 +00:00
sumanvitaandGitHub 4d19f36d39 chore, refactor(egen) : changes file extension from .json to .yaml, refactored code as per template guidelines (#3146)
* fix, refactor, chore(egen): replaced .json to .yaml, refacted the code as per notebook template, performed linter test.

* refactor, chore(egen): Rephrases sentences, changes from google.cloud import aiplatform instead to import google.cloud.aiplatform as aip

* refactor(egen(egen): cleared cell output

* refactor, chore(egen): changes made as per PR comments, perfomed linter test
2024-06-27 20:02:21 +00:00
AmyandGitHub 2bb39a0e10 moving some data from personal bucket to 'cloud-samples-data' bucket (#2794) 2024-03-22 20:42:25 +00:00
kittyabsandGitHub c1d1e23cd0 Added missing link and made other edits to "Custom training with pre-built Google Cloud Pipeline Components". No impact on code (#2710)
* Added missing link and made other edits to "Custom training with pre-built Google Cloud Pipeline Components". No impact on code

* Update custom_model_training_and_batch_prediction.ipynb

fixed typo
2024-02-23 20:16:58 +00:00