Commit Graph
76 Commits
Author SHA1 Message Date
c3998c9898 chore, refactor(egen): New template edits, clean up code refactor, remove future tense (#3403)
* Adapted notebook with new notebook template

* Added variable value which is used in further steps

* Added required permissions for service account

* chore, refactor: Follows new template, removes future tense, restricted links, refactors the cleaning up section

* feat: adds '-m' while deleting the cloud storage bucket

* chore: addresses the review comments

* chore: removes version mention for bison models and adds references to model versions and supported rlhf models

---------

Co-authored-by: nileshspringml <nilesh.mahajan@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-08-12 14:37:52 +00:00
8a2b41c5f1 fix, refactor, chore: Fix docker steps, Colab steps, follows new template etc. (#3385)
* chore: adds Colab Enterprise link

* fix, refactor, chore: Fixes the docker container image creation steps and predictions step, refactors the code to skip unnecessary steps and markdown text correction and simplification

* fix, chore: fixes typos in serving script, markdown text corrections, adds custom folder removal step in the cleanup section

* fix: adds --project for Colab steps and replaces the docker build and docker push with gcloud builds submit command for Colab

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-08-09 21:39:47 +00:00
bd89533c50 chore, feat, refactor(Egen): Clean up steps, template update etc. (#3341)
* chore, feat, refactor: Remove future tense, follows new notebook template, adds cleanup step for deleting the pipelines and models created, refactors the utility functions to fetch model resource

* fix: corrects the notebook name in the links

* chore: addresses review comments

* fix: loads the model from resource name before deletion

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-31 05:47:58 +00:00
75bd4bb561 fix: fixes the path in the colab enterprise link (#3303)
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-24 19:52:20 +00:00
c20d754717 chore(egen): Follows new template, removes IS_TESTING, REGION --> LOCATION (#3207)
* chore: Follows new template, removes IS_TESTING, relaces REGION with LOCATION

* chore: Cloud console --> Google Cloud console

* chore: addresses review comments, sets delete_bucket to True to remove the GCS bucket in the cleaning up step

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-21 17:57:56 +00:00
5d8489e3e0 chore, refactor, feat(egen): follows new template, simplifies code, adds cleanup step (#3259)
* chore, refactor, feat: follows new template, simplifies code for display-names, adds steps for deleting the resources in the cleaning up section

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-12 20:18:01 +00:00
16f117a7eb chore, refactor, feat(egen): Follows new template, markdown fixes, refactors (#3227)
* chore, refactor, feat: follows new template, fixes typos, updates dsl.Condition to ds.If, rewords the headings and organizes them as per the tutorial, adds a cleanup step for the pipeline file

* chore, fix: minor sentence corrections, removes undefined UUID parameter

* chore: addresses the review comments

* fix: fixes the var name pipeline --> pipeline_job

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-12 05:22:54 +00:00
0f01fd7c18 chore(egen): follows new template, fixes typos, K80-->T4, REGION-->LOCATION etc. (#3214)
* chore: follows new template, fixes typos, removes unnecessary code-highlights, replaces K80 with T4, replaces REGION with LOCATION, removes IS_TESTING in cleaning up section, makes sentence/heading corrections and re-organizes some subsections as per the tutorial

* chore, refactor: minor markdown corrections, updates machine_type description and code to suit the explanation

* chore: addresses review comments and corrects 'uploading to a Vertex AI model resource' to 'uploading to Vertex AI Model Registry'

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-12 00:31:39 +00:00
650272e370 chore, fix(egen): follows new template, markdown updates and fixes (#3221)
* chore: follows new template, remove IS_TESTING, replace REGION with LOCATION, organizes headings and styles, removes unnecessary code highlights

* fix: removes USER var, adds IS_COLAB var

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-10 21:36:46 +00:00
2e330ab7ab fix,chore,refactor(Egen): replaced aip with aiplatform and k80 GPU with T4 GPU, changed the versions of images,.keras extension added when saving and uploading the model, removed import statement of os module, refactored code according to template guidelines and performed linter test. (#3213)
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-09 03:34:01 +00:00
a749fa3377 chore, refactor(egen): replace dsl.Condition with dsl.If, sentence corrections (#3199)
* refactor(egen): refacted markdown as per notebook template

* chore, refactor: removes future tense, minor markdown fixes, removes UUID and uses -unique

* chore, fix: replaces dsl.Condition with dsl.If, fixes some sentences and explanation

---------

Co-authored-by: sumanvita-springml <sumanvita.kandregula@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-03 13:45:04 +00:00
bb3d3bde21 fix, chore, refactor(egen): REST api fixes, KFP v2 refactor, new template (#3180)
* fix, chore, refactor: fixes the issue with job creation request, refactors to use the latest SDKs, follows the new template, sentence corrections

* chore: removes future tense, minor sentence corrections

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-07-01 20:40:05 +00:00
d43c5fee3d chore, refactor(egen): follows new template, removes IS_TESTING, spell corrections (#3148)
* chore, refactor: follows new template, removes IS_TESTING in clean up, contracts words and some steps, spell correct, K80 to T4

* chore: addresses review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-26 22:34:35 +00:00
01317fda2a fix, chore, refactor(egen): Fix TF version, replace K80 with T4 etc. (#3143)
* chore, fix, refactor: template fixes, replace K80 with T4, remove IS_TESTING, reword some sentences and headings

* fix, refactor, feat, chore: replace K80 with T4, fix the TF version, template based fixes, remove local files in the clean up step

* chore: addresses review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-26 22:08:55 +00:00
7132270fb2 chore, feat(egen): template fixes, adds clean up steps (#3117)
* chore, feat: template fixes, adds steps to remove training job, local files, and remove future tense and contract long words

* fix: replaces REGION with LOCATION while creating the bucket

* refactor: reduces the budget_milli_node_hours to 1000 from 8000

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-24 13:10:40 +00:00
02bdc8e3cb fix, chore, refactor, feat(egen): Replaces K80 to T4, gcr with Artifact Registry etc. (#3111)
* fix, chore, refactor, feat: replaces K80 to T4, replaces gcr pushes to artifact registry pushes, template based fixes, clean up for local files, remove IS_TESTING

* fix: defines the IS_COLAB step before the condition

* fix: REGION is replaced by LOCATION while creating the artifact registry

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-24 13:07:48 +00:00
ebd738e863 fix, refactor, chore(egen): Fix TF version support, template format (#3120)
* refactor(egen): Refactored code according to the template guidelines

* fix, refactor: fixes the compatible tf version and removes unnecessary import in the clean up step

* fix: adds matplotlib in the installation step

---------

Co-authored-by: rohith-egen <rohith.alla@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-21 21:11:43 +00:00
8895075d28 chore: renames 'matching_engine' to 'vector_search' and 'Matching Engine' to 'Vector Search' (#3096)
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-17 12:40:24 +00:00
7cd354436e chore, refactor, fix, feat(egen): template structure+K80GPU+contract words+clean up (#3085)
* chore, refactor, fix, feat: template structure, removes redundant code, contracts content, replaces K80 with T4, cleans up local files

* chore,chore, fix: addresses review comments+ sets replica=1,accelerator_count=4

* fix, chore: reduces the GPU count to 1, rewords the title

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-14 21:12:17 +00:00
bd10dede25 refactor(egen): follows new template+sentence and other minor corrections (#3073)
* chore: restructures according to template, contracts text and cells, organizes headings

* refactor: removes the IS_TESTING conditions for steps involving redis instance

* chore: addresses review comments

* fix: adds back the IS_TESTING conditions for redis commands to skip in the test environment

* chore: removes duplicate comment

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-13 20:52:43 +00:00
7e8bf8a126 fix, refactor, chore(egen): follows new template, adds dataflow enable step, refactor (#3089)
* Updated template of notebook

* fix, refactor, chore: follows new template, adds dataflow enable step, cleanup steps for files, removes IS_TESTING in cleanup, REGION==>LOCATION

* chore: addresses the review comments

---------

Co-authored-by: nileshspringml <nilesh.mahajan@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-13 20:36:50 +00:00
f5c17c0700 refactor, chore, feat(egen): contracts content, steps & template based fixes (#3075)
* refactor, chore, feat: removes/contracts long explanations, template based fixes, adds step to remove locally generated files

* fix: adds the missing import, replaces code markdown with bold style at some places

* fix: replaces REGION with LOCATION

* chore: addresses review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-12 22:37:30 +00:00
ac8f7ad196 chore, fix, feat(egen): follows template + fix docker run + adds clean up (#3072)
* chore, fix: restructures as per template, rewords some sentences, removes sudo in the option docker run

* feat: adds clean up step for local files

* chore: reverts lowercase to camelcase for artifact registry and other review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-11 18:06:50 +00:00
7afdeb05f5 chore, fix(egen): corrections from template, guidelines, & fixes docker tag command (#3031)
* chore: replaces K80 with T4, Cloud ML with Vertex AI, REGION with LOCATION, restructures from template, & removes future tense

* fix: fixes the docker tag command for colab

* chore, fix: addresses the review comments, tf is pinned to 2.15.1 as the latest tf causes issues

* chore: replaces of with or

* chore: removes the collapsed license comment

* fix: removes the test env specific package update step and contracts the installations into one step to keep tf as 2.15

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-07 21:10:09 +00:00
2df29ec118 chore, fix(egen): restructuring from template, markdown corrections & dataset object fix (#3028)
* chore: restructures according to the template, removes future tense, removes IS_TESTING, unused imports

* chore, fix: addresses the review comments, converts npy array to list for running in py-3.9

* chore: adds --it's-- in the sentence

* chore: Getting started -->  Get started

* chore: removes the collapsed license comment

* fix: extracts list from numpy objects instead of Dataset object

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-07 20:58:54 +00:00
030024cc51 refactor, chore, fix(egen): GPU usage fixes, template & markdown fixes (#3050)
* refactor, chore, fix: replaces K80 with T4, fixes to follow the template, fixes to utilize the defined accelerators while training

* chore: addresses the review comments

---------

Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-05 20:51:27 +00:00
db9c182716 refactor, chore(egen): Removes boilerplate, heading fixes, and other corrections from template (#3021)
* refactor: removes boilerplate code, future tenses, and fixes heading styles

* chore: replaces REGION with LOCATION to match the notebook template

* chore: markdown heading fixes according to guidelines

---------

Co-authored-by: rohith-egen <rohith.alla@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-03 23:48:13 +00:00
54ae9a17b2 chore(egen): removes future tense, adds colab enterprise link, and colab only steps (#3025)
* chore: updates the notebook according to the latest template

* chore: linter test

* refactor: removes IS_TESTING, and os import

* ran linter test using linter.sh

---------

Co-authored-by: rohith-egen <rohith.alla@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-03 18:24:18 +00:00
263dba54bc fix, refactor(egen): Replaces K80 with T4 GPUs, corrections from template (#3013)
* fix: n1-standard-8 changed to n1-standard-16 and Tesla K80 changed to Tesla T4 + refactored code according to the template

* refactor: keeping the machine types same. Original issue with the K80 accelerators as they're no longer supported

---------

Co-authored-by: rohith-egen <rohith.alla@egen.ai>
Co-authored-by: krishr2d2 <krishna.movva@egen.ai>
2024-06-01 00:15:24 +00:00
bda6fb06d8 Fixes the build failure (#2195)
* feat: boilerplate reduction 59

* fix: bucket

* spark runtime

* dataproc runtine -> 1.1.20

* updates the build steps + clean up steps

* ran linter test

* adds gcloud components update step for test env + adds needed services in before you begin section

* ran linter test

* moves the gcloud components update to the start + updates experiment fetching

* ran linter test

* updates the aiplatform sdk to the latest

* ran linter test

* removes gcloud components update and adds try except at get_dataframe() method

* ran linter test

* initializes experiment + adds gcloud update + removes try catch

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-29 19:33:36 +00:00
Krishna Chaithanya MovvaandGitHub df20a2fe50 Fixes the 400 model server issue (#2190)
* fixes the sklearn versions + section structure updates based on template

* ran linter test
2023-08-17 17:29:39 +00:00
edb90d4255 Fixes the issues from PR#1871 (#2186)
* fix: boilerplate reduction 41

* fix: install

* fix: TFDV version

* fix: increase wait time

* increase wait time

* replace INPUT_GS_PATH with TRAINING_DATASET while copying data in GCS + removes future tense + clean up step for batch job + moves the learn more section above clean up section

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-16 16:42:43 +00:00
9ce9cec0a8 Reduction 88 (#2178)
* fix: boilerplate reduction 88

* fix: install

* fix: reduce dataset size

* fix: > 24hrs

* fix: reduce dataset for testing

* fix: reduce dataset for testing

* updates sklearn and fixes the version, elaborates some existing descriptions and adds updates based on the template

* ran linter test

* changes FILE_NAME to LOCAL_FILE_NAME and adds a clean up step for the local saved model

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-14 17:39:11 +00:00
42bc870ee3 Fixes timestamp issue + Elaborates some text descriptions (#2177)
* fix: boilerplate reduction 33

* fix: project ID

* fix: df type

* fix: uuid

* fix: uuid

* fix: uuid

* fix: uuid

* fix: timestamp

* fixes the timestamp issue + cleans up the descriptions

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-14 16:04:28 +00:00
07ec84687e Reduces the max_steps parameter + Textual content updates and fixes (#2168)
* fix: issue 2125

* removes duplicate parameters, reduces max_steps to 100, fixes grammar and updates realted to the writing guidelines

* removes f from the string parameter

* ran linter test

* sets max_steps to 20 and adds lines in the cleanup step to remove the pipeline jobs

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-10 20:49:01 +00:00
Krishna Chaithanya MovvaandGitHub 067fab6aba Adds preprocessing steps from the original dataset (#2165)
* adds the preprocessing steps on the original dataset + updates for boiler plate reduction

* ran linter test

* adds project-id and ticks in the sql queries

* ran linter test
2023-08-09 16:25:14 +00:00
68452d30ce Typo fixes + Template related updates + Grammar + Dataflow service dependancy (#2155)
* upgrade: boilerplate reduction 38

* fixes the pipeline-root-path typo, adds dataflow api dependency in the before-you-begin section, restructures the sections according to the template, fixes grammar and headings

* ran linter test

---------

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2023-08-08 04:23:28 +00:00
Krishna Chaitanya MovvaandGitHub a76ffe0103 moves the owners line from community file to the official file for SDK_FBProphet_forecasting_online.ipynb notebook (#1254) 2022-11-16 16:06:01 -05:00
168c5c2f15 Add custom tabular classification model evaluation notebook (#1219)
* adds the custom tabular classification evaluation notebook

* removes json import

* ran linter test

* fixes the model serving code and adds image

* ran linter test

* adds notebook to the codeowners file

* addresses the review comments: updates text, adds headings

* ran linter test

* boiler-plate reduction, addresses the review comments, adds a constant for table id

* ran linter test

* updates the installation step

* ran linter test

* removes the --user flag while installation

* ran linter test

Co-authored-by: Soheila Zangeneh <49654056+soheilazangeneh@users.noreply.github.com>
2022-11-16 10:12:14 -05:00
Krishna Chaitanya MovvaandGitHub d45bbb36df Adding the official version for SDK-FB-prophet-online-forecasting-notebook (#317)
Checklist for moving the notebook to the main repo: 
- [x] Use the [notebook template](https://togithub.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/notebook_template.ipynb) as a starting point.
- [x] Follow the style and grammar rules outlined in the above notebook template.
- [x] Verify the notebook runs successfully in Colab since the automated tests cannot guarantee this even when it passes.
- [x] Passes all the required automated checks. You can locally test for formatting and linting with these [instructions](https://togithub.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/docs/contributing.md#code-quality-checks).
- [ ] You have consulted with a tech writer to see if tech writer review is necessary. If so, the notebook has been reviewed by a tech writer, and they have approved it.
- [x] This notebook has been added to the [CODEOWNERS](https://togithub.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/docs/CODEOWNERS) file under `# Official Notebooks` section, pointing to the author or the author's team.
- [x] The Jupyter notebook cleans up any artifacts it has created (datasets, ML models, endpoints, etc) so as not to eat up unnecessary resources.
2022-11-16 00:26:14 +00:00
ee805e7846 Creates a notebook for batch-prediction pipeline with BQ I/O on a custom-tabular model (#992)
* adds the custom-model-bq-io-batch-prediction-pipeline notebook to the official folder

* removes the unused libraries and variables

* ran linter test

* adds a raise exception statement to prevent auto-test check as this notebook is just to serve as a template

* resolves review comments: fixes names, removes trailing comma, fixes link format

* removes the raise excpetion step

* ran linter test

* cleans up code, explains sections and renames the notebook

* removes unused variable batch_predict_task

* removes unnecessary imports

* ran linter test

* updates the dependencies

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-11-09 16:16:48 -08:00
5c403dde3d Minor updates for automl-tabular-classification-model evaluation notebook (#1184)
* upgrades gcpc to latest, adds minor textual corrections

* ran linter test

* minor textual corrections

* ran linter test

* fixes sentence cases, unnecessary capitalizations, article corrections and sentence corrections

* ran linter test

* renames the ground_truth_column argument in classificationeval component to target_field_name as per version 1.0.26 and updates the reference link

* ran linter test

* upgrades gcpc to latest, adds minor textual corrections

* ran linter test

* minor textual corrections

* ran linter test

* fixes sentence cases, unnecessary capitalizations, article corrections and sentence corrections

* ran linter test

* renames the ground_truth_column argument in classificationeval component to target_field_name as per version 1.0.26 and updates the reference link

* ran linter test

* fixes the prediction_score_column unsupported error, adds class_labels field, adds the targetfieldremover component, minor structural/textual updates

* ran linter test & updates the image

* removes unnecessary newline

* ran linter test

Co-authored-by: Soheila Zangeneh <49654056+soheilazangeneh@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-11-07 08:15:33 -08:00
a70f93fe0a Resolves name collisions in featurestore-pandas notebook + elaborates descriptions (#1199)
* updates featurestore name with uuid to avoid name collisions, elaboration, textual corrections

* ran linter test

* addresses review comments: adds headings + textual corrections

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-11-03 12:43:25 -07:00
015809b948 Cleans up official/migration/UJ3 notebook and fixes the issue from regression logs (#1063)
* cleans up the notebook,replaces docker with cloud build, textual edits still in progress

* cleans up the notebook

* ran linter test

* changes tf train/serve version to 2.9

* ran linter test

* adds '=' to fix a typo

* ran linter test

* updates the opencv installation package

* ran linter test

* updates the opencv installation dependencies

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-10-07 09:37:12 -07:00
b746bc80d4 Resolves shell-output issue in google_cloud_pipeline_components_model_upload_predict_evaluate.ipynb notebook (#878)
* resolves the shell-output issue + updates the structure based on the template

* ran linter test

* fixes issues from review: textual updates, delete_bucket=False

* ran linter test

Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-09-26 16:27:29 -07:00
1476453603 Moves Sentiment-Analysis notebook from community to official folder (#868)
* moves the sentiment_analysis notebook from community to official folder after making the updates

* removes unused modules

* ran linter test

* updates the dataset's GCS links and notebook links in the heading

* ran linter test

* fixes the typo(=)

* ran linter test

* removes wait() calls and IS_TESTING condition

* ran linter test

Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-09-20 07:35:15 -07:00
fa91e45018 Adds the updated managed_notebooks/predictive_maintenance notebook to the official folder (#521)
* adds the updated predictive-maintenance (managed)notebook from community to official folder

* ran linter test

* resubmitting during phase2

* ran linter test

* addresses the review comments: updates based on the new template, sets delete_bucket to False

* ran linter test

* replaces timestamp with uuid

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-16 09:39:07 -07:00
fa27309134 Fixes the issues from the regression tests (#822)
* fixes tensorflow version, replaces timestamp with uuid, adds service-account and minor textual changes

* removes second defnition of random library

* ran linter test

* uncomments the user flag and updates the installation command

* ran linter test

* removes the extra backslash

* ran linter test

* updates the installation step to fix long running compatibility checks

* ran linter test

* updates the METADATA path during installation steps

* ran linter test

* adds google-api-core version in the installation

* ran linter test

* removes METADATA step during installation

* ran linter test

* updates google api-core & auth versions

* ran linter test

* fixes tensorflow version, replaces timestamp with uuid, adds service-account and minor textual changes

* removes second defnition of random library

* ran linter test

* uncomments the user flag and updates the installation command

* ran linter test

* removes the extra backslash

* ran linter test

* updates the installation step to fix long running compatibility checks

* ran linter test

* updates the METADATA path during installation steps

* ran linter test

* adds google-api-core version in the installation

* ran linter test

* removes METADATA step during installation

* ran linter test

* updates google api-core & auth versions

* ran linter test

* fixes the issues from review: cell descriptions, parameter definitions, tense changes, 3rd person --> 2nd person, list model after pipeline run

* ran linter test

* removes the METADATA hack and updates the installations

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-16 09:11:30 -07:00
07b4c37135 Automl links fix (#742)
* fixing links to open notebook - main and images

* linter test changes

* fixes the papermill execution error(hard-coded bucket link was the cause)

* ran linter test

* adds minor textual changes

* ran linter test

* fixes issues from review: future tense, copyright year, section placement, latest sdk methods, new updates from the template

* ran linter test

Co-authored-by: Manuel Amunategui <manuel.amunategui@springml.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-12 12:34:53 -07:00
Krishna Chaitanya MovvaandGitHub 15bb4cea73 Fixes the issues(missing variable) from regression tests, adds minor updates from the template. (#823)
* adds the missing delete_bucket variable, adds steps to configure SERVICE_ACCOUNT

* removes unnecessary random import

* ran linter test

* removes the src folder dependency to run on Colab, adds the pipeline.wait step, updates the cleanup steps

* ran linter test

* fixed issues from review: section posistions, tense changes, section descriptions, template updates

* ran linter test
2022-08-12 08:07:11 -07:00
161965cfb0 Fixes the exception in batch-explain notebook in the official folder (#810)
* fixes exception(reg-test), replaces timestamp with uuid, minor changes

* ran linter test

* resolved review comments: license year, Vertex AI SDK, dataset after objective and future tense

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-11 10:25:51 -07:00
728dec8526 Updates the custom-tabular-bq-managed-dataset notebook according to the new template (#780)
* updates the configuring steps, replaces timestamp with uuid, expands the imports

* ran linter test

* separates the vertex-ai and bigquery initialization steps

* adds comment to cell_24

* adds blank line to cell_24:7:1

* adds blank line to cell_24:7:1

* ran linter test

* fixes aiplatform+bigquery installation compatibility issue

* ran linter test

* fixes installation dependencies

* ran linter test

* fixes the issues from the review: future tense, section positions, updates from the latest template

* ran linter test

* fixes the issues from the review: Code formatting, delete redundant cells, resource name changes

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-08-10 08:41:50 -07:00
Krishna Chaitanya MovvaandGitHub f8203b9f46 UUID replaces Timestamp in the notebook template (#745)
* replaces timestamp with uuid #create *task #tag1 replace the TIMESTAMP with uuid in other official notebooks

* ran linter test

* updates the uuid code

* fixes the comment style highlighted through linter-test

* ran linter test

* adds length argument to uuid function defaulted to 8

* ran linter test
2022-08-08 11:20:33 -04:00
406de6fe9a Adds the updated sdk-feature-store-pandas notebook to official from community folder (#488)
* adds the updated sdk-feature-store-pandas notebook to official and removes it from the community

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: nayaknishant <nishantnayak@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-08-03 12:30:10 -07:00
dbf19acde4 Adds the updated telecom-subscriber-churn-prediction notebook to official and removes from community (#506)
* updates and adds the telecom-subscriber-churn-prediction notebook to official and removes from the community

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-06-05 12:53:21 -07:00
1aaa833153 Adds manual-scaling config and explanation to the Automl-forecasting-batch notebook in official folder (#514)
* adds manual-scaling config and explanation to the notebook

* ran linter test after installing linter requirement updates

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-06-05 12:50:23 -07:00
9470be0900 adds the updated service-account code to mlops/stage3/get_started_with_dataproc_serverless_pipeline_components notebook (#529)
* adds the updated service-account setting code to notebook

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-05-04 10:04:55 -07:00
479a0c6271 adds the updated service-account code to mlops/stage3/get_started_with_automl_pipeline_components notebook (#528)
* adds the updated service-account setting code to the notebook

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-05-04 10:04:13 -07:00
053f9c397f adds the updated service-account code to mlops/stage3/get_started_with_kubeflow_pipelines notebook (#527)
* adds updated service-account setting code to the notebook

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-05-04 10:03:39 -07:00
fe1a3c0bc9 Adds Colab part and minor changes to ml_ops/stage2/get_started_bqml_training notebook (#491)
* adds the ml_ops/stage2/get_Started_bqml_training notebook to official and removes the same from community folder

* ran linter test

* updates the textual content

* ran linter test

* moves the updated stage2/get-started-bqml notebook back to the communit folder

* ran linter test

* updates the header according to the template

* ran linter test

* adds colab part and minor changes

* ran linter test

* retains the newly added code lost in conflicts

* ran linter test

* converts vertex to vertex ai

* ran linter test

* moves deletion of temporary BQ table outside delete_storage condition

* ran linter test

* adds bigquery-storage dependency to the notebook tested on Colab

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-22 11:09:49 -07:00
0137cd106e adds Colab part and minor changes to ml_ops/stage2/get_started_vertex_experiments notebook in community folder (#493)
* updates the get-started-vertex-experiments notebook in the community folder

* ran linter test

* adds the costs section

* ran linter test

* adds colab part and minor changes

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-21 09:20:29 -07:00
8b3b63d714 Adds Colab part and minor changes to ml_ops/stage2/get_started_automl_training notebook (#492)
* updates the get-started-automl-training notebook

* ran linter test

* adds --user flag during installation step

* ran linter test

* updates the clean up step

* ran linter test

* adds colab part and minor changes

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-04-21 09:19:34 -07:00
bf79916f29 Adds Colab part to ml_ops/stage1/get_started_bq_datasets notebook (#490)
* adds the updated mlops-stage1-get_started_bq_datasets notebook to the official branch and removes it from the community branch

* removes second instance of create_bigquery_dataset() function

* ran linter test successfully

* adds costs section

* ran linter test successfully

* updates the dependency installation step and GCS bucket explanation

* ran linter test

* adds pyarrow to the installations

* ran linter test

* removes unnecessary installations + adds silent install + moves the notebook back from official to community folder + adds IS_TESTING condition during clean-up

* ran linter test

* resolves the move up?? comment and builtin comment

* ran linter test

* updates textual content about package installation

* ran linter test

* resolves the future-tense and  dependency installations comments

* ran linter test

* updates the header according to template

* ran linter test

* adds Colab part and minor changes

* ran linter test

* updates the enable apis step in setup project section

* ran linter test

* changes vertex to vertex ai

* ran linter test

* moves temporary BQ table deletion outside the delete_storage condition

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-21 09:17:49 -07:00
3bc32a1d48 Adds Colab part to the ml_ops/stage3/get_started_with_automl_pipelines notebook (#472)
* updates the get-started-automl-pipelines in the mlops/stage3 folder inside community folder

* replaces the unused variable deploy_op with _

* removes the unused Model import

* adds the costs section

* ran linter test

* adds Colab part to the notebook

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-12 11:52:06 -07:00
d66f851554 Adds Colab part to the ml_ops/stage3/get_started_with_kubeflow_pipelines notebook (#471)
* updates the mlops/stage3/get_started_with_kubeflow_pipelines.ipynb notebook

* fixes unused variables

* fixes conflicting function names

* ran linter test

* adds colab changes

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-12 11:51:34 -07:00
03dc17d9d7 updates ml_ops/stage3/get_started_with_dataproc_serverless_pipelines notebook (adds delete-batch code + adds colab part + updates textual content) in community folder (#464)
* updates: adds delete-batch code  + adds colab part + updates textual content

* sets delete_bucket to False as default

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-12 11:49:26 -07:00
a4909f823e Adds changes for Colab support to the ml_ops/stage2/get-started-with-vertex-featstore notebook (#462)
* updates get-started-featurestore notebook in mlops/stage2

* ran linter test

* adds the colab changes and minor textual changes

* ran linter test

* adds the colab changes to the notebook and minor textual changes

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-12 11:48:58 -07:00
f84e1b9cbc Updates ml_ops/stage3/get-started-kubeflow-pipeline-notebook in the community folder (#449)
* updates the mlops/stage3/get_started_with_kubeflow_pipelines.ipynb notebook

* fixes unused variables

* fixes conflicting function names

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-08 09:31:05 -07:00
0e24ba565d Updates ml_ops/stage3/get-started-automl-pipeline-notebook in the community folder (#440)
* updates the get-started-automl-pipelines in the mlops/stage3 folder inside community folder

* replaces the unused variable deploy_op with _

* removes the unused Model import

* adds the costs section

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-08 09:29:37 -07:00
63031ed364 Updates ml_ops/stage2/get_started_vertex_featurestore notebook in community folder. (#426)
* updates get-started-featurestore notebook in mlops/stage2

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
2022-04-08 09:26:57 -07:00
93af43419c Updates Mlops/stage2/get-started-automl-training notebook in the community folder (#409)
* updates the get-started-automl-training notebook

* ran linter test

* adds --user flag during installation step

* ran linter test

* updates the clean up step

* ran linter test

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-04-07 13:03:29 -07:00
Krishna Chaitanya MovvaandGitHub 6445ed37c9 Updates the Mlops/stage2/ get-started-vertex-experiments notebook in the community folder (#410)
* updates the get-started-vertex-experiments notebook in the community folder

* ran linter test

* adds the costs section

* ran linter test
2022-03-25 10:52:21 -05:00
f806b4927b Adds updated Mlops/stage1/get-started-bq notebook to the community folder (#392)
* adds the updated mlops-stage1-get_started_bq_datasets notebook to the official branch and removes it from the community branch

* removes second instance of create_bigquery_dataset() function

* ran linter test successfully

* adds costs section

* ran linter test successfully

* updates the dependency installation step and GCS bucket explanation

* ran linter test

* adds pyarrow to the installations

* ran linter test

* removes unnecessary installations + adds silent install + moves the notebook back from official to community folder + adds IS_TESTING condition during clean-up

* ran linter test

* resolves the move up?? comment and builtin comment

* ran linter test

* updates textual content about package installation

* ran linter test

* resolves the future-tense and  dependency installations comments

* ran linter test

* updates the header according to template

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-03-24 14:16:51 -05:00
2a5d6650ee Adds updated Mlops/stage2/get-started-bqml-training notebook to the community folder (#397)
* adds the ml_ops/stage2/get_Started_bqml_training notebook to official and removes the same from community folder

* ran linter test

* updates the textual content

* ran linter test

* moves the updated stage2/get-started-bqml notebook back to the communit folder

* ran linter test

* updates the header according to the template

* ran linter test

Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-03-24 14:11:01 -05:00
ec6d508793 adds automl-text-sentiment-analysis-online notebook (#293)
* adds automl-text-sentiment-analysis-online notebook

* adds the cleaned up automl-text-sentiment-analysis notebook after running linter test

* adds textual content on what the dataset predicts in the dataset section

* ran the linter test after the update

* adds textual content on what the dataset predicts in the dataset section

* ran the linter test after the update

* corrects the IMPORT_FILE parameter in the notebook

* ran linter test after update

* deletes the source file from the community/sdk folder

* updates the colab, git & workbench links in the notebook

* ran linter test

* updates the license year to 2022 and simplifies the clean-up step for bucket-deletion

* ran linter test

* adds TESTING env condition while deleting the buckets

* ran linter test successfully

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-03-09 13:09:10 -06:00
5ecfc80cb9 Adds minor changes(license year and clean-up step) to Sdk automl video action recognition batch notebook (#355)
* adds the automl-video-action-recognition-notebook

* ran linter test

* fixes the dag variable by replacing with job

* ran linter test

* fixes the dag variable by replacing with job

* ran linter test

* corrects the IMPORT_FILE parameter in the notebook

* ran linter test after update

* updates the colab, git & vertex-ai links

* ran linter test

* updates the license year to 2022 and simplifies the lean-up step for bucket created

* ran linter test

* removes the file from the community folder

* adds the TESTING env condition while deleting the buckets

* ran linter test successfully

* adds TESTING env condition while deleting the bucket

* ran linter test successfully

Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com>
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com>
2022-03-04 12:44:39 -06:00