Andrew Ferlitsch and GitHub
c60e8d1e3d
feat: add build to generate web index of official notebooks ( #1971 )
...
* feat: add build to generate web index of official notebooks
* fix: review comments
* fix: store results
2023-06-13 20:08:04 +00:00
Andrew Ferlitsch and GitHub
6e4fb5396a
tune: AR repeaper ( #1935 )
2023-06-06 17:15:16 +00:00
Andrew Ferlitsch and GitHub
3d9a462528
feat: delete AR repos ( #1932 )
2023-06-05 21:31:51 +00:00
Andrew Ferlitsch and GitHub
b002e7ba2c
tune: selecting notebooks ( #1908 )
...
* tune: selecting notebooks
* test: add test for select_notebook
* fix: random algo
* test: convert to pytest
* fix: review
* fix: remove path ..
* fix: assert
2023-06-01 21:34:14 +00:00
Andrew Ferlitsch and GitHub
c94be301cb
feat: results viewer ( #1905 )
...
* feat: results viewer
* tune: comment
2023-05-30 15:21:23 +00:00
Andrew Ferlitsch and GitHub
6ea28621ac
feat: read accumulative result files ( #1837 )
...
* feat: read accumulative result files
* fix: build results dir
* fix: build results dir
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: selection algo
* fix: debug
* fix: debug
* fix: debug
* fix: debug
* fix: debug
* fix: debug
* fix: debug select algo
* fix: debug select algo
* fix: debug select algo
* fix: debug select algo
* fix: filtering
* fix: review comments
* fix: rm test_filter
2023-05-12 17:44:54 +00:00
Andrew Ferlitsch and GitHub
c638e6d983
fix: add test percent param ( #1821 )
2023-05-05 23:07:13 +00:00
Andrew Ferlitsch and GitHub
6195e7bbf9
Ci cd 2 ( #1802 )
...
* update: record tallied results to GCS bucket
* fix: test percent
* update: accumulator support
* fix: use args for where to store results
* fix: args for results file
* fix: gs bucket ptrfix
* fix: tuning args
* fix: ci/cd test on openin artifacts bucket
* fix: 2nd try at bucket issue
* fix: 2nd try at bucket issue
* debug: bucket issue
* debug: bucket issue
* debug: bucket issue
* debug: bucket issue
* debug: bucket issue
* debug: bucket issue
* debug: no entries written
* debug: no entries written
* debug: not accumulating
* debug: not accumulating
* debug: not accumulating
* debug: not accumulating
* Update requirements.txt
* debug: not accumulating
* debug: not accumulating
* debug: matching notebook name
* debug: pandas problem
* debug: import issues
* debug: load
* Update requirements.txt
* Update requirements.txt
* debug: read csv
* debug: read csv
* debug: read csv
* debug: indxer
* debug: indexer
* debug: accum
* debug: accum
* debug: accum
* debug: accum
* debug: accum
* debug: accum
* debug: accum
* debug: casting
* debug: casting
* debug: casting
* debug: casting
* debug: duration nit
* debug: duration nit
* fix: flaky
* fix: BUILD_ID
* fix: BUILD_ID
* fix: BUILD_ID
* fix: BUILD_ID
* fix: BUILD_ID
* fix: BUILD_ID
* fix: BUILD_ID
* fix: review
* fix: review
* fix: review
* fix: biz logic
* fix: biz logic
* fix: review
* fix: build_id required
* fix: use json format
* review: JSON simplifing
* review: JSON simplifing
* update: NotbookExecutionResult updates
* fix: revert to JSON
* update: use util to read from bucket
* fix: remove pandas inmport
2023-05-05 20:03:49 +00:00
Andrew Ferlitsch and GitHub
fd3cc2a2cc
update: test_percent and dry_run for weekly regr ( #1795 )
...
* update: test_percent for weekly regr
* update: add dry_run and test_percent
* fix: review comment
2023-04-24 21:41:21 +00:00
Andrew Ferlitsch and GitHub
cb41dfc8bd
test: flaky script ( #1794 )
2023-04-22 14:53:19 +00:00
Andrew Ferlitsch and GitHub
9b5e734dca
test: list of passing notebooks ( #1785 )
...
* test: list of passing notebooks
* test: list of failing notebooks
2023-04-21 19:17:43 +00:00
Andrew Ferlitsch and GitHub
40a31daaef
Revert "update: thread safe limiter ( #1718 )" ( #1724 )
...
This reverts commit 37ba322412
2023-04-12 16:33:16 +00:00
Andrew Ferlitsch and GitHub
37ba322412
update: thread safe limiter ( #1718 )
...
* update: thread safe limiter
* update: thread safe limiter
* fix: missing install
2023-04-12 00:05:36 +00:00
Andrew Ferlitsch and GitHub
770508b4f0
fix: delete experiment and fs ( #1709 )
...
* fix: delete experiment and fs
* fix False typo
* fix: 2nd try at Falsee typo
* fix: update_time
2023-04-10 21:56:20 +00:00
Andrew Ferlitsch and GitHub
c8f7b910f2
fix: add back missing not ( #1687 )
2023-04-05 22:48:40 +00:00
Andrew Ferlitsch and GitHub
78b2aa87f3
Cleanup bucket ( #1682 )
...
* fix: cleanup buckets
* fix: review comments
* fix: review comments
* fix: delete only vertex notebook testing buckets
* fix: fine tune
* fix: fine tune
2023-04-03 22:38:31 +00:00
Andrew Ferlitsch and GitHub
0b9582341c
fix: cleanup buckets ( #1679 )
...
* fix: cleanup buckets
* fix: review comments
* fix: review comments
* fix: delete only vertex notebook testing buckets
* fix: fine tune
2023-04-03 21:29:35 +00:00
Andrew Ferlitsch and GitHub
7bd3814dd9
quotas still exceeded, reduce rate limit ( #1661 )
2023-03-30 17:03:53 +00:00
Andrew Ferlitsch and GitHub
3e0a6634a6
further lower rate limit
2023-03-21 15:03:37 -07:00
Andrew Ferlitsch
b88a775d33
fix: lower rate limit
2023-03-15 15:52:39 +00:00
Andrew Ferlitsch
0a7a2f6eeb
fix: add more cleanup
2023-03-14 20:06:07 +00:00
ivanmkc@google.com
73fbc762fe
Added tqdm to requirements.txt
2023-03-06 20:00:53 -05:00
9a6c36d016
Fixed cleanup code for matching engine index endpoint ( #1493 )
...
Co-authored-by: ivanmkc@google.com <ivanmkc@google.com >
2023-01-30 09:52:32 -05:00
Andrew Ferlitsch and GitHub
0cd6146a6c
fix: restore requirements.txt ( #1468 )
2023-01-13 09:52:24 -08:00
Andrew Ferlitsch and GitHub
b5d19719e0
Update requirements.txt
...
The <2.11 syntax does not work, since it is interpreted as I/O redirection on the command line.
2023-01-05 12:23:28 -08:00
59536e9e61
Fix google-api-core version to last known working version ( #1402 )
...
Co-authored-by: Ivan Cheung <ivanmkc@google.com >
2023-01-05 10:30:48 -08:00
9b5be742c1
Fixed timeout value ( #1370 )
...
Co-authored-by: Ivan Cheung <ivanmkc@google.com >
2022-12-19 11:41:02 -05:00
Andrew Ferlitsch and GitHub
3d19ffb131
fix: timeout issue for notebook test ( #1365 )
2022-12-16 18:27:09 -08:00
8c265ee1ef
Added matching engine resource managers ( #1314 )
...
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com >
2022-12-01 13:41:23 -08:00
Ivan Cheung and GitHub
6ccad69a18
Reduce boilerplate for custom tabular bq notebook ( #1207 )
...
* Reduced boilerplate
Ran linter and fixed install
Added pyarrow
Fixed pip install
Reverted unneeded changes
Fixed pip install
Linted code and added missing preprocessor call
Fixed preprocessor
Default to us-central1
* Ran linter
* Simplified service account section
2022-11-10 17:47:13 -05:00
Ivan Cheung and GitHub
35d71b410e
Boilerplate reduction: Notebook template ( #1204 )
...
* Reduced notebook boilerplate
* Fixed lint issues
* Added unique suffix note
* Added unique string processor and moved tests to own folder
* Fixed broken link
* Added message about updating links
* Fixed typo
* Added missing import
* Added back useful instructions
* Addressed comments
* Removed matching engine
* Fixed comments
2022-11-08 14:10:50 -05:00
f184411666
Create a function to get notebook python version for execution ( #828 )
...
* Create a function to get notebook python version for execution
* Inject python version to yaml file
* Fix python version references
* Add python string to python version variable
* add python version extraction script (untested)
* Create a function to get notebook python version for execution
* Inject python version to yaml file
* Fix python version references
* Add python string to python version variable
* Remove one notebook condition
* Remove extra check and use python 3 as default version
* Use python3.9 as default version
* Update python version notebook parser
* Add python version to the notebook template
* Fix bug
* Update python version parser function
* Add python version to a notebook for testing
* Run linter
* Use regex in python version parser function
* Add new notebook for testing
* Use better variable name
* fix typo
* Use f string instead +
* Use python from env instead of using _PYTHON_VERSION
* Use simpler regex
* Add python version test notebook
* Fixed a mistake
* Updated notebook template with python version
* Fixed python version format
* Print log contents to stdout
* Remove failing notebook
* Run linter
* Remove extra steps in the printed log
* Edit comments
* Run linter
* Run linter
* Revert test changes
Co-authored-by: AG Sol <aarongabriel@google.com >
2022-09-26 14:32:53 -07:00
Ivan Cheung and GitHub
84fd10f408
fix: Fixed kernel spec by forcing it to use python3 ( #889 )
2022-08-19 18:43:07 -07:00
2885a7a70f
Migrated matching engine notebook to official and added VPC network support to CI ( #836 )
...
* Added matching engine notebook official
* Ran linter
* Added matching engine to .cloud-build/test_notebook_vm.txt
Co-authored-by: Andrew Ferlitsch <aferlitsch@google.com >
2022-08-11 18:30:22 -07:00
Soheila Zangeneh and GitHub
8db9ce0415
Move download link up ( #813 )
2022-08-04 15:05:18 -04:00
Ivan Cheung and GitHub
8c7fb38136
Added missing dependency ( #814 )
2022-08-04 14:53:17 -04:00
Soheila Zangeneh and GitHub
710e6ea0d8
Run commands in quiet mode ( #789 )
...
* Run commands in quiet mode in both yaml files
* Test CI against single and multiple notebooks
2022-08-04 11:00:49 -04:00
4dce246b57
Add single notebook test file and remove broken notebook from the list ( #801 )
...
Co-authored-by: Ivan Cheung <ivans.mailbox@gmail.com >
2022-08-04 10:39:09 -04:00
Ivan Cheung and GitHub
2d6afa8313
Added better download instructions ( #738 )
...
* Improve download instruc
tions
* Added web download to notebook results list
2022-07-22 18:21:05 -04:00
Ivan Cheung and GitHub
00cfb20c36
Notebook CI: Fixed variable replacement to handle numbers ( #739 )
...
* Removed replacement of variable comparisons
* Fixed issue with numbers in replacement content
2022-07-22 11:50:39 -04:00
Ivan Cheung and GitHub
14c87ae702
Removed replacement of variable comparisons ( #737 )
2022-07-21 19:21:02 -04:00
Ivan Cheung and GitHub
145cdd0928
Added a service account injection ( #695 )
...
* Added a service account injection
* Revert this
* Added service account injection
* Fixed cloud build file
* Added gcloud version debug info
* Fixed sa injection
* Removed test file
* Revert CODEOWNERS
2022-07-21 15:04:10 -04:00
Ivan Cheung and GitHub
37c52abc11
Added a virtualenv to the notebook execution tests which should help with dependency issues ( #693 )
...
* Added a virtualenv
* Added a virtualenv for the single yaml
* Debug
* Fixed diff logic for all notebooks case
* Added missing -c
2022-06-26 13:31:52 -04:00
dffdb15c17
Add default kernel_name for notebook execution ( #497 )
...
Currently, there is no kernel_name. Hence, the execution test cannot run for notebooks that don't have kernels defined in their .ipynb file.
Side-note: We should use lint to remove the kernel_name from .ipynb as well, as it could include info specific to the author's environment.
Co-authored-by: Karl Weinmeister <11586922+kweinmeister@users.noreply.github.com >
2022-06-23 19:18:49 -04:00
Ivan Cheung and GitHub
881a2b45c5
Improved git diff logic ( #687 )
2022-06-22 21:49:09 -04:00
Ivan Cheung and GitHub
8f0c73b32c
Fixed cleanup scripts ( #676 )
2022-06-21 20:13:37 -07:00
Karl Weinmeister and GitHub
81b70e779c
ci: Remove protobuf from requirements.txt
2022-06-19 09:38:45 -05:00
Karl Weinmeister and GitHub
085713a818
ci: Add protobuf to requirements.txt
2022-06-18 17:20:45 -05:00
Ivan Cheung and GitHub
0250879f65
Added notebook substitution for common case of 1 notebook ( #625 )
2022-06-10 14:44:43 -04:00
Ivan Cheung and GitHub
f730d6b9de
Added ability to test a single notebook ( #608 )
...
* Added ability to test a single notebook
* Added output_url to table
* Removed ML Ops notebooks
2022-06-06 10:30:31 -04:00
