spoffy/fix-import-error-s3-no-redis
dependabot/npm_and_yarn/express-4.20.0
latest_candidate
main
spoffy/webdriver-logs
dependabot/npm_and_yarn/webpack-5.94.0
dependabot/npm_and_yarn/dompurify-3.1.3
latest
dependabot/npm_and_yarn/elliptic-6.5.7
dependabot/npm_and_yarn/axios-1.7.4
dependabot/npm_and_yarn/micromatch-4.0.8
berhalak/build-test
ignore-alert
link-to-issue-templates
spoffy/rename-candidate-action-job
dependabot/npm_and_yarn/fast-xml-parser-4.4.1
spoffy/playwright
spoffy/grist-ee-defaults
dependabot/npm_and_yarn/ws-8.17.1
dependabot/npm_and_yarn/tar-6.2.1
dependabot/npm_and_yarn/braces-3.0.3
jordigh/native-arm64
paulfitz/preview
paulfitz/smoosh
test-server-reset
dsagal-readme-gvisor
readme-update-dec2023
paulfitz/bundle-widget-prep
jv-linkstate-bubbles-tooltips
jv-linkstate-bubbles-base
jv-bidirectional-tests
preview
bidirectional
chainlink-fix
alex/skip-fstrings-3.9
alex/upgrade-pyodide
alex/3.11-tests
alex/_importParsedFileAsNewTable
poc-engine-data-layer
poc-engine
sponsors-section
removing-missing-key-error
friendly-locale
messytables-requirements
add-page-name
markdown-cells
v1.1.12
v1.1.11
v1.1.10
v1.1.9
v1.1.8
v1.1.7
v1.1.6
v1.1.5
v1.1.4
v1.1.3
v1.1.2
v1.1.1
v1.1.0
v1.0.9
v1.0.8
v1.0.7
v1.0.6
v1.0.5
v1.0.4
v1.0.3
v0.7.9
v0.7.8
v0.7.7
v0.7.6
v0.7.5
v0.7.4
v0.7.1
v0.7.2
v0.7.3
v1.1.13
v1.1.14
v1.1.15
v1.1.16
v1.1.17
v1.1.18
${ noResults }
2 Commits (4023e8b3069ecd0952fca88a5042c3fd1c2e6b53)
Author | SHA1 | Message | Date |
---|---|---|---|
Alex Hall | 52469c5a7e |
(core) Improve parsing formula from completion
Summary: The previous code for extracting a Python formula from the LLM completion involved some shaky string manipulation which this improves on. Overall the 'test results' from `runCompletion` went from 37/47 to 45/47 for `gpt-3.5-turbo-0613`. The biggest problem that motivated these changes was that it assumed that code was always inside a markdown code block (i.e. triple backticks) and so if there was no block there was no code. But the completion often consists of *only* code with no accompanying explanation or markdown. By parsing the completion in Python instead of JS, we can easily check if the entire completion is valid Python syntax and accept it if it is. I also noticed one failure resulting from the completion containing the full function (instead of just the body) and necessary imports before that function instead of inside. The new parsing moves import inside. Test Plan: Added a Python unit test Reviewers: paulfitz Reviewed By: paulfitz Subscribers: paulfitz Differential Revision: https://phab.getgrist.com/D3922 |
1 year ago |
Cyprien P | 1ff93f89c2 |
(core) Porting the AI evaluation script
Summary: Porting script that run an evaluation against our formula dataset. To test you need an openai key (see here: https://platform.openai.com/) or hugging face (it should work as well), then checkout the branch and run `OPENAI_API_KEY=<my_openai_api_key> node core/test/formula-dataset/runCompletion.js` Test Plan: Needs manually testing: so far there is no plan to make it part of CI. The current score is somewhere around 34 successful prompts over a total of 47. Reviewers: paulfitz Reviewed By: paulfitz Subscribers: jarek Differential Revision: https://phab.getgrist.com/D3816 |
2 years ago |