Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
a1d31e41ad
gristlabs_grist-core/sandbox/grist
History
Alex Hall a1d31e41ad (core) Prevent the AI assistant response from including class definitions 
Summary:
Sometimes the model repeats the classes given in the prompt which would mess up extracting the actual formula. This diff solves this by:

1. Changes the generated Python schema so that (a) the thing that needs completing is a plain top level function instead of a property/method inside the class and (2) the classes are fully valid syntax, which makes it easier to
2. Remove classes from the parsed Python code when converting the completion to a formula.
3. Tweak the prompt wording to discourage including classes in general, especially because sometimes the model tries to solve the problem by defining extra methods/attributes/classes.

While I was at it, I changed type hints to use builtins (e.g. `list` instead of `List`) to prevent `from typing import List` which was happening sometimes and would look weird in a formula. Similarly I removed `@dataclass` since that also implies an import, and this also fits with the tweaked wording that the classes are fake.

Test Plan:
Added a new test case to the formula dataset which triggers the unwanted behaviour. The factors that seem to trigger the problem are (1) a small schema so the classes are easier to repeat and (2) the need to import modules, which the model wants to place before all other code. The case failed before this diff and succeeded after. The tweaked wording reduces the chances of repeating the classes but didn't eliminate it, so forcibly removing the classes in Python was needed.

There were also a couple of other existing cases where repeating the classes was observed before but not after.

Overall the score increased from 49 to 51 out of 69 (including the new case). At one point the score was 53, but changes in whitespace were enough to make it drop again.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4000
2023-08-18 12:50:09 +02:00
..
functions REQUEST with unknown and undetectable encoding (#600) 2023-08-02 16:49:37 -04:00
imports (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
acl.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
action_obj.py (core) External requests 2022-06-17 21:53:20 +02:00
action_summary.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
actions.py (core) AddOrUpdateRecord user action 2022-02-03 16:22:51 -05:00
attribute_recorder.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
autocomplete_context.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
codebuilder.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
column.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
csv_patch.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
depend.py (core) Implement trigger formulas (generalizing default formulas) 2021-06-25 22:53:07 -04:00
docactions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
docmodel.py (core) Comments 2022-10-17 13:38:21 +02:00
engine.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
fake_std_streams.py (core) Use fake stdout/stderr when evaluating formulas 2023-06-14 18:24:27 -04:00
formula_prompt.py (core) Prevent the AI assistant response from including class definitions 2023-08-18 12:50:09 +02:00
friendly_errors.py (core) Upgrade friendly-traceback and its dependencies 2022-10-28 14:44:14 +02:00
gencode.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
grist.py (core) Use MixedTypesKey for sort_by arg of lookupRecords to avoid errors in Python 3 2021-11-09 18:08:29 +02:00
identifiers.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
import_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
lookup.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
main.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
match_counter.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
migrations.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
moment.py (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
objtypes.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
parse_data.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
records.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
relabeling.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
relation.py (core) Polish and enable Reference List widget 2021-08-12 11:51:21 -07:00
runtests.py python(tests): print logging when running tests in verbose mode 2022-09-20 17:22:28 +02:00
sandbox.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
schema.py (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
sort_specs.py (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
summary.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
table_data_set.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
table.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
test_acl_renames.py (core) Update ACL resources/rules when tables/columns get renamed 2021-01-04 22:03:01 -05:00
test_actions.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_codebuilder.py (core) Upgrade to latest versions of asttokens and wrapt 2023-07-26 21:48:21 +02:00
test_column_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_completion.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_default_formulas.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_depend.py (core) Fix recursion error in invalidate_deps 2021-06-18 15:42:20 +02:00
test_derived.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_display_cols.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_docmodel.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_engine.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_find_col.py Correct spelling mistakes 2022-02-19 09:46:49 +00:00
test_formula_error.py (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add 2023-04-25 15:28:40 -04:00
test_formula_prompt.py (core) Prevent the AI assistant response from including class definitions 2023-08-18 12:50:09 +02:00
test_formula_undo.py (core) Mark column changing actions as indirect when adding data to empty column 2022-05-25 16:21:04 +02:00
test_functions.py (core) Don't swallow TypeErrors in functions like SUM 2022-10-25 12:15:13 +02:00
test_gencode.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_import_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_lookups.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_match_counter.py (core) Import Hashable from six.moves.collections_abc 2022-02-22 00:27:51 +02:00
test_migrations.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_moment.py (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
test_record_func.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_recordlist.py (core) Return RecordSet instead of list from property access when possible, to allow further property access 2022-04-05 18:05:00 +02:00
test_reflist_rel.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_relabeling.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_renames2.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_renames.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_replay.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
test_requests.py REQUEST with unknown and undetectable encoding (#600) 2023-08-02 16:49:37 -04:00
test_rules_grid.py (core) Implementing row conditional formatting 2022-08-09 20:11:36 +02:00
test_rules.py (core) Updating RawData views 2022-05-04 21:41:42 +02:00
test_side_effects.py (core) Ensure formulas return something and don't assign to attributes of rec 2022-05-23 21:06:18 +02:00
test_sort_spec.py (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
test_summary2.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary_choicelist.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary_undo.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_table_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_table_data_set.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_temp_rowids.py (core) Allow using negative rowIds to add records and refer to them in Reference values. 2020-12-15 16:39:56 -05:00
test_textbuilder.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_treeview.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_trigger_formulas.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_twowaymap.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_types.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_undo.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
test_urllib_patch.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
test_user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
test_useractions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
testsamples.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
testscript.json (core) Updating RawData views 2022-05-04 21:41:42 +02:00
testutil.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
textbuilder.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
treeview.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
twowaymap.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
tzdata.data (core) apply some dependabot suggestions 2022-09-07 14:15:34 -04:00
urllib_patch.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
useractions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
usercode.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
usertypes.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
xmlrunner.py (core) Run python unit tests again in python 3 2021-06-30 21:46:18 +02:00