Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2025-06-13 20:53:59 +00:00
Code Issues Projects Releases Wiki Activity
ea8a59c5e9
gristlabs_grist-core/sandbox/grist
History
Alex Hall bb7cf6ba20 (core) Modify prompt so that model may say it cannot help with certain requests. 
Summary:
This tweaks the prompting so that the user's message is given on its own instead of as a docstring within Python. This is so that the prompt makes sense when:

- the user asks a question such as "Can you write me a formula which does ...?" rather than describing their formula as a docstring would, or
- the user sends a message that doesn't ask for a formula at all (https://grist.slack.com/archives/C0234CPPXPA/p1687699944315069?thread_ts=1687698078.832209&cid=C0234CPPXPA)

Also added wording for the model to refuse when the user asks for something that the model cannot do.

Because the code (and maybe in some cases the model) for non-ChatGPT models relies on the prompt consisting entirely of Python code produced by the data engine (which no longer contains the user's message) those code paths have been disabled for now. Updating them now seems like undesirable drag, I think it'd be better to revisit this when iteration/experimentation has slowed down and stabilised.

Test Plan:
Added entries to the formula dataset where the response shouldn't contain a formula, indicated by the value `1` for the new column `no_formula`.

This is somewhat successful, as the model does refuse to help in some of the new test cases, but not all. Performance on existing entries also seems a bit worse, but it's hard to distinguish this from random noise. Hopefully this can be remedied in the future with more work, e.g. automatic followup messages containing example inputs and outputs.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3936
2023-06-27 15:57:56 +02:00
..
functions (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
imports (core) Fix issue that ints would be imported with a trailing ".0" from Google Sheets. 2023-02-26 15:24:15 -05:00
acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
acl.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
action_obj.py (core) External requests 2022-06-17 21:53:20 +02:00
action_summary.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
actions.py (core) AddOrUpdateRecord user action 2022-02-03 16:22:51 -05:00
autocomplete_context.py (core) Show example values in formula autocomplete 2022-09-28 19:42:36 +02:00
codebuilder.py (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add 2023-04-25 15:28:40 -04:00
column.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
csv_patch.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
depend.py (core) Implement trigger formulas (generalizing default formulas) 2021-06-25 22:53:07 -04:00
docactions.py (core) Undo bug - restoring dependencies for trigger formulas 2021-12-06 16:09:41 +01:00
docmodel.py (core) Comments 2022-10-17 13:38:21 +02:00
engine.py (core) Use fake stdout/stderr when evaluating formulas 2023-06-14 18:24:27 -04:00
fake_std_streams.py (core) Use fake stdout/stderr when evaluating formulas 2023-06-14 18:24:27 -04:00
formula_prompt.py (core) Modify prompt so that model may say it cannot help with certain requests. 2023-06-27 15:57:56 +02:00
friendly_errors.py (core) Upgrade friendly-traceback and its dependencies 2022-10-28 14:44:14 +02:00
gencode.py (core) Error explanations from friendly-traceback 2022-08-12 19:45:00 +02:00
grist.py (core) Use MixedTypesKey for sort_by arg of lookupRecords to avoid errors in Python 3 2021-11-09 18:08:29 +02:00
identifiers.py (core) Remove accents when picking identifiers 2021-08-24 14:47:50 +02:00
import_actions.py (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
logger.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
lookup.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
main.py (core) Porting back AI formula backend 2023-02-08 17:15:59 +01:00
match_counter.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
migrations.py (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
moment.py (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
objtypes.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
parse_data.py (core) Fix issue that ints would be imported with a trailing ".0" from Google Sheets. 2023-02-26 15:24:15 -05:00
records.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
relabeling.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
relation.py (core) Polish and enable Reference List widget 2021-08-12 11:51:21 -07:00
runtests.py python(tests): print logging when running tests in verbose mode 2022-09-20 17:22:28 +02:00
sandbox.py add a pyodide-based "sandbox" flavor (#437) 2023-03-06 16:56:25 -05:00
schema.py (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
sort_specs.py (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
summary.py (core) Fix issue with summary update. 2023-03-15 13:54:15 +01:00
table_data_set.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
table.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
test_acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
test_acl_renames.py (core) Update ACL resources/rules when tables/columns get renamed 2021-01-04 22:03:01 -05:00
test_actions.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_codebuilder.py (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add 2023-04-25 15:28:40 -04:00
test_column_actions.py (core) Nice summary table IDs 2022-07-14 12:09:56 +02:00
test_completion.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
test_default_formulas.py (core) Fix lookups in default formulas 2020-09-10 18:06:06 -04:00
test_depend.py (core) Fix recursion error in invalidate_deps 2021-06-18 15:42:20 +02:00
test_derived.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
test_display_cols.py (core) Automatically remove empty summary table rows 2022-07-08 18:56:41 +02:00
test_docmodel.py (core) Backend restrictions for raw data widgets 2022-02-15 22:04:32 +02:00
test_engine.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
test_find_col.py Correct spelling mistakes 2022-02-19 09:46:49 +00:00
test_formula_error.py (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add 2023-04-25 15:28:40 -04:00
test_formula_prompt.py (core) Modify prompt so that model may say it cannot help with certain requests. 2023-06-27 15:57:56 +02:00
test_formula_undo.py (core) Mark column changing actions as indirect when adding data to empty column 2022-05-25 16:21:04 +02:00
test_functions.py (core) Don't swallow TypeErrors in functions like SUM 2022-10-25 12:15:13 +02:00
test_gencode.py (core) Error explanations from friendly-traceback 2022-08-12 19:45:00 +02:00
test_import_actions.py (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
test_logger.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_lookups.py (core) Respect sort_by in lookupOne, and allow reverse sorting 2022-10-24 20:10:08 +02:00
test_match_counter.py (core) Import Hashable from six.moves.collections_abc 2022-02-22 00:27:51 +02:00
test_migrations.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_moment.py (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
test_record_func.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_recordlist.py (core) Return RecordSet instead of list from property access when possible, to allow further property access 2022-04-05 18:05:00 +02:00
test_reflist_rel.py (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
test_relabeling.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
test_renames2.py (core) Fix bug with renaming when a formula uses a local name for a user table. 2023-04-02 22:57:38 -04:00
test_renames.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
test_replay.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
test_requests.py (core) External requests 2022-06-17 21:53:20 +02:00
test_rules_grid.py (core) Implementing row conditional formatting 2022-08-09 20:11:36 +02:00
test_rules.py (core) Updating RawData views 2022-05-04 21:41:42 +02:00
test_side_effects.py (core) Ensure formulas return something and don't assign to attributes of rec 2022-05-23 21:06:18 +02:00
test_sort_spec.py (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
test_summary2.py (core) Fix issue with summary update. 2023-03-15 13:54:15 +01:00
test_summary_choicelist.py (core) Test undo for all Python summary table tests 2022-09-07 18:02:02 +02:00
test_summary_undo.py (core) Nice summary table IDs 2022-07-14 12:09:56 +02:00
test_summary.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
test_table_actions.py (core) Nice summary table IDs 2022-07-14 12:09:56 +02:00
test_table_data_set.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_temp_rowids.py (core) Allow using negative rowIds to add records and refer to them in Reference values. 2020-12-15 16:39:56 -05:00
test_textbuilder.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
test_treeview.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_trigger_formulas.py (core) Restoring separated transform columns 2023-01-03 17:54:16 +01:00
test_twowaymap.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
test_types.py (core) Nicer conversion from numeric to text 2021-09-30 14:16:24 +02:00
test_undo.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
test_urllib_patch.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
test_user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
test_useractions.py (core) Fix bug where duplicating a table would cause an error when a column had mismatching label and colId 2023-03-24 08:29:32 -04:00
testsamples.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
testscript.json (core) Updating RawData views 2022-05-04 21:41:42 +02:00
testutil.py (core) Implement trigger formulas (generalizing default formulas) 2021-06-25 22:53:07 -04:00
textbuilder.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
treeview.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
twowaymap.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00
tzdata.data (core) apply some dependabot suggestions 2022-09-07 14:15:34 -04:00
urllib_patch.py (core) Remaining Python 3 compatibility changes 2021-06-25 12:00:58 +02:00
user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
useractions.py (core) Implement exported functions without relying on ActiveDoc.docData 2023-06-07 22:30:01 +02:00
usercode.py (core) Raise syntax errors that Python can format nicely to show the location 2021-09-24 15:07:09 +02:00
usertypes.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
xmlrunner.py (core) Run python unit tests again in python 3 2021-06-30 21:46:18 +02:00