Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
7fd48364df
gristlabs_grist-core/app/common
History
Alex Hall 7fd48364df (core) Improved error messages, retries, and handling of token limits in AI assistant 
Summary:
In a nutshell:

- More specific and helpful error messages are shown to the user
- API requests are only retried when needed
- The system deals with reaching the maximum token limit better, especially by switching to a model with a bigger limit

In more detail:

- `COMPLETION_MODEL` configuration has been removed. By default `gpt-3.5-turbo-0613` is used which accepts 4k tokens. If that's not enough, `gpt-3.5-turbo-16k-0613` is used instead.
- Switching to the bigger model happens when either the prompt is too long by itself (the API immediately returns an error code) or the model reaches the 4k limit itself in the process of generating a response and thus returns an incomplete response. The latter case is made possible by removing the `max_tokens: 1500` in the request, which was very generous and would have lead to switching to the more expensive model more often than needed. The downside is that the user has to wait a bit longer for the response.
- If the bigger 16k token limit is also exceeded, the assistant immediately responds (instead of retrying as before) with an error message including suggestions. The suggestions include restarting the conversation if and only if the user has sent multiple messages.
- If a request fails because Grist has reached its OpenAI monthly billing quota, the assistant immediately responds (instead of retrying as before) with an error message suggesting that the user try again tomorrow.
- If a request fails for some other reason, the assistant retries, and if all attempts fail then the user is told to try again in a few minutes and is shown the exact error message, including the API response if there is one.
- Retrying only happens when an API request fails, whereas previously the system also retried errors from a much bigger scope which included calls to the sandbox. The downside is that the hugging face assistant no longer retries, although that code is currently disabled anyway.
- The assistant no longer waits an additional second after the final retry attempt fails.

Test Plan: Added a new server test file with several unit tests using faked OpenAI responses, including the happy path which wasn't really tested before.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3955
2023-07-18 16:01:37 +02:00
..
themes (core) Implement AI Assistant UI V2 2023-07-13 10:30:35 -04:00
ACLPermissions.ts (core) Implement checkbox for SchemaEdit permission in Access Rules UI. 2023-01-18 10:55:44 -05:00
ACLRuleCollection.ts (core) Implement checkbox for SchemaEdit permission in Access Rules UI. 2023-01-18 10:55:44 -05:00
ActionBundle.ts (core) deal with write access for attachments 2022-11-15 09:52:32 -05:00
ActionDispatcher.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ActionGroup.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
ActionRouter.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ActionSummarizer.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
ActionSummary.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
ActiveDocAPI.ts (core) Billing for formula assistant 2023-07-10 13:24:08 +02:00
AlternateActions.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
ApiError.ts (core) Billing for formula assistant 2023-07-10 13:24:08 +02:00
arrayToString.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
AssistancePrompts.ts (core) Improved error messages, retries, and handling of token limits in AI assistant 2023-07-18 16:01:37 +02:00
AsyncCreate.ts (core) add a python3 button 2021-09-16 10:06:04 -04:00
AsyncFlow.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
asyncIterators.ts (core) Add AzureExternalStorage 2022-05-09 21:44:57 +02:00
AttachmentColumns.ts (core) control the distribution of attachment metadata 2022-12-22 09:10:30 -05:00
BaseAPI.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
BasketClientAPI.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
BigInt.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
BillingAPI.ts (core) Billing for formula assistant 2023-07-10 13:24:08 +02:00
BinaryIndexedTree.js Correct spelling mistakes 2022-02-19 09:46:49 +00:00
BrowserSettings.ts (core) Simple localization support and currency selector. 2021-08-26 13:36:49 -07:00
CircularArray.js (core) Moving client and common tests to core 2022-08-23 19:20:10 +02:00
ColumnFilterFunc.ts (core) Fix date filter for DateTime columns. 2023-01-18 10:18:15 +01:00
ColumnGetters.ts (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
CommTypes.ts (core) deleting queue from single webhook 2023-07-18 11:46:10 +02:00
csvFormat.ts (core) Add our very own implementation of tokenfield (aka pillbox, tag list) 2021-05-03 17:16:24 -04:00
CustomWidget.ts (core) Widget options api 2022-01-13 11:10:17 +01:00
declarations.d.ts (core) Guess date format during type conversion 2022-02-21 22:39:47 +02:00
delay.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DisposableWithEvents.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DocActions.ts (core) Fix issue with lodash's map interpreting objects with length as array-like 2023-07-01 21:07:39 -04:00
DocData.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
DocDataCache.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
DocLimits.ts (core) Keep track of row counts per table 2022-08-03 08:13:33 -07:00
DocListAPI.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
DocSnapshot.ts (core) Add AzureExternalStorage 2022-05-09 21:44:57 +02:00
DocumentSettings.ts (core) add a python3 button 2021-09-16 10:06:04 -04:00
DocUsage.ts (core) Keep track of row counts per table 2022-08-03 08:13:33 -07:00
emails.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
EncActionBundle.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ErrorWithCode.ts (core) give more detailed reasons for access denied when memos are present 2021-02-15 17:02:24 -05:00
Features.ts (core) Billing for formula assistant 2023-07-10 13:24:08 +02:00
FilterState.ts (core) New date filter with a calendar view 2022-12-20 09:58:42 +01:00
Formula.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
getCurrentTime.ts (core) Adds dots menu to access rules page item 2023-01-12 09:40:45 +01:00
GranularAccessClause.ts (core) deal with write access for attachments 2022-11-15 09:52:32 -05:00
GristServerAPI.ts (core) Remove a bunch of dead code 2021-07-01 18:38:21 +02:00
gristTypes.ts (core) Comments 2022-10-17 13:38:21 +02:00
gristUrls.ts (core) Fixing cursor position for filtered linked section. 2023-07-07 19:04:30 +02:00
gutil.ts (core) Porting the AI evaluation script 2023-03-15 14:54:28 +01:00
InactivityTimer.ts (core) Speed up and upgrade build. 2022-06-27 16:10:10 -04:00
Install.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
InstallAPI.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
Interval.ts (core) Fix bugs with intervals 2022-08-25 12:38:36 -07:00
isHiddenTable.ts (core) Fixing cursor position for filtered linked section. 2023-07-07 19:04:30 +02:00
KeyedMutex.ts (core) fix sync to s3 when doc is marked as dirty but proves to be clean 2020-11-10 08:12:31 -05:00
KeyedOps.ts (core) limit retries of uploads to external store in tests 2022-06-06 16:19:41 -04:00
LocaleCodes.ts (core) Extending default locale list 2021-09-24 15:10:13 +02:00
Locales.ts Adding new flags for supported locales (#420) 2023-02-03 18:56:24 -05:00
LoginSessionAPI.ts (core) Improve dark mode 2023-04-12 01:58:48 -04:00
marshal.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
MemBuffer.js (core) move home server into core 2020-07-21 20:39:10 -04:00
NumberFormat.ts (core) Multi-column configuration 2022-10-17 09:51:19 +02:00
NumberParse.ts (core) Guess numeric formatting options 2022-03-03 21:32:03 +02:00
orgNameUtils.ts (core) Add optional telemetry to grist-core 2023-06-07 12:00:51 -04:00
parseDate.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
plugin.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
PluginInstance.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Prefs.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
RecentItems.js (core) Moving client and common tests to core 2022-08-23 19:20:10 +02:00
RefCountMap.ts (core) Fix an insidious bug in RefCountMap, manifesting as JS errors some time after import. 2020-11-06 09:24:56 -05:00
RelativeDates.ts (core) Set DateTime timezone during xlsx import 2023-05-24 11:39:49 -04:00
resetOrg.ts (core) Show usage banners in doc menu of free team sites 2022-05-26 15:01:35 -07:00
roles.ts (core) Add tip for "Add New" button 2023-01-16 16:50:42 -08:00
RowFilterFunc.ts (core) Download as CSV button on sections 2021-05-27 15:48:12 +02:00
schema.ts (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
ShareAnnotator.ts (core) Avoid flagging support user as collaborator 2022-12-14 01:23:49 -05:00
sharing.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
SortFunc.ts (core) Fix the empty values last option in multicol sorting 2023-01-31 12:14:22 +01:00
SortSpec.ts (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
StringUnion.ts (core) Allow configuring (mostly hiding) various little bits of UI 2022-05-27 14:32:05 +02:00
TableData.ts (core) Fixing cursor position for filtered linked section. 2023-07-07 19:04:30 +02:00
TabularDiff.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
tagManager.ts (core) Record new user sign-ups 2022-03-12 14:34:46 -08:00
tbind.ts (core) Configure more comprehensive eslint rules for Typescript 2021-04-26 18:54:55 -04:00
Telemetry.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
TestState.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ThemePrefs-ti.ts (core) Implement AI Assistant UI V2 2023-07-13 10:30:35 -04:00
ThemePrefs.ts (core) Implement AI Assistant UI V2 2023-07-13 10:30:35 -04:00
Themes.ts (core) Add dark mode to user preferences 2022-09-05 19:17:32 -07:00
timeFormat.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
tpromisified.ts (core) Configure more comprehensive eslint rules for Typescript 2021-04-26 18:54:55 -04:00
Triggers-ti.ts (core) API reworked to use POST to create webhook and DELET to remove it 2023-07-14 15:01:46 +02:00
Triggers.ts (core) API reworked to use POST to create webhook and DELET to remove it 2023-07-14 15:01:46 +02:00
tsconfig.json (core) show package.json version when hovering on Grist icon in grist-core 2022-10-12 16:02:01 -04:00
tsvFormat.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
uploads.ts (core) Extending Google Drive integration scope 2021-10-01 10:47:12 +02:00
urlUtils.ts (core) remove metrics 2020-09-29 18:57:56 -04:00
UserAPI.ts (core) deleting queue from single webhook 2023-07-18 11:46:10 +02:00
UserConfig.ts (core) Remove REPL code 2021-07-20 15:17:03 +02:00
ValueConverter.ts (core) Implement exported functions without relying on ActiveDoc.docData 2023-06-07 22:30:01 +02:00
ValueFormatter.ts (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
ValueGuesser.ts (core) Move guessing logic for column types to run in node once for all columns. 2022-05-19 12:49:51 -04:00
ValueParser.ts (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
WidgetOptions.ts Export table schema (#459) 2023-03-16 17:37:24 -04:00