Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
7fd48364df
gristlabs_grist-core/app/server/lib
History
Alex Hall 7fd48364df (core) Improved error messages, retries, and handling of token limits in AI assistant 
Summary:
In a nutshell:

- More specific and helpful error messages are shown to the user
- API requests are only retried when needed
- The system deals with reaching the maximum token limit better, especially by switching to a model with a bigger limit

In more detail:

- `COMPLETION_MODEL` configuration has been removed. By default `gpt-3.5-turbo-0613` is used which accepts 4k tokens. If that's not enough, `gpt-3.5-turbo-16k-0613` is used instead.
- Switching to the bigger model happens when either the prompt is too long by itself (the API immediately returns an error code) or the model reaches the 4k limit itself in the process of generating a response and thus returns an incomplete response. The latter case is made possible by removing the `max_tokens: 1500` in the request, which was very generous and would have lead to switching to the more expensive model more often than needed. The downside is that the user has to wait a bit longer for the response.
- If the bigger 16k token limit is also exceeded, the assistant immediately responds (instead of retrying as before) with an error message including suggestions. The suggestions include restarting the conversation if and only if the user has sent multiple messages.
- If a request fails because Grist has reached its OpenAI monthly billing quota, the assistant immediately responds (instead of retrying as before) with an error message suggesting that the user try again tomorrow.
- If a request fails for some other reason, the assistant retries, and if all attempts fail then the user is told to try again in a few minutes and is shown the exact error message, including the API response if there is one.
- Retrying only happens when an API request fails, whereas previously the system also retried errors from a much bigger scope which included calls to the sandbox. The downside is that the hugging face assistant no longer retries, although that code is currently disabled anyway.
- The assistant no longer waits an additional second after the final retry attempt fails.

Test Plan: Added a new server test file with several unit tests using faked OpenAI responses, including the happy path which wasn't really tested before.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3955
2023-07-18 16:01:37 +02:00
..
AccessTokens.ts (core) add an access token mechanism to help with attachments in custom widgets 2022-07-19 11:55:18 -04:00
ACLFormula.ts (core) add OWNERS='owners', EDITOR='editors', VIEWER='viewers' to condition formulas 2021-03-19 18:20:33 -04:00
ActionHistory.ts (core) Fix issue with lodash's map interpreting objects with length as array-like 2023-07-01 21:07:39 -04:00
ActionHistoryImpl.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
ActiveDoc.ts (core) deleting queue from single webhook 2023-07-18 11:46:10 +02:00
ActiveDocImport.ts Split out new importFileAsNewTable method for grist-static (#564) 2023-07-12 15:57:02 +02:00
AppEndpoint.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
AppSettings.ts (core) give preliminary support in core for storing snapshots in S3-compatible stores via minio-js client 2022-12-21 11:41:31 -05:00
Assistance.ts (core) Improved error messages, retries, and handling of token limits in AI assistant 2023-07-18 16:01:37 +02:00
Authorizer.ts (core) Add optional telemetry to grist-core 2023-06-07 12:00:51 -04:00
BrowserSession.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
checksumFile.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
Client.ts (core) Add optional telemetry to grist-core 2023-06-07 12:00:51 -04:00
Comm.ts (core) i18 2022-09-29 18:02:09 +02:00
configureMinIOExternalStorage.ts Abort when MinIO bucket does not have versioning enabled #545 (#546) 2023-07-10 06:24:55 -04:00
dbUtils.ts (core) remove ormconfig.js from saas build; more sendgrid logging 2023-03-01 17:02:47 -05:00
DiscourseConnect.ts (core) Adding GristConnect login system 2022-05-18 20:28:25 +02:00
DocApi.ts (core) deleting queue from single webhook 2023-07-18 11:46:10 +02:00
DocClients.ts (core) add a yarn run cli tool, and add a sqlite gristify option 2022-07-14 12:00:30 -04:00
DocManager.ts (core) port some more test/browser tests to newer selenium 2023-04-12 13:00:53 -04:00
DocPluginData.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DocPluginManager.ts (core) add an access token mechanism to help with attachments in custom widgets 2022-07-19 11:55:18 -04:00
DocSession.ts (core) User language switcher 2023-01-26 09:47:14 +01:00
DocSnapshots.ts Track saved version per hour, day, week, month, year, and number of times a version with parameter (#509) 2023-05-12 11:38:29 -04:00
DocStorage.ts fix a node-sqlite3-ism that breaks record removal in grist-static (#566) 2023-07-11 05:52:06 -04:00
DocStorageManager.ts (core) dust off electron build a little bit 2022-07-29 11:19:26 -04:00
docUtils.d.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
docUtils.js Correct spelling mistakes 2022-02-19 09:46:49 +00:00
DocWorker.ts (core) Billing for formula assistant 2023-07-10 13:24:08 +02:00
DocWorkerMap.ts (core) Add unquarantine command to admin CLI 2022-08-15 13:04:55 -07:00
ExcelFormatter.ts (core) Currency from grist column is persistent when exporting to excel 2023-05-08 10:39:01 +02:00
ExpandedQuery.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Export.ts (core) For exporting XLSX, do it memory-efficiently in a worker thread. 2023-06-01 12:06:48 -04:00
ExportCSV.ts (core) For exporting XLSX, do it memory-efficiently in a worker thread. 2023-06-01 12:06:48 -04:00
ExportTableSchema.ts (core) Fix imports into reference columns, and support two ways to import Numeric as a reference. 2023-05-02 10:28:14 -04:00
ExportXLSX.ts (core) tweak handler for aborted connections to work on modern node 2023-06-16 10:20:53 -04:00
expressWrap.ts (core) For exporting XLSX, do it memory-efficiently in a worker thread. 2023-06-01 12:06:48 -04:00
ExternalStorage.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
extractOrg.ts (core) upgrade typeorm so we can support newer postgres 2022-09-02 15:34:21 -04:00
FileParserElement.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
filterUtils.ts (core) uncheck FullCopy special when copying/forking a document 2021-04-29 08:56:54 -04:00
FlexServer.ts External storage: split checkBackend and configure (follow-up #545) (#567) 2023-07-13 02:44:46 -04:00
ForwardAuthLogin.ts (core) clean up interaction of forward auth with session 2022-06-15 13:06:12 -04:00
GoogleAuth.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
GoogleExport.ts (core) For exporting XLSX, do it memory-efficiently in a worker thread. 2023-06-01 12:06:48 -04:00
GoogleImport.ts (core) Adding DELETE /api/docs/webhooks/queue endpoint to clear the queue 2022-12-01 12:23:19 +01:00
GranularAccess.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
GristServer.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
gristSessions.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
guessExt.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
hashingUtils.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
HashUtil.ts (core) add more detail to /compare endpoint 2020-09-18 16:31:29 -04:00
HostedMetadataManager.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
HostedStorageManager.ts (core) Persist forks in home db 2023-02-20 22:46:36 -05:00
httpEncoding.ts (core) External requests 2022-06-17 21:53:20 +02:00
IBilling.ts (core) Add basic activation page to grist-ee 2022-08-23 10:30:52 -07:00
IChecksumStore.ts (core) revamp snapshot inventory 2020-10-30 13:52:46 -04:00
ICreate.ts External storage: split checkBackend and configure (follow-up #545) (#567) 2023-07-13 02:44:46 -04:00
IDocStorageManager.ts (core) add a yarn run cli tool, and add a sqlite gristify option 2022-07-14 12:00:30 -04:00
idUtils.ts (core) Persist forks in home db 2023-02-20 22:46:36 -05:00
IElectionStore.ts (core) move some material to core that slipped through in a rebase 2020-07-23 11:29:05 -04:00
initialDocSql.ts (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
INotifier.ts (core) Speed up and upgrade build. 2022-06-27 16:10:10 -04:00
ISandbox.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
IShell.ts (core) dust off electron build a little bit 2022-07-29 11:19:26 -04:00
ITestingHooks-ti.ts (core) Converting server-side Comm.js to typescript 2022-06-07 15:47:17 -04:00
ITestingHooks.ts (core) Converting server-side Comm.js to typescript 2022-06-07 15:47:17 -04:00
log.ts fix log.add under electron (#478) 2023-04-03 13:29:35 -04:00
LogMethods.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
manifest.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
MinimalLogin.ts A set of tweaks to simplify electron packaging (#421) 2023-02-13 15:52:17 -05:00
MinIOExternalStorage.ts Abort when MinIO bucket does not have versioning enabled #545 (#546) 2023-07-10 06:24:55 -04:00
NSandbox.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
NullSandbox.ts (core) Fix snapshot migrations 2023-05-23 15:50:15 -04:00
OnDemandActions.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
PermissionInfo.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
Permit.ts (core) Add new Grist login page 2022-04-01 15:24:19 -07:00
places.ts A set of tweaks to simplify electron packaging (#421) 2023-02-13 15:52:17 -05:00
PluginEndpoint.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
PluginManager.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
ProcessMonitor.ts (core) Add optional telemetry to grist-core 2023-06-07 12:00:51 -04:00
ProxyAgent.ts (core) Add logging of errors whenever ProxyAgent is used, and a test for it. 2023-05-17 10:21:53 -04:00
reportTimeTaken.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
Requests.ts (core) Add logging of errors whenever ProxyAgent is used, and a test for it. 2023-05-17 10:21:53 -04:00
requestUtils.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
RowAccess.ts (core) control the distribution of attachment metadata 2022-12-22 09:10:30 -05:00
SafePythonComponent.ts (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
SamlConfig.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
SandboxControl.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
sandboxUtil.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
sendAppPage.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
ServerColumnGetters.ts (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
ServerLocale.ts (core) Converting server-side Comm.js to typescript 2022-06-07 15:47:17 -04:00
serverUtils.ts (core) Add optional telemetry to grist-core 2023-06-07 12:00:51 -04:00
Sessions.ts (core) Converting server-side Comm.js to typescript 2022-06-07 15:47:17 -04:00
Sharing.ts (core) port some more test/browser tests to newer selenium 2023-04-12 13:00:53 -04:00
shortDesc.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
shutdown.js support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
SqliteCommon.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
SQLiteDB.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
SqliteNode.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
TableMetadataLoader.ts (core) open documents without blocking on data engine 2021-10-01 10:18:56 -04:00
TagChecker.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
Telemetry.ts (core) Add Support Grist page and nudge 2023-07-04 17:36:59 -04:00
TestingHooks.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
TestLogin.ts (core) make the support account configurable, and allow listing public sites 2022-10-14 14:23:59 -04:00
Throttle.ts (core) Fix bugs with intervals 2022-08-25 12:38:36 -07:00
TimeQuery.ts (core) Adds a UI panel for managing webhooks 2023-05-08 18:25:27 -04:00
Triggers.ts (core) deleting queue from single webhook 2023-07-18 11:46:10 +02:00
UnsafeNodeComponent.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
uploads.ts (core) remove deprecated /download endpoint in favor of newer /api/docs/NNNN/download 2022-09-20 15:26:04 -04:00
WidgetRepository.ts (core) A few assorted one-line fixes 2023-01-09 23:31:04 -05:00
WorkCoordinator.ts support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
workerExporter.ts (core) Rearrange ExportXLSX code and fix ExportsAccessRules test that became flaky 2023-06-02 10:23:33 -04:00