This adds an ASSISTANT_CHAT_COMPLETION_ENDPOINT which can be used
to enable AI Assistance instead of an OpenAI API key. The assistant
then works against compatible endpoints, in the mechanical sense.
Quality of course will depend on the model. I found some tweaks
to the prompt that work well both for Llama-2 and for OpenAI's models,
but I'm not including them here because they would conflict with some
prompt changes that are already in the works.
Co-authored-by: Alex Hall <alex.mojaki@gmail.com>
@ -81,7 +81,7 @@ Here are some specific feature highlights of Grist:
- On OSX, you can use native sandboxing.
- On any OS, including Windows, you can use a wasm-based sandbox.
* Translated to many languages.
* Support for an AI Formula Assistant (using OpenAI gpt-3.5-turbo).
* Support for an AI Formula Assistant (using OpenAI gpt-3.5-turbo or comparable models).
* `F1` key brings up some quick help. This used to go without saying. In general Grist has good keyboard support.
* We post progress on [𝕏 or Twitter or whatever](https://twitter.com/getgrist).
@ -302,7 +302,19 @@ PORT | port number to listen on for Grist server
REDIS_URL | optional redis server for browser sessions and db query caching
GRIST_SNAPSHOT_TIME_CAP | optional. Define the caps for tracking buckets. Usage: {"hour": 25, "day": 32, "isoWeek": 12, "month": 96, "year": 1000}
GRIST_SNAPSHOT_KEEP | optional. Number of recent snapshots to retain unconditionally for a document, regardless of when they were made
OPENAI_API_KEY | optional. Used for the AI formula assistant. Sign up for an account on OpenAI and then generate a secret key [here](https://platform.openai.com/account/api-keys).
AI Formula Assistant related variables (all optional):
Variable | Purpose
-------- | -------
ASSISTANT_API_KEY | optional. An API key to pass when making requests to an external AI conversational endpoint.
ASSISTANT_CHAT_COMPLETION_ENDPOINT | optional. A chat-completion style endpoint to call. Not needed if OpenAI is being used.
ASSISTANT_MODEL | optional. If set, this string is passed along in calls to the AI conversational endpoint.
ASSISTANT_LONGER_CONTEXT_MODEL | optional. If set, requests that fail because of a context length limitation will be retried with this model set.
OPENAI_API_KEY | optional. Synonym for ASSISTANT_API_KEY that assumes an OpenAI endpoint is being used. Sign up for an account on OpenAI and then generate a secret key [here](https://platform.openai.com/account/api-keys).
At the time of writing, the AI Assistant is known to function against OpenAI chat completion endpoints for gpt-3.5-turbo and gpt-4.
It can also function against the chat completion endpoint provided by <ahref="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a>.