(core) Modify prompt so that model may say it cannot help with certain requests.

Summary:
This tweaks the prompting so that the user's message is given on its own instead of as a docstring within Python. This is so that the prompt makes sense when:

- the user asks a question such as "Can you write me a formula which does ...?" rather than describing their formula as a docstring would, or
- the user sends a message that doesn't ask for a formula at all (https://grist.slack.com/archives/C0234CPPXPA/p1687699944315069?thread_ts=1687698078.832209&cid=C0234CPPXPA)

Also added wording for the model to refuse when the user asks for something that the model cannot do.

Because the code (and maybe in some cases the model) for non-ChatGPT models relies on the prompt consisting entirely of Python code produced by the data engine (which no longer contains the user's message) those code paths have been disabled for now. Updating them now seems like undesirable drag, I think it'd be better to revisit this when iteration/experimentation has slowed down and stabilised.

Test Plan:
Added entries to the formula dataset where the response shouldn't contain a formula, indicated by the value `1` for the new column `no_formula`.

This is somewhat successful, as the model does refuse to help in some of the new test cases, but not all. Performance on existing entries also seems a bit worse, but it's hard to distinguish this from random noise. Hopefully this can be remedied in the future with more work, e.g. automatic followup messages containing example inputs and outputs.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3936

sandbox/grist/formula_prompt.py

@@ -150,7 +150,7 @@ def class_schema(engine, table_id, exclude_col_id=None, lookups=False):
   return result
 
 
-def get_formula_prompt(engine, table_id, col_id, description,
+def get_formula_prompt(engine, table_id, col_id, _description,
                        include_all_tables=True,
                        lookups=True):
   result = ""
@@ -165,9 +165,7 @@ def get_formula_prompt(engine, table_id, col_id, description,
   result += "    @property\n"
   result += "    # rec is alias for self\n"
   result += "    def {}(rec) -> {}:\n".format(col_id, return_type)
-  result += '        """\n'
-  result += '{}\n'.format(indent(description, "        "))
-  result += '        """\n'
+  result += "        # Please fill in code only after this line, not the `def`\n"
   return result
 
 def indent(text, prefix, predicate=None):

sandbox/grist/test_formula_prompt.py

@@ -151,9 +151,7 @@ class Table2:
     @property
     # rec is alias for self
     def new_formula(rec) -> float:
-        """
-        description here
-        """
+        # Please fill in code only after this line, not the `def`
 ''')
 
   def test_get_formula_prompt(self):
@@ -183,9 +181,7 @@ class Table1:
     @property
     # rec is alias for self
     def text(rec) -> str:
-        """
-        description here
-        """
+        # Please fill in code only after this line, not the `def`
 ''')
 
     self.assert_prompt("Table2", "ref", '''\
@@ -199,9 +195,7 @@ class Table2:
     @property
     # rec is alias for self
     def ref(rec) -> Table1:
-        """
-        description here
-        """
+        # Please fill in code only after this line, not the `def`
 ''')
 
     self.assert_prompt("Table3", "reflist", '''\
@@ -219,9 +213,7 @@ class Table3:
     @property
     # rec is alias for self
     def reflist(rec) -> List[Table2]:
-        """
-        description here
-        """
+        # Please fill in code only after this line, not the `def`
 ''')
 
   def test_convert_completion(self):