(core) Implement trigger formulas (generalizing default formulas)

Summary:
Trigger formulas can be calculated for new records, or for new records and
updates to certain fields, or all fields. They do not recalculate on open,
and they MAY be set directly by the user, including for data-cleaning.

- Column metadata now includes recalcWhen and recalcDeps fields.
- Trigger formulas are NOT recalculated on open or on schema changes.
- When recalcWhen is "never", formula isn't calculated even for new records.
- When recalcWhen is "allupdates", formula is calculated for new records and
  any manual (non-formula) updates to the record.
- When recalcWhen is "", formula is calculated for new records, and changes to
  recalcDeps fields (which may be formula fields or column itself).
- A column whose recalcDeps includes itself is a "data-cleaning" column; a
  value set by the user will still trigger the formula.
- All trigger-formulas receive a "value" argument (to support the case above).

Small changes
- Update RefLists (used for recalcDeps) when target rows are deleted.
- Add RecordList.__contains__ (for `rec in refList` or `id in refList` checks)
- Clarify that Calculate action has replaced load_done() in practice,
  and use it in tests too, to better match reality.

Left for later:
- UI for setting recalcWhen / recalcDeps.
- Implementation of actions such as "Recalculate for all cells".
- Allowing trigger-formulas access to the current user's info.

Test Plan: Added a comprehensive python-side test for various trigger combinations

Reviewers: paulfitz, alexmojaki

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2872

sandbox/grist/schema.py

@@ -15,7 +15,7 @@ import six
 
 import actions
 
-SCHEMA_VERSION = 21
+SCHEMA_VERSION = 22
 
 def make_column(col_id, col_type, formula='', isFormula=False):
   return {
@@ -78,6 +78,14 @@ def schema_create_actions():
       # E.g. Foo.person may have a visibleCol pointing to People.Name, with the displayCol
       # pointing to Foo._gristHelper_DisplayX column with the formula "$person.Name".
       make_column("visibleCol",       "Ref:_grist_Tables_column"),
+
+      # Instructions when to recalculate the formula on a column with isFormula=False (previously
+      # known as a "default formula"). Values are RecalcWhen constants defined below.
+      make_column("recalcWhen",       "Int"),
+
+      # List of fields that should trigger a calculation of a formula in a data column. Only
+      # applies when recalcWhen is RecalcWhen.DEFAULT, and defaults to the empty list.
+      make_column("recalcDeps",       "RefList:_grist_Tables_column"),
     ]),
 
     # DEPRECATED: Previously used to keep import options, and allow the user to change them.
@@ -285,6 +293,17 @@ def schema_create_actions():
   ]
 
 
+class RecalcWhen(object):
+  """
+  Constants for column's recalcWhen field, which determine when a formula associated with a data
+  column would get calculated.
+  """
+  DEFAULT = 0         # Calculate on new records or when any field in recalcDeps changes. If
+                      # recalcDeps includes this column itself, it's a "data-cleaning" formula.
+  NEVER = 1           # Don't calculate automatically (but user can trigger manually)
+  MANUAL_UPDATES = 2  # Calculate on new records and on manual updates to any data field.
+
+
 # These are little structs to represent the document schema that's used in code generation.
 # Schema itself (as stored by Engine) is an OrderedDict(tableId -> SchemaTable), with
 # SchemaTable.columns being an OrderedDict(colId -> SchemaColumn).