(core) Add test_replay for easily replaying data sent to the sandbox purely within python

Summary:
Run JS with a value for SANDBOX_BUFFERS_DIR, then run test_replay in python with the same value to replay just the python code.

See test_replay.py for more info.

Test Plan:
Record some data, e.g. `SANDBOX_BUFFERS_DIR=manual npm start` or `SANDBOX_BUFFERS_DIR=server ./test/testrun.sh server`.

Then run `SANDBOX_BUFFERS_DIR=server python -m unittest test_replay` from within `core/sandbox/grist` to replay the input from the JS.

Sample of the output will look like this:

```
Checking /tmp/sandbox_buffers/server/2021-06-16T15:13:59.958Z
True
Checking /tmp/sandbox_buffers/server/2021-06-16T15:16:37.170Z
True
Checking /tmp/sandbox_buffers/server/2021-06-16T15:14:22.378Z
True
```

Reviewers: paulfitz, dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2866

sandbox/grist/test_replay.py

@@ -0,0 +1,91 @@
+"""
+Replay binary data sent from JS to reproduce behaviour in the sandbox.
+
+This isn't really a test and it doesn't run under normal circumstances,
+but it's convenient to run it alongside other tests to measure total coverage.
+
+This is a tool to directly run some python code of interest to make it easier to do things like:
+
+- Use a debugger within Python
+- Measure Python code coverage from JS tests
+- Rapidly iterate on Python code without having to repeatedly run the same JS
+    or write a Python test from scratch.
+
+To use this, first set the environment variable RECORD_SANDBOX_BUFFERS_DIR to a directory path,
+then run some JS code. For example you could run some tests,
+or run `npm start` and then manually interact with a document in a way that triggers
+desired behaviour in the sandbox.
+
+This will store files like $RECORD_SANDBOX_BUFFERS_DIR/<subdirectory>/(input|output)
+Each subdirectory corresponds to a single sandbox process so that replays are isolated.
+JS tests can start many instances of the sandbox and thus create many subdirectories.
+`input` contains the binary data sent from JS to Python, `output` contains the data sent back.
+Currently, the name of each subdirectory is the time it was created.
+
+Now run this test with the same value of RECORD_SANDBOX_BUFFERS_DIR. For each subdirectory,
+it will read in `input` just as it would read the pipe from JS, and send output to a file
+`new_output` in the same subdirectory. Then it will compare the data in `output` and `new_output`.
+The outputs will usually match but there are many reasons they might differ:
+
+- Functions registered in JS tests (e.g. via plugins) but not in the python unit tests.
+- File paths in tracebacks.
+- Slight differences between standard and NaCl interpreters.
+- Functions involving randomness or time.
+
+In any case the point is usually not whether or not the outputs match, but to directly run
+just the python code of interest.
+"""
+
+from __future__ import print_function
+
+import marshal
+import os
+import unittest
+
+from main import run
+from sandbox import Sandbox
+
+
+def marshal_load_all(path):
+  result = []
+  with open(path, "rb") as f:
+    while True:
+      try:
+        result.append(marshal.load(f))
+      except EOFError:
+        break
+
+  return result
+
+
+class TestReplay(unittest.TestCase):
+  maxDiff = None
+
+  def test_replay(self):
+    root = os.environ.get("RECORD_SANDBOX_BUFFERS_DIR")
+    if not root:
+      self.skipTest("RECORD_SANDBOX_BUFFERS_DIR not set")
+    for dirpath, dirnames, filenames in os.walk(root):
+      if "input" not in filenames:
+        continue
+
+      print("Checking " + dirpath)
+
+      input_path = os.path.join(dirpath, "input")
+      output_path = os.path.join(dirpath, "output")
+      new_output_path = os.path.join(dirpath, "new_output")
+      with open(input_path, "rb") as external_input:
+        with open(new_output_path, "wb") as external_output:
+          sandbox = Sandbox(external_input, external_output)
+          run(sandbox)
+
+      original_output = marshal_load_all(output_path)
+
+      # _send_to_js does two layers of marshalling,
+      # and NSandbox._onSandboxData parses one of those layers before writing,
+      # hence original_output is 'more parsed' than marshal_load_all(new_output_path)
+      new_output = [marshal.loads(b) for b in marshal_load_all(new_output_path)]
+
+      # It's usually not worth asserting a match, see comments at the top of the file
+      print("Match:", original_output == new_output)
+      # self.assertEqual(original_output, new_output)