6 Commits (main)

Author	SHA1	Message	Date
Dmitry S	534615dd50	(core) Update logging in sandbox code, and log tracebacks as single log messages. ... Summary: - Replace logger module by the standard module 'logging'. - When a log message from the sandbox includes newlines (e.g. for tracebacks), keep those lines together in the Node log message. Previously each line was a different message, making it difficult to view tracebacks, particularly in prod where each line becomes a separate message object. - Fix assorted lint errors. Test Plan: Added a test for the log-line splitting and escaping logic. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3956	11 months ago
Paul Fitzpatrick	66643a5e6b	add a pyodide-based "sandbox" flavor (#437) ... This adds a new `GRIST_SANDBOX_FLAVOR=pyodide` option where the version of Python used for the data engine is wasm, and so can be run by node like the rest of the back end. It still runs as a separate process. There are a few small version changes made to packages to avoid various awkwardnesses present in the current versions. All existing tests pass. This is very experimental. To use, you'll need something with a bash shell and make. First do: ``` cd sandbox/pyodide make setup # README.md and Makefile have details cd .. ``` Then running Grist as: ``` GRIST_SANDBOX_FLAVOR=pyodide yarn start ``` should work. Adding a formula with content: ``` import sys; return sys.version ``` should return a different Python version than other sandboxes. The motivation for this work is to have a form of sandboxing that will work on Windows for Grist Electron (for Linux we have gvisor/runsc, for Mac we have sandbox-exec, but I haven't found anything comparable for Windows). It also brings a back-end-free version of Grist a bit closer, for use-cases where that would make sense - such as serving a report (in the form of a Grist document) on a static site.	1 year ago
Paul Fitzpatrick	4222f1ed32	(core) communicate with sandbox via standard pipes ... Summary: This switches to using stdin/stdout for RPC calls to the sandbox, rather than specially allocated side channels. Plain text error information remains on stderr. The motivation for the change is to simplify use of sandboxes, some of which support extra file descriptors and some of which don't. The new style of communication is made the default, but I'm not committed to this, just that it be easy to switch to if needed. It is possible I'll need to switch the communication method again in the near future. One reason not to make this default would be windows support, which is likely broken since stdin/stdout are by default in text mode. Test Plan: existing tests pass Reviewers: dsagal, alexmojaki Reviewed By: dsagal, alexmojaki Differential Revision: https://phab.getgrist.com/D2897	3 years ago
Alex Hall	84ddbc448b	(core) Add test_replay for easily replaying data sent to the sandbox purely within python ... Summary: Run JS with a value for SANDBOX_BUFFERS_DIR, then run test_replay in python with the same value to replay just the python code. See test_replay.py for more info. Test Plan: Record some data, e.g. `SANDBOX_BUFFERS_DIR=manual npm start` or `SANDBOX_BUFFERS_DIR=server ./test/testrun.sh server`. Then run `SANDBOX_BUFFERS_DIR=server python -m unittest test_replay` from within `core/sandbox/grist` to replay the input from the JS. Sample of the output will look like this: ``` Checking /tmp/sandbox_buffers/server/2021-06-16T15:13:59.958Z True Checking /tmp/sandbox_buffers/server/2021-06-16T15:16:37.170Z True Checking /tmp/sandbox_buffers/server/2021-06-16T15:14:22.378Z True ``` Reviewers: paulfitz, dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2866	3 years ago
Alex Hall	16f297a250	(core) Simple Python 3 compatibility changes ... Summary: Changes that move towards python 3 compatibility that are easy to review without much thought Test Plan: The tests Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2873	3 years ago
Paul Fitzpatrick	b82eec714a	(core) move data engine code to core ... Summary: this moves sandbox/grist to core, and adds a requirements.txt file for reconstructing the content of sandbox/thirdparty. Test Plan: existing tests pass. Tested core functionality manually. Tested docker build manually. Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2563	4 years ago