153 Commits (c9f9b70b67e0f44b467b5206ca5d2301988bad51)

Author	SHA1	Message	Date
Paul Fitzpatrick	6e15d44cf6	(core) start applying defenses for untrusted document uploads ... Summary: This applies some mitigations suggested by SQLite authors when opening untrusted SQLite databases, as we do when Grist docs are uploaded by the user. See: https://www.sqlite.org/security.html#untrusted_sqlite_database_files Steps implemented in this diff are: * Setting `trusted_schema` to off * Running a SQLite-level integrity check on uploads Other steps will require updates to our node-sqlite3 fork, since they are not available via the node-sqlite3 api (one more reason to migrate to better-sqlite3). I haven't yet managed to create a file that triggers an integrity check failure without also being detected as corruption by sqlite at a more basic level, so that is a TODO for testing. Test Plan: existing tests pass; need to come up with exploits to actually test the defences and have not yet Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2909	3 years ago
Alex Hall	8a940676e9	(core) Generic tools for recording pycalls, deterministic mode. ... Summary: Replaces https://phab.getgrist.com/D2854 Refactoring of NSandbox: - Simplify arguments to NSandbox.spawn. Only half the arguments were used depending on the flavour, adding a layer of confusion. - Ensure the same environment variables are passed to both flavours of sandbox - Simplify passing down environment variables. Implement deterministic mode with libfaketime and a seeded random instance. - Include static prebuilt libfaketime.so.1, may need another solution in future for other platforms. Recording pycalls: - Add script recordDocumentPyCalls.js to open a single document outside of tests. - Refactor out recordPyCalls.ts to support various uses. - Add afterEach hook to save all pycalls from server tests under $PYCALLS_DIR - Make docTools usable without mocha. - Add useLocalDoc and loadLocalDoc for loading non-fixture documents Test Plan: Made a document with formulas NOW() and UUID() Compare two document openings in normal mode: diff <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout) \ <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout) Output: < 1623407499.58132, --- > 1623407499.60376, 1195c1195 < "B": "bd2487f6-63c9-4f02-bbbc-5c0d674a2dc6" --- > "B": "22e1a4fd-297f-4b86-91a2-bc42cc6da4b2" `export DETERMINISTIC_MODE=1` and repeat. diff is empty! Reviewers: paulfitz Reviewed By: paulfitz Differential Revision: https://phab.getgrist.com/D2857	3 years ago
Paul Fitzpatrick	9f234b758d	(core) freshen grist-core build ... Summary: * adds a smoke test to grist-core * fixes a problem with highlight.js failing to load correctly * skips survey for default user * freshens docker build Utility files in test/nbrowser are moved to core/test/nbrowser, so that gristUtils are available there. This increased the apparent size of the diff as "./" import paths needed replacing with "test/nbrowser/" paths. The utility files are untouched, except for the code to start a server - it now has a small grist-core specific conditional in it. Test Plan: adds test Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2768	4 years ago