gristlabs_grist-core

Commit Graph

Author	SHA1	Message	Date
Dmitry S	2eec48b685	(core) Support adjusting OOM score for child sandbox processes. Summary: Also update error handling in NSandbox initialization to avoid node exiting when sandbox can't be created. Info on oom_score and choom: https://man7.org/linux/man-pages/man1/choom.1.html Empirically, in docker and under fargate, oom_score (visible in /proc/PID/oom_score) seems to be calculated approximately as ((ProcessRSS / TotalMemory * 1000 + 999 + oom_score_adj) * 2/3) though this doesn't correspond to any documentation I could find. In addition, under docker / fargate it does not work to set oom_score_adj (with choom or via /proc/PID/oom_score_adj) to a negative value to give priority to a node process. That's why this diff adjust the score up for sandbox processes instead. Test Plan: Checked that grist-omnibus built with this change respects the variable, and sandbox processes end up with higher oom_score values. Reviewers: paulfitz Reviewed By: paulfitz Subscribers: paulfitz Differential Revision: https://phab.getgrist.com/D4112	10 months ago
Paul Fitzpatrick	b3b7410ede	(core) open documents without blocking on data engine Summary: With this diff, when a user opens a Grist document in a browser, they will be able to view its contents without waiting for the data engine to start up. Once the data engine starts, it will run a calculation and send any updates made. Changes to the document will be blocked until the engine is started and the initial calculation is complete. The increase in responsiveness is useful in its own right, and also reduces the impact of an extra startup time in a candidate next-generation sandbox. A small unrelated fix is included for `core/package.json`, to catch up with a recent change to `package.json`. A small `./build schema` convenience is added to just rebuild the typescript schema file. Test Plan: added test; existing tests pass - small fixes needed in some cases because of new timing Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D3036	3 years ago

Author

SHA1

Message

Date

Dmitry S

2eec48b685

(core) Support adjusting OOM score for child sandbox processes.

Summary:
Also update error handling in NSandbox initialization to avoid node
exiting when sandbox can't be created.

Info on oom_score and choom: https://man7.org/linux/man-pages/man1/choom.1.html

Empirically, in docker and under fargate, oom_score (visible in
/proc/PID/oom_score) seems to be calculated approximately as

    ((ProcessRSS / TotalMemory * 1000 + 999 + oom_score_adj) * 2/3)

though this doesn't correspond to any documentation I could find.

In addition, under docker / fargate it does not work to set oom_score_adj (with choom or via /proc/PID/oom_score_adj) to a negative value to give priority to a node process. That's why this diff adjust the score up for sandbox processes instead.

Test Plan:
Checked that grist-omnibus built with this change respects the
variable, and sandbox processes end up with higher oom_score values.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4112

Paul Fitzpatrick

b3b7410ede

(core) open documents without blocking on data engine

Summary:
With this diff, when a user opens a Grist document in a browser, they will be able to view its contents without waiting for the data engine to start up. Once the data engine starts, it will run a calculation and send any updates made. Changes to the document will be blocked until the engine is started and the initial calculation is complete.

The increase in responsiveness is useful in its own right, and also reduces the impact of an extra startup time in a candidate next-generation sandbox.

A small unrelated fix is included for `core/package.json`, to catch up with a recent change to `package.json`.

A small `./build schema` convenience is added to just rebuild the typescript schema file.

Test Plan: added test; existing tests pass - small fixes needed in some cases because of new timing

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3036

2 Commits (15a73fdaf0ef11acc8da3759c09a928b8961dd5f)