gristlabs_grist-core/buildtools/prepare_python3.sh

#!/bin/bash

set -e

echo "Making Python3 sandbox"
if [ ! -e sandbox_venv3 ]; then
  python3 -m venv sandbox_venv3
fi

echo "Updating Python3 packages"
sandbox_venv3/bin/pip install --no-deps -r sandbox/requirements3.txt
echo "Python3 packages ready in sandbox_venv3"
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942 2021-07-27 23:43:21 +00:00			`#!/bin/bash`

			`set -e`

(core) freshen core README; support python3 in grist-core docker image Summary: This updates the grist-core README to list specific features of Grist, to make it easier for a casual visitor to get a sense of its scope. Adds links to some new resources (reviews, templates, grist v airtable post) that could also help. Adds python3 to docker image so that templates work without fuss. Test Plan: existing tests should pass Reviewers: georgegevoian Reviewed By: georgegevoian Subscribers: dsagal, anaisconce Differential Revision: https://phab.getgrist.com/D3204 2022-01-07 17:06:04 +00:00			`echo "Making Python3 sandbox"`
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942 2021-07-27 23:43:21 +00:00			`if [ ! -e sandbox_venv3 ]; then`
(core) freshen core README; support python3 in grist-core docker image Summary: This updates the grist-core README to list specific features of Grist, to make it easier for a casual visitor to get a sense of its scope. Adds links to some new resources (reviews, templates, grist v airtable post) that could also help. Adds python3 to docker image so that templates work without fuss. Test Plan: existing tests should pass Reviewers: georgegevoian Reviewed By: georgegevoian Subscribers: dsagal, anaisconce Differential Revision: https://phab.getgrist.com/D3204 2022-01-07 17:06:04 +00:00			`python3 -m venv sandbox_venv3`
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942 2021-07-27 23:43:21 +00:00			`fi`

(core) freshen core README; support python3 in grist-core docker image Summary: This updates the grist-core README to list specific features of Grist, to make it easier for a casual visitor to get a sense of its scope. Adds links to some new resources (reviews, templates, grist v airtable post) that could also help. Adds python3 to docker image so that templates work without fuss. Test Plan: existing tests should pass Reviewers: georgegevoian Reviewed By: georgegevoian Subscribers: dsagal, anaisconce Differential Revision: https://phab.getgrist.com/D3204 2022-01-07 17:06:04 +00:00			`echo "Updating Python3 packages"`
			`sandbox_venv3/bin/pip install --no-deps -r sandbox/requirements3.txt`
			`echo "Python3 packages ready in sandbox_venv3"`