Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
6282558abd
gristlabs_grist-core/sandbox/requirements3.txt

56 lines
1.4 KiB
Plaintext
Raw Normal View History

(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
#
(core) Upgrade to Python 3.11 Summary: Replaced mentions of Python 3.9 with 3.11 Test Plan: this Reviewers: paulfitz, georgegevoian Reviewed By: paulfitz, georgegevoian Subscribers: dsagal, georgegevoian, paulfitz Differential Revision: https://phab.getgrist.com/D3980
2023-08-04 15:59:52 +00:00
# This file is autogenerated by pip-compile with Python 3.11
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# by the following command:
#
# pip-compile --output-file=core/sandbox/requirements3.txt core/sandbox/requirements3.in
#
add a pyodide-based "sandbox" flavor (#437) This adds a new `GRIST_SANDBOX_FLAVOR=pyodide` option where the version of Python used for the data engine is wasm, and so can be run by node like the rest of the back end. It still runs as a separate process. There are a few small version changes made to packages to avoid various awkwardnesses present in the current versions. All existing tests pass. This is very experimental. To use, you'll need something with a bash shell and make. First do: ``` cd sandbox/pyodide make setup # README.md and Makefile have details cd .. ``` Then running Grist as: ``` GRIST_SANDBOX_FLAVOR=pyodide yarn start ``` should work. Adding a formula with content: ``` import sys; return sys.version ``` should return a different Python version than other sandboxes. The motivation for this work is to have a form of sandboxing that will work on Windows for Grist Electron (for Linux we have gvisor/runsc, for Mac we have sandbox-exec, but I haven't found anything comparable for Windows). It also brings a back-end-free version of Grist a bit closer, for use-cases where that would make sense - such as serving a report (in the form of a Grist document) on a static site.
2023-03-06 21:56:25 +00:00
astroid==2.14.2
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
(core) Fix updating attributes inside f-strings when columns are renamed Summary: Upgrades asttokens and uses the newly released support for astroid trees in `asttokens.ASTText` which is needed to deal with f-strings (as opposed to `asttokens.ASTTokens`). Test Plan: Added a Python unit test Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D4027
2023-09-07 10:01:42 +00:00
asttokens==2.4.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via
# friendly-traceback
# stack-data
(core) Improve encoding detection for csv imports, and make encoding an editable option. Summary: - Using a sample of data was causing poor detection if the sample were cut mid-character. Switch to using line-based detection. - Add a simple option for changing encoding. No convenient UI is offered since config UI is auto-generated, but this at least makes it possible to recover from bad guesses. - Upgrades chardet library for good measure. - Also fixes python3-building step, to more reliably rebuild Python dependencies when requirements3.* files change. Test Plan: Added a python-side test case, and a browser test that encodings can be switched, errors are displayed, and wrong encodings fail recoverably. Reviewers: alexmojaki Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D3979
2023-08-18 21:03:27 +00:00
chardet==5.1.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
et-xmlfile==1.0.1
# via openpyxl
executing==1.1.1
# via
# friendly-traceback
# stack-data
friendly-traceback==0.7.48
# via -r core/sandbox/requirements3.in
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942
2021-07-27 23:43:21 +00:00
iso8601==0.1.12
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
lazy-object-proxy==1.6.0
# via astroid
openpyxl==3.0.10
# via -r core/sandbox/requirements3.in
(core) Add PHONE_FORMAT function using the phonenumberslight library Summary: Add phonenumberslite-8.12.57 to requirements Implement PHONE_FORMAT function. Test Plan: Added doctest test cases Reviewers: jarek Reviewed By: jarek Subscribers: jarek Differential Revision: https://phab.getgrist.com/D3665
2022-10-21 12:13:13 +00:00
phonenumberslite==8.12.57
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
pure-eval==0.2.2
# via
# friendly-traceback
# stack-data
python-dateutil==2.8.2
# via -r core/sandbox/requirements3.in
roman==3.3
# via -r core/sandbox/requirements3.in
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942
2021-07-27 23:43:21 +00:00
six==1.16.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via
# -r core/sandbox/requirements3.in
# asttokens
# friendly-traceback
# python-dateutil
# unittest-xml-reporting
Update some python deps to prepare python>=3.10 compat ABC classes have moved from collections to collections.abc since python 3.3, and retro compat has ended with 3.8
2022-08-19 08:16:56 +00:00
sortedcontainers==2.4.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
stack-data==0.5.1
# via friendly-traceback
typing-extensions==4.4.0
(core) Upgrade to Python 3.11 Summary: Replaced mentions of Python 3.9 with 3.11 Test Plan: this Reviewers: paulfitz, georgegevoian Reviewed By: paulfitz, georgegevoian Subscribers: dsagal, georgegevoian, paulfitz Differential Revision: https://phab.getgrist.com/D3980
2023-08-04 15:59:52 +00:00
# via -r core/sandbox/requirements3.in
(core) support python3 in grist-core, and running engine via docker and/or gvisor Summary: * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work. * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run. - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox. - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing. - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage. - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container. * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`). The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments. I haven't looked closely at imports beyond core. Test Plan: tested manually against regular grist and grist-core, including imports Reviewers: alexmojaki, dsagal Reviewed By: alexmojaki Differential Revision: https://phab.getgrist.com/D2942
2021-07-27 23:43:21 +00:00
unittest-xml-reporting==2.0.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via -r core/sandbox/requirements3.in
(core) Upgrade to latest versions of asttokens and wrapt Summary: Upgrades asttokens mainly because I suspected it would have helped with an error in a user's document (see https://grist.slack.com/archives/C0234CPPXPA/p1686145370484509) but we were unable to confirm that. Also adds a related test that I (wrongly) expected to have a similar problem before upgrading asttokens. Upgrades wrapt for Python 3.11 support. Closes https://github.com/gristlabs/grist-core/issues/534. Not upgrading the dependencies for Python 2 because pip is giving me errors when I try to install to the `thirdparty` folder. Both these upgrades are motivated by issues specific to Python 3 so this doesn't seem worth pursuing immediately. Test Plan: Existing tests. Reviewers: paulfitz Reviewed By: paulfitz Subscribers: paulfitz Differential Revision: https://phab.getgrist.com/D3973
2023-07-26 16:50:36 +00:00
wrapt==1.15.0
(core) Use a new docker image for tests in Jenkins Summary: The image Jenkins used up to now was from 2021. We updated buildtools/jenkins-ec2/Dockerfile since then but couldn't use new image; we can now that an issue with mounts in gvisor has been resolved. The new image uses newer Chrome (111 vs 95); 111 is the latest version that works with the webdriver version we are using (after that one, copy-pasting tests fail). In addition, this adds a more precise way to maintain python dependencies: by specifying top-level dependencies in the new core/sandbox/requirements3.in. An updated build step compiles that into requirements3.txt, and syncs venv to that using pip-sync. This addresses the issue that previously, removing a module from dependencies would not have caused the build to remove it from sandbox_venv3. Test Plan: Existing tests pass on the new image. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3952
2023-07-18 22:27:09 +00:00
# via astroid
Reference in New Issue Copy Permalink