gristlabs_grist-core/.github/workflows/main.yml
Paul Fitzpatrick bcbf57d590 (core) bump mocha version to allow parallel tests; move more tests to core
Summary:
This uses a newer version of mocha in grist-core so that tests can be run in parallel. That allows more tests to be moved without slowing things down overall. Tests moved are venerable browser tests; only the ones that "just work" or worked without too much trouble to are moved, in order to keep the diff from growing too large. Will wrestle with more in follow up.

Parallelism is at the file level, rather than the individual test.

The newer version of mocha isn't needed for grist-saas repo; tests are parallelized in our internal CI by other means. I've chosen to allocate files to workers in a cruder way than our internal CI, based on initial characters rather than an automated process. The automated process would need some reworking to be compatible with mocha running in parallel mode.

Test Plan: this diff was tested first on grist-core, then ported to grist-saas so saas repo history will correctly track history of moved files.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3927
2023-06-27 02:55:34 -04:00

146 lines
4.4 KiB
YAML

name: CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows running this workflow manually from the Actions tab
workflow_dispatch:
jobs:
build_and_test:
runs-on: ubuntu-latest
strategy:
# it is helpful to know which sets of tests would have succeeded,
# even when there is a failure.
fail-fast: false
matrix:
python-version: [3.9]
node-version: [14.x]
tests:
- ':lint:python:client:common:smoke:'
- ':server-1-of-2:'
- ':server-2-of-2:'
- ':nbrowser-^[A-G]:'
- ':nbrowser-^[H-L]:'
- ':nbrowser-^[M-O]:'
- ':nbrowser-^[P-S]:'
- ':nbrowser-^[^A-S]:'
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
cache: 'yarn'
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install Python packages
run: |
pip install virtualenv
yarn run install:python
- name: Install Node.js packages
run: yarn install
- name: Run eslint
if: contains(matrix.tests, ':lint:')
run: yarn run lint:ci
- name: Make sure bucket is versioned
if: contains(matrix.tests, ':server-')
env:
AWS_ACCESS_KEY_ID: administrator
AWS_SECRET_ACCESS_KEY: administrator
run: aws --region us-east-1 --endpoint-url http://localhost:9000 s3api put-bucket-versioning --bucket grist-docs-test --versioning-configuration Status=Enabled
- name: Build Node.js code
run: yarn run build:prod
- name: Run smoke test
if: contains(matrix.tests, ':smoke:')
run: VERBOSE=1 DEBUG=1 MOCHA_WEBDRIVER_HEADLESS=1 yarn run test:smoke
- name: Run python tests
if: contains(matrix.tests, ':python:')
run: yarn run test:python
- name: Run client tests
if: contains(matrix.tests, ':client:')
run: yarn run test:client
- name: Run common tests
if: contains(matrix.tests, ':common:')
run: yarn run test:common
- name: Run server tests with minio and redis
if: contains(matrix.tests, ':server-')
run: |
export TEST_SPLITS=$(echo $TESTS | sed "s/.*:server-\([^:]*\).*/\1/")
MOCHA_WEBDRIVER_HEADLESS=1 yarn run test:server
env:
GRIST_DOCS_MINIO_ACCESS_KEY: administrator
GRIST_DOCS_MINIO_SECRET_KEY: administrator
TEST_REDIS_URL: "redis://localhost/11"
GRIST_DOCS_MINIO_USE_SSL: 0
GRIST_DOCS_MINIO_ENDPOINT: localhost
GRIST_DOCS_MINIO_PORT: 9000
GRIST_DOCS_MINIO_BUCKET: grist-docs-test
- name: Run main tests without minio and redis
if: contains(matrix.tests, ':nbrowser-')
run: |
export GREP_TESTS=$(echo $TESTS | sed "s/.*:nbrowser-\([^:]*\).*/\1/")
MOCHA_WEBDRIVER_SKIP_CLEANUP=1 MOCHA_WEBDRIVER_HEADLESS=1 yarn run test:nbrowser --parallel --jobs 3
env:
TESTS: ${{ matrix.tests }}
services:
# https://github.com/bitnami/bitnami-docker-minio/issues/16
minio:
image: bitnami/minio:latest
env:
MINIO_DEFAULT_BUCKETS: "grist-docs-test:public"
MINIO_ROOT_USER: administrator
MINIO_ROOT_PASSWORD: administrator
ports:
- 9000:9000
options: >-
--health-cmd "curl -f http://localhost:9000/minio/health/ready"
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
candidate:
needs: build_and_test
if: ${{ success() && github.event_name == 'push' }}
runs-on: ubuntu-latest
steps:
- name: Fetch new candidate branch
uses: actions/checkout@v3
- name: Update candidate branch
uses: ad-m/github-push-action@8407731efefc0d8f72af254c74276b7a90be36e1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: latest_candidate
force: true