Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2026-03-02 04:09:24 +00:00
Code Issues Projects Releases Wiki Activity

(core) support python3 in grist-core, and running engine via docker and/or gvisor

Browse Source 
Summary:
 * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work.
 * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run.
   - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox.
   - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing.
   - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage.
   - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container.
 * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`).

The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments.

I haven't looked closely at imports beyond core.

Test Plan: tested manually against regular grist and grist-core, including imports

Reviewers: alexmojaki, dsagal

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D2942
This commit is contained in:
Paul Fitzpatrick
2021-07-27 19:43:21 -04:00
parent cd0c6de53e
commit bb8cb2593d
34 changed files with 3584 additions and 152 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
app/server/lib/GoogleAuth.ts
View File

@@ -3,7 +3,7 @@ import {ApiError} from 'app/common/ApiError';
import {parseSubdomain} from 'app/common/gristUrls';
import {expressWrap} from 'app/server/lib/expressWrap';
import * as log from 'app/server/lib/log';
import {getOriginUrl} from 'app/server/lib/requestUtils';
import {getOriginUrl, optStringParam, stringParam} from 'app/server/lib/requestUtils';
import * as express from 'express';
import {URL} from 'url';
@@ -93,13 +93,13 @@ export async function googleAuthTokenMiddleware(
res: express.Response,
next: express.NextFunction) {
// If access token is in place, proceed
if (!req.query.code) {
if (!optStringParam(req.query.code)) {
throw new ApiError("Google Auth endpoint requires a code parameter in the query string", 400);
} else {
try {
const oAuth2Client = _googleAuthClient();
// Decrypt code that was send back from Google Auth service. Uses GOOGLE_CLIENT_SECRET key.
const tokenResponse = await oAuth2Client.getToken(req.query.code);
const tokenResponse = await oAuth2Client.getToken(stringParam(req.query.code));
// Get the access token (access token will be present in a default request configuration).
const access_token = tokenResponse.tokens.access_token!;
req.query.access_token = access_token;
@@ -122,7 +122,7 @@ export function addGoogleAuthEndpoint(
messagePage: (req: express.Request, res: express.Response, message: any) => any
) {
if (!process.env.GOOGLE_CLIENT_SECRET) {
log.error("Failed to create GoogleAuth endpoint: GOOGLE_CLIENT_SECRET is not defined");
log.warn("Failed to create GoogleAuth endpoint: GOOGLE_CLIENT_SECRET is not defined");
expressApp.get(authHandlerPath, expressWrap(async (req: express.Request, res: express.Response) => {
throw new Error("Send to Google Drive is not configured.");
}));
@@ -136,20 +136,22 @@ export function addGoogleAuthEndpoint(
// our request. It is encrypted (with CLIENT_SECRET) and signed with redirect url.
// In state query parameter we will receive an url that was send as part of the request to Google.
if (req.query.code) {
if (optStringParam(req.query.code)) {
log.debug("GoogleAuth - response from Google with valid code");
messagePage(req, res, { code: req.query.code, origin: req.query.state });
} else if (req.query.error) {
log.debug("GoogleAuth - response from Google with error code", req.query.error);
if (req.query.error === "access_denied") {
messagePage(req, res, { error: req.query.error, origin: req.query.state });
messagePage(req, res, { code: stringParam(req.query.code),
origin: stringParam(req.query.state) });
} else if (optStringParam(req.query.error)) {
log.debug("GoogleAuth - response from Google with error code", stringParam(req.query.error));
if (stringParam(req.query.error) === "access_denied") {
messagePage(req, res, { error: stringParam(req.query.error),
origin: stringParam(req.query.state) });
} else {
// This should not happen, either code or error is a mandatory query parameter.
throw new ApiError("Error authenticating with Google", 500);
}
} else {
const oAuth2Client = _googleAuthClient();
const scope = req.query.scope || DRIVE_SCOPE;
const scope = optStringParam(req.query.scope) || DRIVE_SCOPE;
// Create url for origin parameter for a popup window.
const origin = getOriginUrl(req);
const authUrl = oAuth2Client.generateAuthUrl({