From 4a1099ad070f755de1c25ef36d8d524b3bd52234 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jaros=C5=82aw=20Sadzi=C5=84ski?= Date: Fri, 30 Sep 2022 17:26:31 +0200 Subject: [PATCH 1/2] Adding a description and instruction document on how to translate main application --- help/translations.md | 145 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100644 help/translations.md diff --git a/help/translations.md b/help/translations.md new file mode 100644 index 00000000..80b8d3d2 --- /dev/null +++ b/help/translations.md @@ -0,0 +1,145 @@ +# Internationalization and Localization + +## General description + +Localization support (translations) in Grist is implemented via +[https://www.i18next.com](https://www.i18next.com/overview/plugins-and-utils) javascript library. It +is used both on the server (node) and the client side (browser). It has very good documentation, +supports all needed features (like interpolation, pluralization and context), and has a rich plugin +ecosystem. It is also very popular and widely used. + +## Localization setup + +Resource files are located in a `static/locales` directory, but Grist can be configured to read them +from any other location by using the `GRIST_LOCALES_DIR` environmental variable. All resource files +are read when the server starts. The default and required language code is `en` (English), all other +languages are optional and will be supported if server can find a resource file with proper language +code. Languages are resolved hierarchically, from most specific to a general one, for example, for +Polish code _pl-PL_, the library will first try _pl-PL_, then _pl_, and then will fallback to a +default language _en_ (https://www.i18next.com/principles/translation-resolution). + +Here is an example of a language resource file `en.core.json` currently used by Grist: + +```json +{ + "Welcome": "Welcome to Grist!", + "Loading": "Loading", + "AddNew": "Add New", + "OtherSites": "Other Sites", + "OtherSitesWelcome": "Your are on {{siteName}}. You also have access to the following sites:", + "OtherSitesWelcome_personal": "Your are on your personal site. You also have access to the following sites:", + "AllDocuments": "All Documents", + "ExamplesAndTemplates": "Examples and Templates", + "MoreExamplesAndTemplates": "More Examples and Templates" +} +``` + +It maps a key to a translated message. It also has an example of interpolation and context features +in the `OtherSitesWelcome` resource key. More information about how to use those features can be +found at https://www.i18next.com/translation-function/interpolation and +https://www.i18next.com/translation-function/context. + +Both client and server code (node.js) use the same resource files. A resource file name format +follows a pattern: [language code].[product].json (i.e. `pl-Pl.core.json`, `en-US.core.json`, +`en.core.json`). Grist can be packaged as several different products, and each product can have its +own translation files that are added to the core. Products are supported by leveraging `i18next` +feature called `namespaces` https://www.i18next.com/principles/namespaces. + +## Translation instruction + +### Client + +The entry point for all translations is a function exported from 'app/client/lib/localization'. + +```ts +import { t } from 'app/client/lib/localization'; +``` + +It is a wrapper around `i18next` exported method with the same interface +https://www.i18next.com/overview/api#t. As a future improvement, all resource keys used in +translation files will be extracted and converted to a TypeScript definition file, for a “compile” +time error detection and and better development experience. Here are couple examples how this method +is used: + +_app/client/ui.DocMenu.ts_ + +```ts + css.otherSitesHeader( + t('OtherSites'), + ..... + ), + dom.maybe((use) => !use(hideOtherSitesObs), () => { + const personal = Boolean(home.app.currentOrg?.owner); + const siteName = home.app.currentOrgName; + return [ + dom('div', + t('OtherSitesWelcome', { siteName, context: personal ? 'personal' : '' }), + testId('other-sites-message') +``` + +_app/client/ui/HomeIntro.ts_ + +```ts +function makeAnonIntro(homeModel: HomeModel) { + const signUp = cssLink({href: getLoginOrSignupUrl()}, 'Sign up'); + return [ + css.docListHeader(t('Welcome'), testId('welcome-title')), +``` + +Some things are not supported at this moment and will need to be addressed in future development +tasks: + +- Date time picker component. It has its own resource files that are already imported by Grist but + not used in the main application. https://bootstrap-datepicker.readthedocs.io/en/latest/i18n.html +- Static HTML files used as a placeholder (for example, for Custom widgets). +- DocTours (guided tours) that can be embedded inside a Grist document. +- Formatting dates. Grist is using `moment.js` library, which has its own i18n support. Date formats + used by Grist are shared between client, server and sandbox code and are not compatible with + `i18next` library. + +### Server + +For server-side code, Grist is using https://github.com/i18next/i18next-http-middleware plugin, +which exposes `i18next` API in the `Request` object. It automatically detects user language (from +request headers) and configures all API methods to use the proper language (either requested by the +client or a default one). `Comm` object and `webSocket` API use a very similar approach, each +`Client` object has its own instance of `i18next` library configured with a proper language (also +detected from the HTTP headers). + +Naturally, most of the text that should be translated on the server side is used by the Error +handlers. This requires a significant amount of work to change how errors are reported to the +client, and it is still in a design state. + +Here is an example of how to use the API to translate a message from an HTTP endpoint in +`HomeServer`. + +_app/server/lib/sendAppPage.ts_ + +```ts +function getPageTitle(req: express.Request, config: GristLoadConfig): string { + const maybeDoc = getDocFromConfig(config); + if (!maybeDoc) { + return req.t('Loading') + '...'; + } + + return handlebars.Utils.escapeExpression(maybeDoc.name); +} +``` + +### Next steps + +- Annotate all client code and create all resource files in `en.core.json` file. Almost all static + text is ready for translation. +- Store language settings with the user profile and allow a user to change it on the Account Page. + Consider also adding a cookie-based solution that custom widgets can use, or extend the + **WidgetFrame** component so that it can pass current user language to the hosted widget page. +- Generate type declaration files at build time to provide `missing key` error detection as soon as + possible. +- Dynamically Include calendar control language resource files based on the currently selected + language. +- Refactor server-side code that is handling errors or creating user-facing messages. Currently, + error messages are created at the place where the Error has occurred. Preferably errors should + include error codes and all information needed to assemble the error message by the client code. +- Add localization support to the `moment.js` library to format dates properly according to the + currently selected language. +- Add support for custom HTML page translation. For example `custom-widget.html` From 03631a14547b72eaf1a1c263b5c1d2eb28b6a9c6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jaros=C5=82aw=20Sadzi=C5=84ski?= Date: Mon, 3 Oct 2022 10:47:42 +0200 Subject: [PATCH 2/2] Improving documentation. Renaming folder to 'documentation' --- app/client/ui/HomeIntro.ts | 2 +- {help => documentation}/translations.md | 9 +++++++-- static/locales/en.core.json | 1 + 3 files changed, 9 insertions(+), 3 deletions(-) rename {help => documentation}/translations.md (92%) diff --git a/app/client/ui/HomeIntro.ts b/app/client/ui/HomeIntro.ts index 8ed16ed1..ed2a6fbf 100644 --- a/app/client/ui/HomeIntro.ts +++ b/app/client/ui/HomeIntro.ts @@ -110,7 +110,7 @@ function makePersonalIntro(homeModel: HomeModel, user: FullUser) { } function makeAnonIntro(homeModel: HomeModel) { - const signUp = cssLink({href: getLoginOrSignupUrl()}, 'Sign up'); + const signUp = cssLink({href: getLoginOrSignupUrl()}, t('SignUp')); return [ css.docListHeader(t('Welcome'), testId('welcome-title')), cssIntroLine('Get started by exploring templates, or creating your first Grist document.'), diff --git a/help/translations.md b/documentation/translations.md similarity index 92% rename from help/translations.md rename to documentation/translations.md index 80b8d3d2..f13e666f 100644 --- a/help/translations.md +++ b/documentation/translations.md @@ -18,6 +18,12 @@ code. Languages are resolved hierarchically, from most specific to a general one Polish code _pl-PL_, the library will first try _pl-PL_, then _pl_, and then will fallback to a default language _en_ (https://www.i18next.com/principles/translation-resolution). +All language variants (e.g., _fr-FR_, _pl-PL_, _en-UK_) are supported if Grist can find a main +language resource file. For example, to support a _fr-FR_ language code, Grist expects to have at +least _fr.core.json_ file. The main language file will be used as a default fallback for all French +language codes like _fr-FR_ or _fr-CA_, in case there is no resource file for a specif variant (like +`fr-CA.core.json`) or some keys are missing from the variant file. + Here is an example of a language resource file `en.core.json` currently used by Grist: ```json @@ -81,7 +87,7 @@ _app/client/ui/HomeIntro.ts_ ```ts function makeAnonIntro(homeModel: HomeModel) { - const signUp = cssLink({href: getLoginOrSignupUrl()}, 'Sign up'); + const signUp = cssLink({href: getLoginOrSignupUrl()}, t('SignUp')); return [ css.docListHeader(t('Welcome'), testId('welcome-title')), ``` @@ -92,7 +98,6 @@ tasks: - Date time picker component. It has its own resource files that are already imported by Grist but not used in the main application. https://bootstrap-datepicker.readthedocs.io/en/latest/i18n.html - Static HTML files used as a placeholder (for example, for Custom widgets). -- DocTours (guided tours) that can be embedded inside a Grist document. - Formatting dates. Grist is using `moment.js` library, which has its own i18n support. Date formats used by Grist are shared between client, server and sandbox code and are not compatible with `i18next` library. diff --git a/static/locales/en.core.json b/static/locales/en.core.json index 1f1a2176..b9818c72 100644 --- a/static/locales/en.core.json +++ b/static/locales/en.core.json @@ -1,5 +1,6 @@ { "Welcome": "Welcome to Grist!", + "SignUp": "Sign up", "Loading": "Loading", "AddNew": "Add New", "OtherSites": "Other Sites",