garrettmills/mtgc
1
0
Fork 0
Code Issues 1 Pull Requests Releases Activity

Replace folder sync with an HTTP sync server, keeping the app offline-first

Browse Source 
Folder sync left too much to the user's sync tool: platform file access
(the Android SAF plugin), choosing the document, merging conflict copies —
and its merge base could fall out of step with the document (absorbing
conflict copies, "forget", picking a new Android folder), reading every
difference as a deletion.

The new mtgc-server crate stores one versioned library per user and accepts
a write only when it names the version it replaces. It never merges: devices
still do that with merge3, through a new sync cycle over a `Remote` trait in
mtgc-core. The base advances only on knowledge — to what was read, or to a
write the server confirmed — and is stored with the server and account it
belongs to, so a different account starts from an empty ancestor instead of
deleting. The working copy is only rewritten under the app-state lock, and
sync commands now run off the main thread.

Sign-in happens in the browser on the server's pages, via the owner's OIDC
provider and/or local username/password accounts (Argon2id, managed with
`mtgc-server user ...`), and returns a per-device revocable token to the app
by PKCE-protected code: a loopback port on desktop, the mtgc:// scheme on
Android.

The Info tab gains sign-in, the first-sync combine/replace choice, the
account's devices, and JSON export/import; the top bar shows sync state.
Includes a Dockerfile, Compose/Caddy example, and scripts to build and push
${DOCKER_REGISTRY}/mtgc/sync-server.
This commit is contained in:
Garrett Mills
2026-10-08 22:32:08 -05:00
parent df2cbb372a
commit de55fc634a
41 changed files with 6365 additions and 1662 deletions
Download Patch File Download Diff File Expand all files Collapse all files

156
README.md
View File

@@ -11,17 +11,19 @@ in a platform-agnostic Rust core, so both platforms share it.
```
src/ React + TypeScript UI (swappable presentation)
src-tauri/ Tauri v2 shell — exposes the core as typed commands
crates/mtgc-core/ Pure Rust: db, fuzzy search, filters, decks, ingester
crates/mtgc-core/ Pure Rust: db, fuzzy search, filters, decks, ingester, sync
crates/mtgc-server/ Optional sync server: one library per user, OIDC and/or passwords
```
Two files:
Two kinds of data, both in the OS app-data dir (e.g.
`~/.local/share/com.garrettmills.mtgc/`):
- **your library `.json`** — owned cards, decks, deck contents. Small,
readable, and the only file you name or move. Put it somewhere synced; see
[Sync](#sync).
- **your library** — owned cards, decks, deck contents. Lives on each device
and works fully offline; sign in to a sync server to share it between
devices (see [Sync](#sync)). **Export library…** in the Info tab saves it as
a JSON backup.
- **`reference.db`** — read-only card data built from Scryfall bulk. Bulky and
re-downloaded per device, so it's never synced. Lives in the OS app-data dir
(e.g. `~/.local/share/com.garrettmills.mtgc/`).
re-downloaded per device, so it's never synced.
## First-time setup
@@ -84,8 +86,8 @@ Notes:
- The `mtgc` binary is a single file but relies on the system WebKit/GTK libs
being present (installed by `devel-setup.sh`).
- The binary does **not** embed card data — first run still downloads the Scryfall
bulk into the OS app-data dir. Your library document defaults to that directory
too; move it into a synced folder with **New Library** (see [Sync](#sync)).
bulk into the OS app-data dir. Your library lives there too; to share it
between devices, see [Sync](#sync).
- The `[profile.release]` settings in `Cargo.toml` (strip + LTO) make the release
build take a few minutes but keep the binary small.
@@ -119,42 +121,118 @@ Marvel Commander — isn't hidden.
## Sync
Your collection is a single JSON file — the only file you manage. Put it in a
folder your sync tool watches (Syncthing, Dropbox, Nextcloud, Seafile, or a USB
stick) and every device stays in step. Choose it with **Open Library** /
**New Library**, or from the **Info** tab.
Your library always lives on the device you're using, so the app works the
same with or without a network. To keep several devices in step, run an
[mtgc sync server](#running-the-sync-server) and sign in to it from each one:
**Info → Sync server**, enter its address, and **Sign in…**. Sign-in happens
in your browser, on the server's own page — with a username and password, or
through your single sign-on provider, whichever the server offers — and the
browser hands the app back a token for that device alone.
Editing in two places merges rather than one side overwriting the other. Sync
is a three-way merge against the last version of the document that device read,
which is what lets it tell "you changed this" from "they deleted it": edit
different cards on a phone offline and on the desktop, and both survive. When
both sides change the *same* field, the other copy's value wins and the change
is reported so you can correct it.
Editing in two places merges rather than one side overwriting the other. Each
device remembers the last version of the library it knows the server had, and
merges three ways against it — which is what lets it tell "you changed this"
from "they deleted it": edit different cards on a phone offline and on the
desktop, and both survive. When both sides change the *same* field, the
server's value wins and the change is reported so you can correct it.
It runs by itself — at launch, on every switch away from and back to the app,
and every few minutes while open. **Sync now** in the Info tab forces a pass.
Sync runs by itself — at launch, on every switch away from and back to the app,
on reconnecting, and every few minutes while open. The top bar shows where it
stands (**Synced**, **Offline — changes will sync**, …); **Sync now** in the
Info tab forces a pass. Offline, edits simply wait on the device: nothing is
lost, and they go out with the next sync that reaches the server.
Nothing here depends on a particular sync tool. If two devices publish while
both are offline, some tools leave a conflict copy behind; mtgc merges any it
finds beside the document and clears it once the contents are safely published.
The first time a device signs in to an account that already has a library,
and the device has one too, it asks whether to **combine** them (nothing from
either is lost) or **use the server's library** instead of this device's.
> **Upgrading from an older build?** Earlier versions had you manage
> `library.db` yourself. The Info tab offers **Convert old library** once; the
> original file is left untouched, so it doubles as a backup.
The Info tab also lists the account's signed-in devices, so a lost phone can be
signed out from another one.
> **Upgrading from folder sync?** Earlier versions kept the library as a JSON
> file in a folder your sync tool watched. Your library is already on each
> device, so signing in to a server picks up from there; the first device to
> sign in uploads it. If one device's copy is out of date, use **Import library
> file…** on it to combine in the `library.json` from the synced folder. Builds
> older still had you manage `library.db` yourself; the Info tab offers
> **Import old library** once for that.
## Running the sync server
`mtgc-server` is a single binary with a SQLite database. Run it behind a
reverse proxy that provides HTTPS (the Android app refuses plain HTTP outside
debug builds). `crates/mtgc-server/deploy/` has a Docker Compose example with
Caddy:
```bash
cd crates/mtgc-server/deploy
cp .env.example .env # set MTGC_PUBLIC_URL, sign-in options
docker compose up -d
docker compose exec mtgc-server mtgc-server user add yourname
```
To build the image yourself and push it to your own registry:
```bash
DOCKER_REGISTRY=registry.example.com npm run sync-server:build # → registry.example.com/mtgc/sync-server:latest
DOCKER_REGISTRY=registry.example.com npm run sync-server:push
```
Both call `scripts/sync-server-image.sh`. `IMAGE_TAG` overrides the `latest`
tag, and `CONTAINER_ENGINE=podman` picks podman when docker is also installed.
Configuration is all environment variables:
| Variable | Default | |
|---|---|---|
| `MTGC_PUBLIC_URL` | *(required)* | The `https://` address users reach the server at, exactly as the proxy serves it |
| `MTGC_BIND` | `0.0.0.0:8080` | Listen address |
| `MTGC_DB` | `mtgc-server.db` | SQLite database path |
| `MTGC_LOCAL_AUTH` | `true` | Username/password accounts |
| `MTGC_OIDC_ISSUER` | — | Turns on single sign-on, e.g. `https://auth.example.com/application/o/mtgc/` |
| `MTGC_OIDC_CLIENT_ID`, `MTGC_OIDC_CLIENT_SECRET` | — | A confidential client, with redirect URI `${MTGC_PUBLIC_URL}/auth/callback` |
| `MTGC_OIDC_NAME` | `single sign-on` | Shown on the button: "Sign in with …" |
| `MTGC_OIDC_SCOPES` | `openid email profile` | |
| `MTGC_ALLOWED_EMAILS` | — | Comma-separated allow list of verified emails; unset leaves it to the provider |
With both sign-in methods on, the sign-in page offers both; with only single
sign-on, it goes straight to the provider. Each provider identity and each
local account is its own user with its own library.
Local accounts are managed from the command line; there's no sign-up page, so
a server on the internet can't be filled with accounts by strangers:
```bash
mtgc-server user add <name> [--display-name <text>] # prompts for a password
mtgc-server user passwd <name>
mtgc-server user remove <name> --yes # also deletes their server-side library
mtgc-server user list
```
The server keeps the last 50 versions of each library in its database
(`library_history`) as a safety net for recovering by hand.
## Develop / test
```bash
cargo test -p mtgc-core # core: db, search, filters, decks, ingest, sync merge
cargo test -p mtgc # shell: document read/write, conflict-copy matching
cargo test -p mtgc-core # core: db, search, filters, decks, ingest, sync merge + cycle
cargo test -p mtgc # shell: config, merge-base storage, sign-in listener
cargo test -p mtgc-server # server: sign-in, compare-and-swap, devices, end-to-end sync
npm run build # typecheck + build frontend
cargo build # build everything incl. the desktop app
```
The sync merge is a pure function over plain structs (`mtgc_core::sync::merge3`),
so the cases that matter — concurrent edits, a delete racing an edit, repeated
merges being a no-op — are unit tests rather than something you have to
reproduce across two devices.
and the sync cycle runs over a `Remote` trait (`mtgc_core::sync::remote`), so the
cases that matter — concurrent edits, a delete racing an edit, a lost reply, a
lost write race — are unit tests rather than something you have to reproduce
across two devices. `crates/mtgc-server/tests/sync_e2e.rs` then runs two
devices against the real server over HTTP.
To try sign-in by hand, run a server locally (`MTGC_PUBLIC_URL=http://127.0.0.1:8080
cargo run -p mtgc-server`, after `cargo run -p mtgc-server -- user add me`) and
sign in from the app with that address. From the Android emulator, the host is
`http://10.0.2.2:8080`; debug builds allow plain HTTP.
To typecheck the Android Kotlin without building the APK (it doesn't invoke
cargo, so it won't disturb a running dev session):
@@ -251,15 +329,11 @@ data…**. The file is validated before it replaces anything.
### Sync on Android
Android can't hand out a filesystem path to a shared folder, so instead of
naming a file you grant access to a *folder* through the system picker:
**Info → Choose folder…**, then pick the one your sync tool watches. mtgc keeps
`library.json` inside it and everything in [Sync](#sync) applies unchanged.
That grant is per-folder and persists across restarts, so the app declares **no
storage permission at all** — no "All files access" prompt. The alternative,
`MANAGE_EXTERNAL_STORAGE`, would hand the app the entire shared volume just to
reach one directory.
Signing in works as on desktop: the browser opens the server's sign-in page,
and returns to the app through the `mtgc://` URL scheme (registered by the
deep-link plugin from `plugins.deep-link` in `tauri.conf.json`). The app needs
no storage permission — the library lives in its private storage, and syncing
is network only.
## Branding