Fix card-data download for Scryfall's new gzipped-JSONL bulk API

"Download card data" had been failing on every platform with "missing field
`download_uri`". Scryfall changed the bulk-data API: entries now carry
`jsonl_download_uri`/`compressed_size` instead of `download_uri`/`size`, and
the payload is gzipped JSONL (one card object per line) rather than an
uncompressed JSON array. An existing reference.db keeps working, which is why
this went unnoticed.

- Rename the BulkEntry fields and decompress with flate2 (the response is
  Content-Type: application/gzip with no Content-Encoding, so ureq won't
  transparently inflate it). rust_backend keeps it pure Rust, so nothing new
  has to cross-compile for Android.
- Replace the streaming JSON-array visitor with a line-based JSONL parser.
  Closure errors now propagate directly instead of being laundered through
  serde::de::Error::custom, so cancellation reports a clean Cancelled;
  override_if_cancelled stays as a safety net and its doc comment is corrected.
- Progress wraps the compressed stream, since compressed_size is what the index
  reports.
- Test fixtures stay readable pretty-printed JSON arrays via a jsonl() helper
  that converts them, so tests still exercise the production parser.

Downloads are far smaller now: 101MB compressed (24 + 77) versus ~740MB, which
matters a lot on a phone.

Also log command failures to the platform log. Tauri command errors are
returned to the WebView as values, so they never reached logcat and the UI only
showed them truncated -- this bug was invisible until then. mlog! is a plain
eprintln!, which tao's Android glue forwards to logcat under RustStdoutStderr,
so no logging dependency is needed. View with: adb logcat -s RustStdoutStderr

Verified against the live API: 36,242 oracle cards, 116,409 printings, trigram
FTS working, and the aarch64 APK still builds.
This commit is contained in:
2026-08-02 14:00:04 -05:00
parent 683f13e6bc
commit 94e4f25ec4
6 changed files with 138 additions and 76 deletions

View File

@@ -77,6 +77,8 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
db.lock().unwrap().ingest_cancel = Some(cancel.clone());
let app_for_progress = app.clone();
crate::mlog!("update_card_data: starting ingest into {}", reference_path.display());
let result = tauri::async_runtime::spawn_blocking({
let cancel = cancel.clone();
move || {
@@ -89,10 +91,15 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
.map_err(|e| e.to_string())?;
db.lock().unwrap().ingest_cancel = None;
result.map_err(|e| e.to_string())?;
let stats = crate::logged("update_card_data", result.map_err(|e| e.to_string()))?;
crate::mlog!(
"update_card_data: ok ({} oracle cards, {} printings)",
stats.oracle_cards,
stats.printings
);
let mut s = db.lock().unwrap();
s.reopen()?;
crate::logged("update_card_data reopen", s.reopen())?;
Ok(status_of(&s))
}
@@ -113,9 +120,15 @@ pub fn import_reference_db(db: Db, path: String) -> Res<Status> {
return Err("That file is already the active card database.".into());
}
crate::mlog!("import_reference_db: from {}", src.display());
// Validate before disturbing the working db, so picking the wrong file is a
// harmless error rather than a broken install.
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string())?;
let cards = crate::logged(
"import_reference_db validate",
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string()),
)?;
crate::mlog!("import_reference_db: source holds {cards} oracle cards");
// Release the handles on the file we're about to replace.
s.close();