Fix card-data download for Scryfall's new gzipped-JSONL bulk API

"Download card data" had been failing on every platform with "missing field
`download_uri`". Scryfall changed the bulk-data API: entries now carry
`jsonl_download_uri`/`compressed_size` instead of `download_uri`/`size`, and
the payload is gzipped JSONL (one card object per line) rather than an
uncompressed JSON array. An existing reference.db keeps working, which is why
this went unnoticed.

- Rename the BulkEntry fields and decompress with flate2 (the response is
  Content-Type: application/gzip with no Content-Encoding, so ureq won't
  transparently inflate it). rust_backend keeps it pure Rust, so nothing new
  has to cross-compile for Android.
- Replace the streaming JSON-array visitor with a line-based JSONL parser.
  Closure errors now propagate directly instead of being laundered through
  serde::de::Error::custom, so cancellation reports a clean Cancelled;
  override_if_cancelled stays as a safety net and its doc comment is corrected.
- Progress wraps the compressed stream, since compressed_size is what the index
  reports.
- Test fixtures stay readable pretty-printed JSON arrays via a jsonl() helper
  that converts them, so tests still exercise the production parser.

Downloads are far smaller now: 101MB compressed (24 + 77) versus ~740MB, which
matters a lot on a phone.

Also log command failures to the platform log. Tauri command errors are
returned to the WebView as values, so they never reached logcat and the UI only
showed them truncated -- this bug was invisible until then. mlog! is a plain
eprintln!, which tao's Android glue forwards to logcat under RustStdoutStderr,
so no logging dependency is needed. View with: adb logcat -s RustStdoutStderr

Verified against the live API: 36,242 oracle cards, 116,409 printings, trigram
FTS working, and the aarch64 APK still builds.
This commit is contained in:
2026-08-02 14:00:04 -05:00
parent 683f13e6bc
commit 94e4f25ec4
6 changed files with 138 additions and 76 deletions

View File

@@ -77,6 +77,8 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
db.lock().unwrap().ingest_cancel = Some(cancel.clone());
let app_for_progress = app.clone();
crate::mlog!("update_card_data: starting ingest into {}", reference_path.display());
let result = tauri::async_runtime::spawn_blocking({
let cancel = cancel.clone();
move || {
@@ -89,10 +91,15 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
.map_err(|e| e.to_string())?;
db.lock().unwrap().ingest_cancel = None;
result.map_err(|e| e.to_string())?;
let stats = crate::logged("update_card_data", result.map_err(|e| e.to_string()))?;
crate::mlog!(
"update_card_data: ok ({} oracle cards, {} printings)",
stats.oracle_cards,
stats.printings
);
let mut s = db.lock().unwrap();
s.reopen()?;
crate::logged("update_card_data reopen", s.reopen())?;
Ok(status_of(&s))
}
@@ -113,9 +120,15 @@ pub fn import_reference_db(db: Db, path: String) -> Res<Status> {
return Err("That file is already the active card database.".into());
}
crate::mlog!("import_reference_db: from {}", src.display());
// Validate before disturbing the working db, so picking the wrong file is a
// harmless error rather than a broken install.
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string())?;
let cards = crate::logged(
"import_reference_db validate",
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string()),
)?;
crate::mlog!("import_reference_db: source holds {cards} oracle cards");
// Release the handles on the file we're about to replace.
s.close();

View File

@@ -7,6 +7,30 @@ use tauri::Manager;
use state::AppState;
/// Write a line to the platform log.
///
/// On Android this reaches `adb logcat` under the tag `RustStdoutStderr`: tao's
/// Android glue `dup2`s stdout/stderr into a pipe and forwards each line to the
/// system log, so a plain `eprintln!` is all that's needed and no logging
/// dependency is warranted. On desktop it's just stderr.
///
/// This matters because Tauri command failures are *returned* to the WebView as
/// values, not raised — so without logging them explicitly they never appear in
/// logcat at all, and the UI only ever shows a truncated version.
#[macro_export]
macro_rules! mlog {
($($arg:tt)*) => { eprintln!("[mtgc] {}", format_args!($($arg)*)) };
}
/// Log a command failure (with its full, untruncated message) and pass the
/// result through unchanged.
pub(crate) fn logged<T>(what: &str, result: Result<T, String>) -> Result<T, String> {
if let Err(e) = &result {
crate::mlog!("{what} failed: {e}");
}
result
}
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
tauri::Builder::default()
@@ -17,6 +41,7 @@ pub fn run() {
.app_data_dir()
.expect("resolve app data dir");
std::fs::create_dir_all(&data_dir).ok();
mlog!("data dir: {}", data_dir.display());
app.manage(Mutex::new(AppState::new(data_dir)));
Ok(())
})