Fix card-data download for Scryfall's new gzipped-JSONL bulk API
"Download card data" had been failing on every platform with "missing field `download_uri`". Scryfall changed the bulk-data API: entries now carry `jsonl_download_uri`/`compressed_size` instead of `download_uri`/`size`, and the payload is gzipped JSONL (one card object per line) rather than an uncompressed JSON array. An existing reference.db keeps working, which is why this went unnoticed. - Rename the BulkEntry fields and decompress with flate2 (the response is Content-Type: application/gzip with no Content-Encoding, so ureq won't transparently inflate it). rust_backend keeps it pure Rust, so nothing new has to cross-compile for Android. - Replace the streaming JSON-array visitor with a line-based JSONL parser. Closure errors now propagate directly instead of being laundered through serde::de::Error::custom, so cancellation reports a clean Cancelled; override_if_cancelled stays as a safety net and its doc comment is corrected. - Progress wraps the compressed stream, since compressed_size is what the index reports. - Test fixtures stay readable pretty-printed JSON arrays via a jsonl() helper that converts them, so tests still exercise the production parser. Downloads are far smaller now: 101MB compressed (24 + 77) versus ~740MB, which matters a lot on a phone. Also log command failures to the platform log. Tauri command errors are returned to the WebView as values, so they never reached logcat and the UI only showed them truncated -- this bug was invisible until then. mlog! is a plain eprintln!, which tao's Android glue forwards to logcat under RustStdoutStderr, so no logging dependency is needed. View with: adb logcat -s RustStdoutStderr Verified against the live API: 36,242 oracle cards, 116,409 printings, trigram FTS working, and the aarch64 APK still builds.
This commit is contained in:
@@ -13,6 +13,11 @@ serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
nucleo = "0.5"
|
||||
ureq = { version = "2", features = ["json"] }
|
||||
# Scryfall serves the bulk downloads as `.jsonl.gz` with `Content-Type:
|
||||
# application/gzip` (not `Content-Encoding: gzip`), so ureq won't transparently
|
||||
# decompress them — we do it ourselves. Pure-Rust backend, so it cross-compiles
|
||||
# to Android without another C dependency.
|
||||
flate2 = { version = "1", default-features = false, features = ["rust_backend"] }
|
||||
anyhow = "1"
|
||||
thiserror = "1"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user