Fix card-data download for Scryfall's new gzipped-JSONL bulk API
"Download card data" had been failing on every platform with "missing field `download_uri`". Scryfall changed the bulk-data API: entries now carry `jsonl_download_uri`/`compressed_size` instead of `download_uri`/`size`, and the payload is gzipped JSONL (one card object per line) rather than an uncompressed JSON array. An existing reference.db keeps working, which is why this went unnoticed. - Rename the BulkEntry fields and decompress with flate2 (the response is Content-Type: application/gzip with no Content-Encoding, so ureq won't transparently inflate it). rust_backend keeps it pure Rust, so nothing new has to cross-compile for Android. - Replace the streaming JSON-array visitor with a line-based JSONL parser. Closure errors now propagate directly instead of being laundered through serde::de::Error::custom, so cancellation reports a clean Cancelled; override_if_cancelled stays as a safety net and its doc comment is corrected. - Progress wraps the compressed stream, since compressed_size is what the index reports. - Test fixtures stay readable pretty-printed JSON arrays via a jsonl() helper that converts them, so tests still exercise the production parser. Downloads are far smaller now: 101MB compressed (24 + 77) versus ~740MB, which matters a lot on a phone. Also log command failures to the platform log. Tauri command errors are returned to the WebView as values, so they never reached logcat and the UI only showed them truncated -- this bug was invisible until then. mlog! is a plain eprintln!, which tao's Android glue forwards to logcat under RustStdoutStderr, so no logging dependency is needed. View with: adb logcat -s RustStdoutStderr Verified against the live API: 36,242 oracle cards, 116,409 printings, trigram FTS working, and the aarch64 APK still builds.
This commit is contained in:
@@ -77,6 +77,8 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
|
||||
db.lock().unwrap().ingest_cancel = Some(cancel.clone());
|
||||
let app_for_progress = app.clone();
|
||||
|
||||
crate::mlog!("update_card_data: starting ingest into {}", reference_path.display());
|
||||
|
||||
let result = tauri::async_runtime::spawn_blocking({
|
||||
let cancel = cancel.clone();
|
||||
move || {
|
||||
@@ -89,10 +91,15 @@ pub async fn update_card_data(app: AppHandle, db: Db<'_>) -> Res<Status> {
|
||||
.map_err(|e| e.to_string())?;
|
||||
|
||||
db.lock().unwrap().ingest_cancel = None;
|
||||
result.map_err(|e| e.to_string())?;
|
||||
let stats = crate::logged("update_card_data", result.map_err(|e| e.to_string()))?;
|
||||
crate::mlog!(
|
||||
"update_card_data: ok ({} oracle cards, {} printings)",
|
||||
stats.oracle_cards,
|
||||
stats.printings
|
||||
);
|
||||
|
||||
let mut s = db.lock().unwrap();
|
||||
s.reopen()?;
|
||||
crate::logged("update_card_data reopen", s.reopen())?;
|
||||
Ok(status_of(&s))
|
||||
}
|
||||
|
||||
@@ -113,9 +120,15 @@ pub fn import_reference_db(db: Db, path: String) -> Res<Status> {
|
||||
return Err("That file is already the active card database.".into());
|
||||
}
|
||||
|
||||
crate::mlog!("import_reference_db: from {}", src.display());
|
||||
|
||||
// Validate before disturbing the working db, so picking the wrong file is a
|
||||
// harmless error rather than a broken install.
|
||||
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string())?;
|
||||
let cards = crate::logged(
|
||||
"import_reference_db validate",
|
||||
mtgc_core::db::inspect_reference_db(&src).map_err(|e| e.to_string()),
|
||||
)?;
|
||||
crate::mlog!("import_reference_db: source holds {cards} oracle cards");
|
||||
|
||||
// Release the handles on the file we're about to replace.
|
||||
s.close();
|
||||
|
||||
@@ -7,6 +7,30 @@ use tauri::Manager;
|
||||
|
||||
use state::AppState;
|
||||
|
||||
/// Write a line to the platform log.
|
||||
///
|
||||
/// On Android this reaches `adb logcat` under the tag `RustStdoutStderr`: tao's
|
||||
/// Android glue `dup2`s stdout/stderr into a pipe and forwards each line to the
|
||||
/// system log, so a plain `eprintln!` is all that's needed and no logging
|
||||
/// dependency is warranted. On desktop it's just stderr.
|
||||
///
|
||||
/// This matters because Tauri command failures are *returned* to the WebView as
|
||||
/// values, not raised — so without logging them explicitly they never appear in
|
||||
/// logcat at all, and the UI only ever shows a truncated version.
|
||||
#[macro_export]
|
||||
macro_rules! mlog {
|
||||
($($arg:tt)*) => { eprintln!("[mtgc] {}", format_args!($($arg)*)) };
|
||||
}
|
||||
|
||||
/// Log a command failure (with its full, untruncated message) and pass the
|
||||
/// result through unchanged.
|
||||
pub(crate) fn logged<T>(what: &str, result: Result<T, String>) -> Result<T, String> {
|
||||
if let Err(e) = &result {
|
||||
crate::mlog!("{what} failed: {e}");
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
#[cfg_attr(mobile, tauri::mobile_entry_point)]
|
||||
pub fn run() {
|
||||
tauri::Builder::default()
|
||||
@@ -17,6 +41,7 @@ pub fn run() {
|
||||
.app_data_dir()
|
||||
.expect("resolve app data dir");
|
||||
std::fs::create_dir_all(&data_dir).ok();
|
||||
mlog!("data dir: {}", data_dir.display());
|
||||
app.manage(Mutex::new(AppState::new(data_dir)));
|
||||
Ok(())
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user