---
name: known-source-failures
description: Dead ends and traps when pulling museum and NASA scans. Use before scraping Getty, NGA, Met HTML, Commons thumbs, IIIF /full/max/, or any new host that looks promising.
---

# Known Source Failures

Record a new trap here in the same task that hit it. Do not re-learn these.

## Do not scrape HTML

- Met presentation pages. Use `pull-met-art-scans` / `scripts/pull-met.mjs`.
- `images.nasa.gov` HTML. Use `pull-nasa-images` / `scripts/pull-nasa.mjs`.
- Commons file pages or `/thumb/` URLs. Use `pull-wikimedia-commons` / `scripts/pull-commons.mjs`.
- Google Arts & Culture HTML. The large GAP files already live on Commons.
- `annualreport.gallery/covers/*.jpg`. Those are web thumbs (often 50–200 KB). Search with `pull-annual-reports`, then take page 1 of the Internet Archive scan.

## Caps that look like originals

- Commons `Special:Redirect/file/...&width=16000` and `Special:FilePath/...?width=16000` return the 3840 px thumb for *Night Watch* HD and *Prodigal Son*. Download `imageinfo[0].url`. A 16k viewer file has to be made off-box from that original.
- NGA IIIF `info.json` can report 20k+ while `/full/max/` serves 4096. *The Last of the Buffalo* is `10602×6252` on Commons; NGA max is 4096.
- Smithsonian IIIF caps *Among the Sierra Nevada* at 3000; Commons has `10500×6298`.
- Getty IIIF for *Europa* is `9710×7540`. Commons has `42582×33430`.
- Rijksmuseum IIIF `https://iiif.micr.io/PJEZO` is a collection JPEG, not the research scan. `info.json` claims 14,645×12,158 but `maxArea` serves about 7,133×5,922. Worse than Commons HD. A transient HTTP 500 on 2026-08-17 was a fetch-path blip; the endpoint later returned 200.
- Operation Night Watch is a 717 GP / 5.6 TB Micrio viewer at rijksmuseum.nl, 5 μm sampling (~5080 PPI). It is not a downloadable original. dezoomify-rs does not support Micrio. Do not stitch it on this VPS, even after a RAM upgrade, unless disk can hold terabytes.
- `hyper-resolution.org` hosts the 2019 20 μm / 44.8 GP Night Watch tiles. Commons `File:The Night Watch - HD.jpg` (`57813×48438`, ~324 PPI) is a reduced export of that viewer. Do not stitch the tiles here. Ingest the Commons HD file when this box has more than 2 GB RAM.
- Commons `File:Rembrandt Harmensz. van Rijn - Nachtwacht - Google Art Project.jpg` is only `28128×23334`. The HD file is larger.

## Met API traps

- `artistOrCulture=true&medium=Paintings` drops etchings and some oils (including Bierstadts). Search `hasImages=true&q=...` with `q` last, then filter in code.
- HTTP 403: back off several seconds. Do not hammer `/search` or `/objects`.
- `primaryImageSmall` and `additionalImages` are often details, versos, or smaller photos. Compare pixels before replacing.

## This VPS

- 2 GB RAM. Never decode a 20k+ original in PIL, Sharp, ImageMagick, `vips copy`, or `vips rotate`. When RAM is upgraded, ingest Night Watch HD and *Prodigal Son* next (16k view + `--pixels-from`). See `vps-memory` and `leads` in `/.well-known/image-sources.json`.
- A 42k Europa scan used 1.5 GB RSS, the kernel OOM-stalled, and Cloudflare returned Error 1033. A second lockup the same day needed a hard reboot.
- `vips` / `vipsthumbnail` on this box refuse files over 20000 px and are address-space capped. Do not set `VIPS_NO_MEMCAP=1`.
- Use `vips thumbnail` / `vipsthumbnail` only on files already at viewer size, or see `catalog-work`.
- `/tmp` is a 256 MB tmpfs. Stage large files on disk (`tmp-scans/`, `/var/tmp`).
- `earlyoom` is installed so a runaway decode is killed before the kernel freezes the box. Do not disable it.

## Same-work mistakes

- Kenwood/Rijksmuseum Google Art self-portrait `12243×15859` is not the Met 1660 Altman self-portrait.
- Whitney 60×96 in. *Last of the Buffalo* replica is not NGA 2014.79.5.
- A larger file of a different object must not replace a catalogued work to inflate PPI.

## Museum sites are provenance

Getty, NGA, MFA Boston, MFA Houston, Alte Pinakothek, Amon Carter, Cleveland, SAAM, the Norman Rockwell Museum, Rijksmuseum, Mauritshuis, the Hermitage, and NARA (`catalog.archives.gov`) appear as `recordUrl` / repository. Pixels for those works come from Commons (or Met/NASA). See `museum-records`.

- Do not scrape `rijksmuseum.nl`, `hyper-resolution.org`, or `iiif.micr.io` for Night Watch pixels. Use the Commons HD file, or leave the 717 GP composite as viewer-only.
- Do not scrape `hermitagemuseum.org` or `mauritshuis.nl` for pixels. Prodigal Son and Andromeda are on Commons.

- Do not scrape `catalog.archives.gov` for pixels. Concorde NARA 17426977 is already on Commons at `4645×5480`.
- Do not treat Century of Speed / Dryden aircraft stills as Hasselblad 70mm. Historic frames are 4×5; modern digital is 35mm full frame.
- Do not scrape `annualreport.gallery` HTML or ingest a full annual-report PDF. Cover page only, via `pull-annual-reports`. Internet Archive BookReader `/page/cover.jpg` often 502s; use the `_jp2.zip` first page or `archive.org/cors/{item}/{file}.pdf` page 0.
- University of Washington `digitalcollections.lib.washington.edu` annual reports have no pull skill. Skip them.