# DownloadPilot (CLI prototype) DownloadPilot is a working prototype of the "Pro" tier of the same product line as the sibling GrabFlow tool. Where GrabFlow downloads exactly one URL per invocation, DownloadPilot's job is a persistent, crash-survivable download QUEUE: you add many URLs to a queue file (all at once or over time), then run it with a configurable number of concurrent workers, and it tracks each item's status -- queued, downloading, done, or failed -- in that same JSON queue file. That queue file is rewritten to disk after every single item finishes (success or failure), so the whole queue survives the process being killed partway through a run; resuming is just running `downloadpilot run` again against the same queue file, and only the items that are not yet done get (re)downloaded. Failed items can also be retried in isolation with `--retry-failed`. This persistent, concurrent, resumable queue -- not any single download's transfer mechanics -- is the genuinely different, "why pay more for Pro" mechanism versus GrabFlow's one-shot single download. This build downloads each queue item as a plain sequential HTTP transfer with optional SHA-256 checksum verification (it does not reimplement GrabFlow's segmented/range-based per-file downloading). Browser-capture integration and true per-item segmented downloads are on the roadmap; see ../plan.md for the full product plan. ## Build from source Requires Go 1.24+, no external dependencies. go build -o downloadpilot . Cross-compile for another platform: GOOS=windows GOARCH=amd64 go build -o downloadpilot.exe . GOOS=darwin GOARCH=arm64 go build -o downloadpilot . ## Usage downloadpilot add --queue --out [--sha256 HASH] downloadpilot run --queue [--workers N] [--retry-failed] downloadpilot status --queue [--json] ### add Appends one new item to a queue file, creating the file if it doesn't exist yet. Each item gets a simple incrementing string id and starts in `queued` status. --queue FILE Queue file to append to (created if it doesn't exist) --out FILE Output file path for this item (required) --sha256 HASH Expected SHA-256 hex digest for this item (optional; omit to skip verification for that item) Example: downloadpilot add --queue q.json https://example.com/a.bin --out a.bin downloadpilot add --queue q.json https://example.com/b.bin --out b.bin \ --sha256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 ### run Loads the queue and processes every item currently in `queued` status, plus (always) any item stuck in `downloading` status -- which can only mean a previous `run` was killed while that item was in flight, since DownloadPilot never persists `downloading` as a deliberate final state -- and, with `--retry-failed`, any item in `failed` status too. Work is distributed across up to `--workers` concurrent goroutines. --queue FILE Queue file to process (required) --workers N Number of concurrent download workers (default 4) --retry-failed Also (re)process items currently marked "failed" For each item, DownloadPilot downloads to `.part`, verifies its SHA-256 if one was given, and only renames it to the final `--out` path on full success. On any failure the partial file is removed, so a failed item never leaves anything at its final `--out` path -- the presence of that file is itself a reliable "this item really finished" signal, same principle as GrabFlow's `.part` convention. **The persistence guarantee**: after every individual item finishes (success or failure), DownloadPilot immediately rewrites the entire queue file to disk (via a temp-file-then-rename, so the write itself is atomic) with the updated statuses. It does not wait until the whole run finishes to save state. That is what makes the queue crash-survivable: if the process is killed at any point, everything that had already finished is correctly recorded as `done` or `failed` on disk, items that hadn't been picked up yet are still `queued`, and running `downloadpilot run` again against the same queue file picks up exactly where it left off -- it does not re-download anything already `done`, and it does not corrupt the queue file across separate process runs. Example: downloadpilot run --queue q.json --workers 4 downloadpilot run --queue q.json --retry-failed ### status Read-only. Prints a table of every item's id/status/url/out plus counts of done/failed/queued. Does not download anything. --queue FILE Queue file to inspect (required) --json Print machine-readable JSON instead of a table Example: downloadpilot status --queue q.json downloadpilot status --queue q.json --json ### Queue file format A JSON file listing queue items with persistent state: { "items": [ { "id": "1", "url": "http://.../a.bin", "out": "downloads/a.bin", "sha256": "", "status": "queued" }, { "id": "2", "url": "http://.../b.bin", "out": "downloads/b.bin", "sha256": "abc123...", "status": "done" } ] } `status` is one of `queued`, `downloading`, `done`, or `failed`. `downloading` is only ever a transient in-memory state while an item is actively being fetched; it should not appear in a queue file after a clean run finishes, but it can legitimately appear if the process was killed while that item was mid-transfer -- `run` always retries such items on the next invocation. `sha256` is optional per item; an empty string means no verification is performed for that item. ## Prebuilt binaries See ../downloads/ for prebuilt binaries (Windows/macOS/Linux) and CHECKSUMS.txt for their SHA-256 hashes. These are unsigned indie builds -- Windows SmartScreen and macOS Gatekeeper will warn on first run, which is expected until a code-signing certificate is in place.