PACKSAFE Split volumes with a verifiable manifest Techlosoft "Archive Vault" product line — Team variant WHAT PACKSAFE IS FOR -------------------- An archive that a team needs to move around is often too big to move as one file. It gets cut into pieces, and the pieces travel separately: different USB sticks, different uploads, different people's downloads folders. The moment they are reassembled somebody asks the only question that matters: "is this set complete and undamaged?" And when the answer is no, the second question: "which one of these twelve files is the bad one?" PackSafe answers both. It splits one tar+gzip stream into fixed-size volumes, checksums every volume individually, checksums the complete pre-split stream, and writes all of that into a manifest. Anybody holding the pieces can prove the set is restorable — or be told the exact volume that is missing, truncated or corrupt — before they try to restore. It is the split-and-prove half of the Archive Vault line. Its siblings: VaultZip makes encrypted archives, ArchiveGuard does seal/verify/shred, ZipDock inspects archives that arrived from somewhere else. None of them split, and none of them can tell you which piece is bad. THE FOUR COMMANDS ----------------- packsafe pack [ ...] --out NAME --volume-size SIZE [--apply] [--json] Build a tar+gzip stream of the inputs, cut it into NAME.001, NAME.002, ... each at most SIZE bytes, and write NAME.manifest.json. DRY RUN BY DEFAULT. Without --apply it computes the real stream, reports how many volumes there would be and exactly how big each one would be, and writes nothing at all. packsafe verify NAME.manifest.json [--json] Check every volume named in the manifest: present? right size? right SHA-256? Per volume the status is one of OK present, right size, right checksum MISSING the file is not there SIZE MISMATCH wrong length (truncated or padded) CHECKSUM MISMATCH right length, wrong bytes UNREADABLE present but could not be read Ends with a verdict on whether a restore can safely proceed. Exit 0 when the set is complete, exit 2 when it is not. packsafe restore NAME.manifest.json --out DIR [--apply] [--json] Verify first and REFUSE to restore if verification fails. Then reassemble the volumes in index order into a temporary file, check the reassembled stream against the whole-stream SHA-256 in the manifest, scan every tar entry for unsafe paths, and only then extract. DRY RUN BY DEFAULT. Without --apply it does every check and lists what would be extracted, and writes nothing into --out. A refused restore never creates --out at all. packsafe list NAME.manifest.json [--json] The archived file list with sizes, read straight out of the manifest. Nothing is read from the volumes, nothing is extracted, nothing is written. SIZES ----- --volume-size accepts raw byte counts and human sizes: 1000000 1MB 500KB 10MiB 2G 64KiB KB / MB / GB / TB are powers of 1000. KiB / MiB / GiB / TiB are powers of 1024. A bare number is bytes. Zero or negative is an error. Every volume except the last is exactly --volume-size bytes. The last one holds the remainder. That is a plain byte split of the stream, so the volume boundaries have no relationship to file boundaries: a single file can and usually does straddle several volumes. THE THREE LAYERS OF INTEGRITY CHECKING -------------------------------------- 1. Per-volume SHA-256. Catches a volume that is missing, truncated, padded, or has had bytes changed. Names the exact volume. 2. Whole-stream SHA-256 of the complete pre-split stream. Catches everything the per-volume check cannot: volumes reassembled in the wrong order, a manifest whose per-volume entries have been edited to match swapped files, a set assembled from two different pack runs. 3. Path scan before extraction. Every tar entry is checked before a single byte is written. Any entry that is absolute, has a drive letter, or contains a ".." path element is refused and the whole restore aborts without creating the output directory. EXIT CODES ---------- 0 success 1 usage error, bad arguments, unreadable or malformed manifest, I/O error 2 integrity failure: a volume is bad, the whole-stream hash does not match, or an unsafe entry path was found WHAT IS NOT IMPLEMENTED ----------------------- VOLUMES ARE NOT ENCRYPTED. This is deliberate and it is the single most important thing to understand about PackSafe. The volumes are plain tar+gzip bytes. Anybody who gets hold of them can read the contents. The manifest also lists every archived file path and size in clear text. Encryption is VaultZip's job in this product line — if the contents are sensitive, make a VaultZip vault first and pack the vault file with PackSafe. The checksums here prove integrity, not confidentiality, and they are not authenticated: someone who can rewrite a volume can also rewrite the manifest to match. Use PackSafe against accident and bit-rot, not against a motivated attacker. NO ERROR CORRECTION. There are no par2-style parity or recovery volumes. PackSafe can tell you volume 7 is corrupt; it cannot repair volume 7. A bad volume must be re-fetched from wherever the set came from. One bad byte in one volume means that whole volume is unusable — but only that volume, which is the point of splitting. NO RESUME OF A PARTIAL PACK. If a pack run is interrupted, the volumes it had already written are left on disk but no manifest exists, so the set is not usable. Re-run pack from the beginning. There is no "continue where it left off". Other current limits: - Regular files and directories only. Symlinks, devices, sockets and FIFOs are skipped with a warning on stderr, and are not restored. - File ownership (uid/gid) is not preserved. Permission bits and modification times are. - Extended attributes and ACLs are not preserved. - Compression level is not configurable; it is gzip default. - Restore always extracts the whole set. There is no selective restore of individual files, and no way to restore from a subset of volumes. - restore writes the reassembled stream to a temporary file in the system temp directory and removes it afterwards, so a restore needs free temp space equal to the compressed archive size. - No cloud or network targets. Everything is local paths. ROADMAP ------- Encryption AES-256-GCM per volume with an authenticated manifest, sharing VaultZip's key handling, so a split set can be confidential as well as verifiable. Parity / recovery par2-style recovery volumes so that a set can volumes survive the loss of N volumes without a re-fetch, turning "which one is bad" into "repaired it". Cloud-target upload pack straight to S3-compatible object storage and verify a remote set in place, so the volumes never need to touch a local disk twice. BUILDING -------- Go 1.24 or newer. Standard library only, no third-party dependencies. go build -o packsafe . Offline builds work: GOPROXY=off go build succeeds because nothing outside the standard library is imported. Prebuilt binaries in dist/: packsafe-linux-amd64 packsafe-darwin-amd64 packsafe-darwin-arm64 packsafe-windows-amd64.exe EXAMPLE SESSION --------------- $ packsafe pack project --out release --volume-size 1MB DRY RUN - nothing written. Re-run with --apply to create the volume set. ... Would write : 6 volumes + release.manifest.json $ packsafe pack project --out release --volume-size 1MB --apply Packed 6 files (5.0 MiB) into 6 volumes. $ packsafe verify release.manifest.json [ 1] release.001 OK 1000000 bytes ... [ 3] release.003 MISSING file not found 5 OK, 1 bad, of 6 volumes Verdict: INCOMPLETE - 1 of 6 volumes unusable, restore must not proceed ... re-fetch release.003 ... $ packsafe restore release.manifest.json --out ./out --apply 6 OK, 0 bad, of 6 volumes Whole-stream checksum OK. RESTORED - 6 entries (5.0 MiB) into ./out