=============================================================================== MirrorFlow 1.0.0 Two-way folder sync with real conflict detection Techlosoft - Sync Reliability Suite (Pro) =============================================================================== WHAT MAKES MIRRORFLOW DIFFERENT Every other tool in the Sync Reliability Suite moves data in one direction. SyncGuard mirrors A onto B. SyncProof fans one folder out to many. SyncLedger pushes and pulls over the network. FolderSync keeps a destination matching a source. All of them share one blind spot: when the two sides differ, they cannot tell "the other side changed" from "I changed". They just overwrite in the direction they were pointed. MirrorFlow keeps a persistent sync-state database recorded at the end of the last successful run. On the next run it compares three things - side A, side B and that recorded baseline - and can therefore say exactly what happened: unchanged identical on both sides new-on-A added on A since the baseline new-on-B added on B since the baseline changed-on-A modified on A only -> propagate A to B changed-on-B modified on B only -> propagate B to A deleted-on-A removed on A, untouched on B -> propagate the delete to B deleted-on-B removed on B, untouched on A -> propagate the delete to A conflict BOTH sides changed since the baseline, or one side deleted a file the other side modified The conflict row is the whole product. Without a baseline a sync tool has to guess, and guessing is how people lose work. INSTALL Precompiled binaries live in dist/: dist/mirrorflow-linux-amd64 dist/mirrorflow-darwin-arm64 dist/mirrorflow-darwin-amd64 dist/mirrorflow-windows-amd64.exe Or build it yourself. It needs Go 1.24+, uses the Go standard library only, has no third-party dependencies and needs no network access to build: go build -o mirrorflow . QUICK START 1) Record a baseline once, while the two folders already agree: mirrorflow init ~/work ~/backup/work --state ~/.mirrorflow/work.json init copies, moves and deletes nothing. 2) Later, see what changed on either side: mirrorflow status ~/work ~/backup/work --state ~/.mirrorflow/work.json 3) Preview the sync (this is a dry run - it writes nothing): mirrorflow sync ~/work ~/backup/work --state ~/.mirrorflow/work.json 4) Do it for real: mirrorflow sync ~/work ~/backup/work --state ~/.mirrorflow/work.json --apply COMMANDS mirrorflow init --state mirrorflow status --state [--json] mirrorflow sync --state [--policy P] [--apply] [--json] mirrorflow help | -h | --help mirrorflow version init Records every path whose contents are currently identical on both sides as the baseline. Paths that already differ are deliberately NOT recorded, so the first status/sync run reports them honestly instead of pretending a side is authoritative. status Read-only. Classifies every file on both sides. Never writes. sync Propagates one-sided changes in the correct direction and handles genuine conflicts according to --policy. Writes nothing unless --apply is given. FLAGS --state Path to the persistent sync-state database. Required. --policy Conflict policy for sync. Default: manual. --apply Actually perform the changes. --json Machine-readable report on stdout. -h, --help Usage. Flags may appear before, after or between the two directory arguments. Both --state and --state= work. The command name must come first. CONFLICT POLICIES manual (default) Do not resolve anything. Report the conflicts, change NOTHING on either side, leave the state file alone and exit with code 2. If a conflict exists, the entire run is refused - not just the conflicting file - so you can never end up half-synced. newest The more recently modified side wins. The losing copy is moved into the loser's .mirrorflow-trash/ first, then the winning content is copied over. larger The larger file wins, same trash-then-copy sequence. keep-both Nothing is overwritten and nothing is trashed. The losing copy is renamed to .conflict-- and BOTH files end up on BOTH sides. The newer side keeps the original name. Delete-versus-modify conflicts (one side deleted a file the other side edited) are treated specially: under newest, larger and keep-both alike, MirrorFlow always keeps the edited file and restores it to the side that deleted it. Bytes that exist can be recovered; bytes that were deleted cannot, so the surviving edit always wins. Under manual it is simply reported and nothing happens. HOW COMPARISON WORKS Files are compared by SHA-256 of their contents, on both sides, against the hash recorded in the baseline. Modification times are used only to break conflict ties under --policy newest. This means touching a file, restoring it from an archive or re-saving identical bytes does NOT trigger a copy - the file is correctly reported as unchanged. SAFETY MODEL * Dry run is the default. Without --apply, MirrorFlow does not create, modify, move or remove a single byte on either side, and does not update the state file. * Every copy is written to .part and then renamed into place, so a crash or a full disk can never leave a half-written file under the real name. * Nothing is ever hard-deleted. When a file has to go - a propagated delete, or a losing copy under --policy newest/larger - it is MOVED into .mirrorflow-trash/ inside that side's root, preserving its relative path. Emptying that directory is your decision, not the tool's. * .mirrorflow-trash/ is skipped when scanning, so trashed files are never synced back or counted as new. * The state file is rewritten only after an --apply run in which every single operation succeeded. It is written via a .part file and renamed, so an interrupted run cannot corrupt your baseline. If any operation fails, the baseline is left untouched and the next run re-detects the work. * The two directories may not be the same directory, and neither may be nested inside the other. EXIT CODES 0 success 1 usage error, missing directory, missing/corrupt state file, or an I/O failure during --apply 2 unresolved conflicts (policy=manual); nothing was changed JSON OUTPUT --json prints one JSON object with: tool, version, command, dirA, dirB, stateFile, policy, apply, dryRun, entries[] (path, status, detail, resolution, hashA, hashB, hashState, sizeA, sizeB, modA, modB), actions[] (kind, path, dest, from, to, side, bytes, desc), summary{status: count}, conflicts, unresolved, bytesMoved, refused, stateUpdated and errors[]. Diagnostics always go to stderr, so stdout stays parseable. ------------------------------------------------------------------------------- WHAT IS IMPLEMENTED ------------------------------------------------------------------------------- * Persistent sync-state baseline (single JSON file you choose the path of). * True three-way comparison: side A, side B and the baseline. * Bidirectional propagation of one-sided edits and one-sided new files, each in the correct direction, in the same run. * SHA-256 content comparison, so mtime-only changes are not copies. * Real conflict detection: both-sides-modified, both-sides-created-differently and delete-versus-modify. * Four conflict policies: manual (default, refuses), newest, larger, keep-both. * Delete propagation into a trash directory instead of unlinking. * Arbitrarily nested subdirectories, unicode and spaces in names, empty files, and binary files of any size. * Dry run by default; --apply required for any write. * Human-readable and --json output. * Local filesystem paths, including anything your OS has mounted (a network share mounted as a normal directory works fine). ------------------------------------------------------------------------------- WHAT IS NOT IMPLEMENTED ------------------------------------------------------------------------------- These are honest limitations, not oversights. Do not assume any of them. * NO real-time watching. MirrorFlow runs when you run it. There is no daemon, no inotify/FSEvents/ReadDirectoryChangesW watcher and no scheduling. Pair it with cron, systemd timers or Task Scheduler yourself. * NO network or cloud endpoints. Both sides must be ordinary directory paths on the machine running the tool. There is no SSH, SFTP, WebDAV, S3, Google Drive, Dropbox or OneDrive support, and no client/server mode. * NO block-level delta transfer. A changed file is copied in full, every time. There is no rsync-style rolling checksum, no chunk reuse and no compression on the wire. Editing one byte of a 4 GB file copies 4 GB. * NO move/rename detection. A file you renamed on one side looks exactly like a delete plus a create, and is synced as one. The content is never lost - the old path lands in .mirrorflow-trash/ and the new path is copied across - but the rename is not recognised as a rename, so the transfer is a full re-copy rather than a cheap rename on the other side. * NO directory-only sync. Empty directories are not created on the other side; directories appear as a side effect of the files inside them. * NO symlink, hardlink, device-node, permission, ownership or extended attribute sync. Non-regular files are skipped and reported as skipped. Copies keep the source file's permission bits and modification time; owner, group and xattrs are not carried over. * NO three-or-more-way sync. One state file describes exactly one pair of directories. Syncing a third machine needs its own pairing and its own state file, and MirrorFlow will not reason about the two pairings together. * NO partial-run resume, locking or concurrency control. Do not run two MirrorFlow processes against the same pair at once, and do not edit the folders while a --apply run is in progress. * NO automatic trash pruning. .mirrorflow-trash/ grows until you empty it. * NO encryption, versioning or history beyond the single trash copy. ------------------------------------------------------------------------------- ROADMAP ------------------------------------------------------------------------------- 1. Real-time watching - a resident mode using inotify, FSEvents and ReadDirectoryChangesW, with debouncing, so changes propagate seconds after they happen instead of on the next manual run. 2. Network and cloud endpoints - treating a remote side as a first-class endpoint over SSH and object storage, so dirA and dirB no longer both have to be local paths. 3. Block-level delta transfer - rolling-checksum chunking so a small edit to a large file transfers only the changed blocks. 4. Moves and renames detected as such - matching content hashes across paths within a run so a rename is applied as a rename on the other side rather than as a delete plus a full re-copy. ------------------------------------------------------------------------------- KNOWN BEHAVIOURS WORTH KNOWING ------------------------------------------------------------------------------- * If you run init while the two sides already differ, the differing paths are not recorded. The next run reports them as new-on-A / new-on-B, or as a conflict if the same path exists on both sides with different content. That is intentional: init refuses to invent a baseline it cannot prove. * After a copy, the destination file is given the source file's modification time so that the two sides stay comparable and the next run is a clean no-op. * Under --policy newest, two conflicting versions with exactly the same modification time cannot be separated and are reported as unresolved. Under --policy larger, the same applies to two versions of the same size. MirrorFlow will not flip a coin with your data. * A file that vanished from both sides is simply dropped from the baseline. Techlosoft - Sync Reliability Suite