LocalLens Techlosoft "Search Intelligence" product line - Team variant =========================================================== WHAT IT IS LocalLens searches many machines' indexes at once. Each teammate runs "locallens index" on their own machine, which produces one small JSON index file. Those index files are collected into a shared folder. LocalLens then queries ALL of them together, attributes every hit to the machine it came from, and recognizes when the SAME FILE (identical bytes, matched by SHA-256 content hash) exists on more than one machine. That last part is the reason this tool exists. A single-index search tool can tell you "this document is on this machine". It cannot answer "who on the team has a copy of this document?", because it only ever sees one machine. LocalLens answers that question. How LocalLens differs from its siblings: SearchForge - one persistent inverted index for one machine. IndexDesk - one index, with facet filtering on top. LocalLens - N indexes from N machines, searched together, with cross-machine identity by content hash. COMMANDS locallens index --out --machine [--max-size 5MB] Walk , index every text-like file, write one index file. Per document it records: relative path, size, mtime, and the SHA-256 hash of the file's contents. Tokenization splits on every non-alphanumeric character and lowercases, producing an inverted index of term -> (document, occurrence count). locallens search --index --index [...] [--dir ] [--machine NAME] [--limit N] [--json] AND-match: a document must contain EVERY term to be returned. Every supplied index is searched. Results are grouped by machine. A document whose content hash also exists on another machine is marked SHARED and all the machines holding a copy are listed. locallens duplicates --dir [--json] Content-hash groups that appear on 2 or more machines: the team's redundant copies, with wasted-byte totals. Wasted bytes for a group = (number of machines with a copy - 1) * size. locallens machines --dir [--json] Inventory of the loaded indexes: machine name, document count, total bytes, and when the index was built. locallens help | -h | --help Flags may be written before, after, or interleaved with positional arguments. --index is repeatable. --dir loads every *.json in a directory; --dir and --index can be combined and duplicates paths are de-duplicated. WHAT IS IMPLEMENTED - Building one index file per machine, from a real directory tree. - SHA-256 content hash, size and mtime recorded per document. - Federated AND search over any number of index files at once. - Per-machine attribution of every hit. - Cross-machine identical-file detection by content hash, including the SHARED marker in search results and the "duplicates" report. - Wasted-byte accounting for redundant copies. - Machine inventory reporting. - --machine filter to narrow results to one machine. - --limit (0 means unlimited; default 20). - --json output for search, duplicates and machines. - Fault isolation: a missing, unreadable, malformed, foreign or internally inconsistent index file is skipped with a stated reason on stderr (and in the JSON output), and every other index is still searched. - Binary files are skipped (NUL byte or a high proportion of control characters in the first 8 KiB). - Files above --max-size are skipped. Suffixes K/KB/KiB, M/MB/MiB, G/GB/GiB, T/TB/TiB are accepted and are all BINARY multiples: 1KB = 1024 bytes. - Search, duplicates and machines never write to index files. They open them read-only. Behavior in two ambiguous situations, chosen deliberately: - Two index files claiming the SAME machine name: "machines" lists both index files separately and prints a note naming the collision. For SHARED marking and for "duplicates", the machine NAME is the identity, so the two indexes count as ONE machine. This is what stops a re-indexed or copied index from inventing fake cross-machine duplicates and inflating the wasted-byte total. Search will show the document once per index file, both attributed to the same machine name. - An index with zero documents (for example a machine whose folder held only binaries): it loads normally, appears in "machines" with 0 documents, and contributes nothing to search or duplicates. WHAT IS *NOT* IMPLEMENTED - READ THIS BEFORE TRUSTING IT - There is NO NETWORK. Nothing is synced, served, discovered or pushed. Index files are shared MANUALLY: each teammate builds their index and copies the file into a common folder (a network share, a synced drive, a git repo, a USB stick). LocalLens only ever reads local files. "Federated" here means "searches many index files at once", not "talks to many hosts". - There is NO ACCESS CONTROL on the contents of a shared index. An index file is a plain, unencrypted JSON document. Anyone who can read it can read every indexed file's PATH, size, mtime and content hash, and can reconstruct much of the original text from the inverted index. Do not index a directory containing anything you would not hand to every person who can reach the shared folder. There is no redaction, no per-user filtering, and no way to share part of an index. - INDEXES ARE SNAPSHOTS AND GO STALE. An index describes a directory as it was at the moment "locallens index" ran. Files created, edited, moved or deleted afterwards are not reflected. A hit can point at a path that no longer exists, and a file present on a teammate's disk right now can be invisible because their index is a week old. The "built" timestamp in "locallens machines" is the only staleness signal, and nothing enforces freshness. Re-indexing is always a full re-scan; there is no incremental update. - No content extraction from binary document formats. PDF, DOCX, XLSX, PPTX, ZIP and similar are detected as binary and skipped entirely, so their text is not searchable. Only plain-text-like files are indexed. - No phrase search, no boolean OR or NOT, no wildcards, no prefix matching, no fuzzy matching, no stemming and no stop words. Terms match whole lowercased alphanumeric tokens only, and multiple terms are always ANDed. - Ranking is a plain sum of term occurrence counts. It is not BM25, TF-IDF or anything statistically calibrated. - Duplicate detection is exact-bytes only. A file that was re-saved, re-encoded, or differs by a single character is a different hash and is NOT reported as a duplicate. Same-named files with different content are correctly NOT duplicates. There is no near-duplicate or similarity detection. - No result snippets or match context, and no line numbers. - Nothing verifies that an index is honest. A hand-edited index file can claim any machine name, any path, and any hash. Structural corruption is detected; deliberate forgery is not. Only load indexes from teammates you trust. - Symlinks are not followed, and hidden files are indexed like any other. - Everything is loaded into memory. This is sized for teams sharing modest document folders, not for indexing whole filesystems. ROADMAP - A real index-sharing service with authentication, so indexes are published and fetched over the network instead of copied by hand, and so a stale index can be detected and refreshed automatically. - Incremental re-indexing: re-scan only what changed since the last run instead of rebuilding the whole index every time. - Per-user permissions, so a shared index cannot leak private paths - the ability to exclude subtrees, redact path components, and scope who may see which parts of a published index. - Content extraction from binary document formats (PDF, DOCX, XLSX, PPTX) so their text becomes searchable instead of skipped. BUILDING go build -o locallens . Go standard library only. No third-party dependencies, no network access needed to build. Prebuilt binaries for linux-amd64, darwin-amd64, darwin-arm64 and windows-amd64 are in dist/. EXAMPLE # each teammate, on their own machine locallens index ~/Documents --out ~/share/alpha.json --machine alpha # then, against the folder everyone copies their index into locallens machines --dir ~/share locallens search quarterly handbook --dir ~/share locallens search budget --dir ~/share --machine beta locallens duplicates --dir ~/share EXIT CODES 0 success, including a search that legitimately found no matches 1 bad invocation, unreadable input, or no usable index at all