Paperless-ngx

Promptfree, for everyone, and the only version there is

Copy the prompt for Claude CodeCopy the prompt for CodexCopy the prompt for Cursor

Copy

Build me a document archive in the shape Paperless-ngx has: scanned paper, read, tagged and searchable, on my own machine — and read the warning first.

Read this first: Paperless-ngx is free and excellent, so nothing is being saved in fees. What you take on is real: the server, the storage, and the certainty that the only copy of a decade of correspondence does not vanish. If you are not going to run a verified off-site backup, do not put your only copies here.

STACK
- Node 20+ with Fastify
- SQLite through better-sqlite3, WAL mode, with FTS5
- tesseract for text recognition, ghostscript and qpdf for PDF handling, ImageMagick for images
- A worker for the consumption pipeline
- Caddy in front, on a private network

THE DATA MODEL
- documents: id, title, original_path, archive_path, sha256_original, sha256_archive, mime, bytes, page_count, created_date, added_at, modified_at, correspondent_id, document_type_id, storage_path_id, asn, content_text, language, ocr_confidence, deleted_at
- correspondents, document_types, tags, storage_paths — the four dimensions everything is filed by
- document_tags: document_id, tag_id
- matching_rules: id, target_kind, target_id, kind, pattern, is_insensitive — how a document gets filed automatically
- notes, custom_fields, custom_field_values
- tasks: id, kind, document_id, status, attempts, error, at — the consumption queue
- audit: id, document_id, action, actor, before_json, after_json, at — append-only
- The original file is never modified. Ever. The archive version with its text layer is a separate file, and both hashes are recorded

CONSUMPTION, WHICH IS THE PIPELINE THAT MATTERS
- A watched folder, an email address, and an upload endpoint. A scanner drops a file and everything else happens by itself
- Deduplicate by hash before anything else: scanning the same page twice is the most common event in this system
- Split multi-document scans on a separator page or a barcode, which is what makes a stack of post into a stack of documents
- Rotate by detected orientation, deskew, and remove blank pages, with the original untouched
- Text recognition with the right language, producing a searchable PDF that keeps the original image and adds an invisible text layer. Never replace the image with recognised text — the recognition is imperfect and the scan is the evidence
- Skip recognition where the PDF already has real text, and record which path was taken
- Record the confidence. A document recognised badly is a document you will never find again, and a low-confidence flag is what gets it re-scanned

FILING AUTOMATICALLY
- Matching rules per correspondent, type and tag: a literal phrase, any word, all words, a regular expression, or a fuzzy match
- Rules run against the recognised text and the filename; every match recorded with which rule fired, so a wrong assignment is diagnosable
- Optionally a simple learned classifier trained on what you have already filed by hand — but always as a suggestion the first time, never as a silent decision
- The date parsed from the document's own content in the formats your correspondence uses, falling back to the filename and then the file time, with the source recorded

FINDING THINGS, WHICH IS THE WHOLE POINT
- FTS5 over recognised text, titles, correspondents and notes, with stemming for your language
- Filters that compose: correspondent, type, tag, date range, and whether it has been read
- The matching text shown in the result with the page number, and the viewer opening at that page
- Saved searches as views
- An archive serial number, printed on the paper you keep, so a physical document and its scan can find each other in both directions

STORAGE
- Files laid out on disk by a template — year, correspondent, title — so they are usable with no software at all. That property is worth more than any feature here
- The database can be rebuilt from the files and their sidecars; write the rebuild command and test it
- Optional encryption at rest, with the honest note that server-side encryption protects against a stolen disk and nothing else

KEEPING IT
- Nightly backup of the database and the documents, off the machine, encrypted, with retention
- A verification job that re-hashes a sample of files and reports any that have changed. Silent corruption over ten years is exactly the failure this archive is meant to prevent
- A restore rehearsed onto a clean machine with the time it took written down
- Export everything — files, metadata as JSON, and the tag structure — in one command

THE INTERFACE
- A list with thumbnails, a viewer with the text layer selectable, and inline editing of the four dimensions
- Bulk editing, because filing happens in batches
- Fast on a phone, since that is where a photograph of a letter starts
- Dark and light

OPERATIONS
- .env: DATABASE_PATH, MEDIA_PATH, CONSUME_PATH, BASE_URL, SESSION_SECRET, OCR_LANGUAGES, IMAP_URL
- Migrations on boot, each once
- Behind a VPN rather than exposed; this is every piece of paper you own
- Health endpoint reporting queue depth, free space and the age of the last verified backup

WHAT MATTERS MOST
The consumption pipeline and the backup. Get deduplication, recognition and filing working on a hundred real documents, then set up the off-site copy and restore from it before you shred anything. The entire value of this system is that it is still there in ten years.

Give me the repository, the consumption worker, migrations, .env.example, the backup, restore and rebuild scripts, and a README that opens with the backup rehearsal.