Loggly

Promptfree, for everyone, and the only version there is

Copy the prompt for Claude CodeCopy the prompt for CodexCopy the prompt for Cursor

Copy

Build me the log management that replaces Loggly.

For ordinary volumes this is a solved free problem, and the design below costs a small server.

STACK
- Vector on every machine as the collector
- Loki with object storage, or SQLite with FTS5 for genuinely small volumes
- Grafana for search and charts
- Node 20+ with Fastify for custom alerting
- Caddy in front

SIZE IT FIRST
- **Measure your daily log bytes before choosing anything.** Under a gigabyte a day, SQLite with FTS5 on one server is genuinely sufficient and needs nothing else running. Under fifty, Loki on a small machine with object storage. Above that, think properly
- The most common mistake in this category is deploying a cluster for a volume a single file could hold

THE PIPELINE
- Structured JSON from the application, one event per line, UTC timestamps
- Vector collects, parses, scrubs and forwards, with a disk buffer so a network problem does not lose logs or block the application
- **Never log synchronously to a network destination from application code.** A slow log endpoint becomes a slow application, and that is how a logging outage becomes a site outage

THE FIELDS THAT MAKE LOGS USEFUL
- timestamp, level, service, environment, host, correlation id, and the message
- **The correlation id, generated at the edge and passed through everything.** One request's whole journey in one query is the difference between logs and log-shaped noise
- The user identifier hashed, never raw, and never anything more personal than that

SCRUBBING, WHICH IS NOT OPTIONAL
- Strip authorisation headers, tokens, passwords and card numbers at the collector, before anything is stored
- Deny by default on patterns that look like secrets, and write a test that a known secret does not survive the pipeline. **Logs are usually the most widely readable copy of your data**

SEARCH AND ALERTING
- Search by label first, then by content within the narrowed set. This is what keeps it fast and cheap
- Alert on rates and on new error signatures, not on individual lines
- **Alert on silence**: a service that stops logging entirely, which no error rule detects
- Every alert carries the query that produced it, so the responder starts from the evidence

RETENTION AND COST
- Tiers: recent on fast storage, older in object storage, and a stated deletion date
- Bytes per service per day, published. That chart is the most effective cost control there is
- A per-service cap with an alert, so one broken loop does not fill a disk overnight

WHAT MATTERS MOST
Sizing honestly, buffering at the collector, and scrubbing before storage. Most log projects fail on the first, break the application on the second, and become a liability on the third.