Byte-range storage
Log Lens indexes your logs without hoarding a second copy of them. When it finds an event, it records where that event lives in the original file, not the full text repeated for every occurrence. This keeps the index small and lets you always retrieve the exact, untouched log text on demand.
How it works
Every occurrence stores a byte range - a byte_start and byte_end pointing back into its source file - plus a short context preview. The full event body is never copied per occurrence. Instead, each issue keeps a single trimmed sample of one representative event, and the true raw text stays in your log file where it already is.
When you open the raw event, Log Lens seeks straight to byte_start in the source file and streams the bytes up to byte_end. Nothing is reconstructed or re-parsed; you get the original slice verbatim.
occurrence to byte_start / byte_end to seek into source file to stream raw bytes
Because ingestion reads files by seeking to an offset and streaming forward, re-scanning a file only reads what's new rather than reloading the whole thing.
Caps you should know
Several limits keep any single giant event from bloating the index or a response. All are configurable.
| What it limits | Config key | Default |
|---|---|---|
| Bytes captured per event during parsing | ingestion.capture_limit | 4,194,304 (4 MiB) |
| Stored sample message / stack / context per issue | ingestion.message_limit | 120,000 |
| Context preview stored per occurrence | ingestion.context_preview_limit | 12,000 |
Raw retrieval is also hard-capped at 4 MiB per read, so an unusually large event returns its first 4 MiB rather than an unbounded response.
Things to keep in mind
- The source file must still exist. Raw retrieval reads from the original path. If the file was deleted, rotated away, or moved, the byte range can no longer be resolved.
- Ranges are per generation. Because offsets are file-specific, rotation is tracked so a range always maps to the right file.
- Samples are trimmed, raw is not. The sample shown on an issue is truncated to the limits above; the raw event is the real bytes from disk.