What Is Log Retention?
Log retention is how long a log record stays in storage before it is deleted or archived. A retention period is set by policy. It decides whether an event from six months ago is still there when you go looking.
Teams keep these records for three reasons: troubleshooting faults, investigating security incidents, and answering audit questions later. Each reason wants a different amount of history.
A failed login this morning matters to whoever is on call. The same event matters to an auditor two years from now. In between, it matters to nobody.
Retention is the control that reconciles those facts. You pay storage cost for every day you keep an event, and you carry investigative risk for every day you do not.
Retention applies to the copy held in your central log platform. It is separate from log rotation, which governs what the originating system does with its own files on disk.
Which Types of Logs Does Retention Apply To?
Retention is set per log category, and four categories cover most estates. Each one earns its period for a different reason.
System logs: the operating system records reboots, kernel messages, and hardware faults. These get read during troubleshooting and rarely after that.
Application logs: your own services write errors, warnings, and transaction records. Volume here is high and the useful life is usually short.
Security logs: authentication attempts, privilege changes, and firewall decisions all land here. They carry the longest investigative tail of the four.
Audit logs: records of who accessed or changed what, kept as evidence. Audit logs outlive operational records by a wide margin. The question they answer arrives late.
We group logs this way because categories are what a retention policy names. A policy that lists individual servers stops working the day a new one appears.
What Drives a Log Retention Period?
Four factors set a retention period, and they pull against each other. Weighing them is the whole exercise.
Investigation windows: a responder needs to reach back to the first sign of a problem. Compromises are often found weeks after the event that started them. A short window can close before anyone has looked.
Audit and review cycles: evidence has to survive until somebody asks for it. Audit cycles run on their own calendar, and it is rarely yours.
Storage cost: every retained day is volume you are paying to hold. Cost scales with ingest rate. A high-volume source is expensive to keep for long.
Query performance: a longer searchable window puts more data behind every query. Search stays quick when your most-read events sit in a smaller tier.
Regulatory compliance obligations also shape retention. They vary by jurisdiction, sector, and data type. Confirm the requirement that applies to your organization rather than adopting a period you read somewhere.
What Are the Storage Tiers for Log Retention?
Storage is commonly split into three tiers, and each trades access speed against cost. Retention periods are usually set per tier rather than per source.
Hot storage: recent events stay immediately searchable here. This is the fastest tier to query and the most expensive per gigabyte.
Warm storage: events you open occasionally sit in this tier. Retrieval takes longer than hot storage and costs noticeably less.
Cold storage: long-term archives live here, often off-site. It is the cheapest tier to hold and the slowest to read back, which suits evidence you keep but rarely open.
Events route to a tier by source, severity, or criticality. Short-lived operational records and long-lived audit evidence then stop sharing a clock.
How Does Retention Relate to Log Indexing?
Retention is usually configured alongside log indexing. The index an event lands in is what sets how long it lives.
Treating an index as a storage tier collapses two settings into one. You define the tier, its filter, and its retention period together. Indexing and retention then cannot drift out of step.
An observability pipeline configures that routing once and applies it to every event arriving afterwards.
What Is a Log Retention Policy?
A log retention policy is the written rule set behind the setting. It says which logs you keep, for how long, and where they live. It also covers who can read them and what happens when the period expires.
A usable policy names log categories rather than individual sources. Grouping by category is what lets a new server inherit the right period on its first day.
Broader log management policies set the surrounding rules on collection, access, and review. We keep retention inside that document rather than in a platform setting nobody reads.
What Are Log Retention Best Practices?
Four practices keep retention working as an estate grows. Each one exists because of a failure teams hit without it.
Write the rules down: name the log categories, the period for each, and the tier they land in. A period that lives only in a platform setting is one nobody can review.
Automate expiry: expired events should leave storage on a schedule rather than when somebody notices the bill. Manual cleanup is how a generous period quietly becomes the largest line on a log platform invoice.
Match retention to your obligations: confirm what your jurisdiction, sector, and data type actually require before setting a number. An audit asking for records you no longer hold is not a gap you can close afterwards.
Review on a cycle: revisit periods when sources change or volumes grow. A very long window on a heavily queried tier slows routine lookups for everyone, and that only shows up months later.
Log search is where all of this surfaces. The window you set is the window you can query.
Explore More IT Terms
Browse our comprehensive IT glossary to learn more about technology terminology.