Log Retention Best Practices: How Long Should You Keep Your Logs?
Most log estates run on one retention period, chosen once and applied to everything. It is usually thirty or ninety days. Whoever installed the platform picked it.
That one number is wrong in both directions at once. It expires the security events you needed. It also holds debug output nobody has queried since it landed. Better log retention best practices break the setting apart, so each category gets its own log retention period and tier.
In this blog, you will:
Compare what too short costs against what too long costs.
Apply seven practices that cut the bill without deleting evidence.
Route events to a tier, then set the period there.
Copy a worked three-tier structure and fit it to your estate.
By the end you can defend every period you set.
Why One Retention Period Across Your Estate Costs You Twice
A single default fails in two opposite ways on the same day. Neither failure shows up in an alert.
Set it too short and the investigation runs out of history. According to IBM's Cost of a Data Breach Report 2026, the average breach takes 247 days to identify and contain.
A ninety-day window cannot reach that far back. The first sign of the incident expired while the platform ran perfectly.
On the other hand, if you set it too long, you pay twice over. Storage is the obvious cost. Query speed is the quiet one. Every search now scans a year of debug output to find yesterday's error.
We see the same shape in most estates. The setting was right for the first source anyone onboarded. Nobody revisited it as four hundred more arrived behind it.
What Are the Log Retention Best Practices?
Seven practices do most of the work here. The order below starts with what changes the bill fastest.
1. Cut the Volume Before You Extend the Time
Retention multiplies whatever you ingest, which means the cheapest retention decision you can make happens before retention starts at all. Drop the sources nobody queries. Then apply log sampling to the repetitive ones that only matter in aggregate.
Halve your ingest and you can afford twice the history for the same money. Teams reach for the period first because it is one field. The volume work pays back further, and we would rather see that done first.
2. Keep the Searchable Window Shorter Than the Retained Window
Keeping an event and searching it are two different costs. Most teams conflate the two. They pay hot-tier prices for a year of history they open twice.
Decide how far back an ordinary log search should reach. Let everything older sit somewhere slower and cheaper. Ninety searchable days behind a year of archive beats a year of everything, on cost and on speed.
3. Set the Period on the Tier, Not on the Source
A period attached to a source has to be maintained per source. That works at forty sources and collapses at four hundred. The newest one always inherits whatever the default was.
Attach the period to the tier instead, then route events into it. A new firewall lands in the security tier on day one and picks up the right period with nobody editing anything.
4. Size the Window From the Longest Investigation You Have Run
Your own incident history beats any published figure. Look at the last three serious investigations. Find how far back each one had to reach.
The longest of those becomes your working floor for security events. You can defend that number in a budget conversation. A benchmark copied from an article never survives one.
5. Protect the Categories You Cannot Recreate
Some records can be regenerated and some cannot. A performance metric rebuilds from a later sample. A record of who changed a permission exists once.
Put the second kind out of reach of cost-cutting, so that when budget pressure arrives the trim comes from volume and from the searchable window. Leave the audit trail alone.
6. Test the Restore Before You Depend on the Archive
Cold storage works only as well as the last time somebody read from it. Plenty of teams discover a broken restore path during the audit that needed it.
Pull a random week out of the archive once a quarter. Confirm the events come back complete and readable. The exercise takes an afternoon, and we have yet to meet a team that regretted running it.
7. Change a Period Forward, Never Backward
Extending a period only affects events arriving after the change. It recovers nothing that has already expired. Shortening one can delete on the day it applies.
Treat a reduction as a deletion and review it as one. We have watched a routine cleanup remove eight months of authentication history. The change looked like a configuration edit, so nobody escalated it.
How Do You Choose the Actual Number?
Three inputs you already hold will settle most periods. None of them is a benchmark.
The reach-back on your last real incident: Pull the timeline of your last serious investigation and measure the gap between the first suspicious event and the day someone noticed. A security tier has to survive that gap.
Your audit and review calendar: Evidence has to outlive the cycle that asks for it, and other people set that cycle. Ask whoever sits in those meetings.
Your own query history: Most platforms can show how far back people actually search. On most estates the answer comes back in days, which argues for a short hot tier and a long cheap one.
Those three inputs will disagree, and the disagreement is the useful part. An audit calendar asking for years sits oddly beside a query history measured in days.
Split by scope instead of averaging the numbers. The long requirement usually applies to a narrow set of events, so route those into their own tier and leave everything else short.
An averaged period costs too much across the bulk of your volume and still falls short for the evidence.
Write the result into your log management policies, not only into a settings screen. A period nobody can find is a period nobody reviews.
One boundary needs stating plainly. Regulatory obligations vary by jurisdiction, sector, and data type. Confirm what applies to your organization with your legal or compliance team. No published figure answers it for you.
A Worked Three-Tier Retention Structure
The table below splits a single default into three tiers. Every period in it is illustrative. They show the shape of the decision, not a recommended number.
Tier | What Routes Here | Illustrative Period | Why It Earns That Period |
Operational | Debug and info events from application and system sources | 14 to 30 days, illustrative | Read during an incident and almost never afterward |
Security | Authentication, privilege changes, firewall decisions | 12 months, illustrative | Carries the long investigative tail, measured in months |
Evidence | Records of who accessed or changed what | Set by your audit cycle | Cannot be recreated, and the question arrives late |
Two things about that structure matter more than the numbers in it.
The operational tier holds most of the volume on the shortest period, which is where the savings come from and where we start most tiering conversations. The evidence tier stays small enough that its period never becomes a budget argument.
Route events into these tiers by source, severity, or criticality. Let log indexing carry the lifetime from there. Your three periods then live in three places instead of four hundred.
What a Retention Policy Needs That a Platform Setting Cannot Hold
A platform setting records the number. A policy records what makes that number defensible. Five things belong in it.
An owner per tier: Name who signs off a change. A period should not get edited quietly during a cost review.
The reason for each period: Write down the reach-back or the audit cycle it came from. A period with a reason survives the next budget conversation.
The exception process: Say how a source earns a longer period than its tier, and name who approves it. Otherwise exceptions get made in the settings screen.
Evidence that expiry ran: Keep a record showing data left storage on schedule. Showing deletion happened matters as much as showing retention did.
A review trigger: Tie the review to an event, such as onboarding a source class or crossing a volume threshold. A date nobody watches will not do it.
Legal hold is the exception that catches teams out. When a hold applies, expiry has to stop for the affected records. That switch needs to exist before anyone asks for it.
Where to Start With Retention Tiers
Start with the tier that holds the most volume, because a change there shows up on the bill first.
List your top ten sources by volume: In most estates a handful of sources produce most of the events.
Ask which of them anyone searches: The gap between what arrives and what gets read surprises most teams.
Move the unread volume to a short tier first: Leave security and audit sources alone until the operational split works.
Set the searchable window from your query history: Push the remainder into cheaper storage instead of deleting it.
Write the three periods down with their reasons: Drafting the policy gets easier once the tiers exist and behave.
One question comes up on every migration of this kind. The events already sitting under the old default do not move when you change the setting, because a period applies to what arrives after it.
So plan for a stretch where two rules run at once. The old volume ages out on the old period while new events land in the tiers. The estate reads as inconsistent for exactly as long as that old period lasts.
Running this as configured stages on one path is what an observability pipeline does. In ObserveOps Infinity, an index doubles as a storage tier that carries its own filter and its own retention period.
Defining the tier and setting its lifetime becomes one action. Events route in by source, severity, or criticality, so operational output and audit evidence stop sharing a clock.
One limit is worth being honest about. No platform can tell you how far back your next investigation will reach.
It can only make the periods cheap to change and easy to see. We would rather see a team fix that visibility first, on whichever unified observability platform it already runs.
Give Every Log Category the Period It Earns
Log retention stops being a cost problem once it stops being one number. Split the setting into a few tiers and give each one a period you can explain. The argument then moves from how much history to which history.
The timing is the uncomfortable part. You set these periods before you know which investigation is coming. Some will turn out short, and the only fix applies to events that have not arrived yet.
So revisit the tiers whenever a source class changes or volume jumps. Check them against your log rotation settings while you are there. A year from now, the periods you can defend beat the history you cannot afford.
FAQs
How long should you keep logs?
Long enough to cover your longest realistic investigation. For security events that is usually months, not weeks. Operational output rarely earns more than a few weeks. Set the period per tier, and confirm any regulatory obligation with your legal team.
What is the 7 year retention policy?
Seven years is a period often associated with financial and audit record keeping. Whether it applies to your logs depends on your jurisdiction, sector, and data type. Confirm the requirement with your legal or compliance team before adopting any figure.
Does a longer retention period make log searches slower?
It can, because a larger searchable window puts more data behind every query. The fix is to keep the hot tier short and move older events to cheaper storage, so routine lookups scan days instead of months.
Can you recover logs after their retention period expires?
Not once expiry has run. Deletion is final, and extending the period afterward only covers events that arrive later. Treat any reduction in a period as a deletion request, and review it with the same care.
Who should own the log retention policy?
It works best with a named owner per tier. Security owns the security period, whoever answers audits owns the evidence period, and the platform team owns the operational one plus the routing that feeds all three.
Author
Ramya Shah
Technical Writer
Ramya Shah is a technical content writer with a computer engineering background and roots in automotive journalism. He covers IT Service Management, observability, IT operations, and AI-driven automation. An early adopter of AI-assisted writing workflows, he turns complex IT processes into clear, engaging content optimized for search and answer engines (AEO), lifting content output and organic visibility.
