Skip to content

The archive you must keep and the data you should not

  • Home
  • Blog
  • The archive you must keep and the data you should not
The archive you must keep and the data you should not

Keep the records that law, contracts and disaster recovery demand — accounting records, tax filings, signed agreements and backups you have actually restored from — and delete everything else on a schedule. A data retention policy business teams can run classifies each data type, assigns it a lifetime, and automates both archiving and deletion, so storage and breach risk stop growing every year.

Key Takeaways

Retention comes down to two decisions per data type: how long it lives and who deletes it when the time comes. The archives you must keep are the ones law, contracts and recovery depend on. Everything else — stale personal data, dead logs, orphaned database copies — is a liability that grows quietly until you delete it on purpose.

  • Keep what law and recovery require: accounting records, tax filings, signed contracts, and backups you have tested by restoring.
  • Delete on a schedule: customer data past the relationship, abandoned accounts, raw logs, stale copies of production databases.
  • A policy is a one-page table — data type, lifetime, location, owner, disposal — not a binder nobody opens.
  • Enforce the lifetimes in the systems themselves: storage lifecycle rules, log rotation, backup expiry. Memory fails; automation does not.
  • Every record kept past its purpose widens the blast radius of a breach.
  • Backups are not archives. They answer "can we recover this week", not "can we prove this in 2030".
How a data retention policy runs from classification to verified deletionFive stages: classify every data store, assign a lifetime, automate expiry, test restores, then purge and log the disposal.How a retention policy runs, start to finish1Classifyevery store2Assign alifetime3Automatethe expiry4Test therestores5Purgeand log it
The retention loop for a small business: classify each data store, assign a lifetime, automate expiry, test restores quarterly, then purge and log the disposal.

What is a data retention policy in a business?

A data retention policy is a short written table that says, for each type of data your business holds, how long you keep it, where it lives, and who deletes it when the clock runs out. It covers databases, backups, logs, email, documents and analytics — anything that stores a record of a person or a transaction.

Two rows do most of the work: the row saying tax filings and signed contracts stay untouched for the statutory period, and the row saying application logs older than 60 days expire automatically on Sunday night. Explicit, boring, written down before anyone needs it. That is the whole spirit.

Why does keeping data too long hurt you in production?

Every retained record widens the blast radius of a breach: a customer table holding eight years of sign-ups is far greater exposure than one holding a single year. Long retention also inflates storage cost, slows restores, and turns every audit or subject-access request into an archaeology project across systems nobody remembers.

Blast radius — how much damage one incident can reach — grows with every row you retain. Privacy law points the same way: the GDPR's storage-limitation principle and Nepal's Individual Privacy Act 2075 both say personal data should exist only while you have a reason to hold it. In practice the pain arrives first — a restore that drags for hours because the database is 90 per cent dead rows, or a hosting bill that creeps up every quarter.

Which records must a business keep, and for how long?

Keep, at full fidelity, the records that law, contracts and disaster recovery depend on: accounting records and tax filings, signed contracts and invoices, statutory company registers, employment records, and the backups that let you survive a ransomware event or a botched migration. The exact period comes from your jurisdiction's tax and company law.

Six years is a common benchmark in several jurisdictions, but the number that matters is the one in your own tax law — confirm it with your accountant, not a blog. Contracts usually outlive the relationship by whatever period someone could still sue you. Then ask a different question: if a key supplier vanished tomorrow, could you rebuild from what you hold? What survives a supplier disappearing is an archive decision too, and so is which accounts a business should own outright.

Which data should you not keep, and when should it go?

Delete, or strip until it identifies nobody, any data with no remaining legal or operational purpose: customer records past the end of the relationship, abandoned accounts, card details you were never permitted to store, raw access and debug logs older than a few weeks, and stale copies of production databases on old servers and laptops.

The test is simple: could you defend holding this to the person it describes? Old form submissions from a 2019 campaign, comment spam with harvested addresses, chat transcripts from a tool you replaced — risk, no upside. Analytics retention is a frequent surprise; most teams can shorten it without losing anything they actually use, so check how your analytics setup was configured before assuming you need years of it.

Keep, tier or delete: a retention decision table by data typeRows mapping common business data types to keep, tier or delete decisions and the disposal method each needs.Keep, tier or delete?KeepAccounting records and tax filingsFull fidelity for the statutory period your tax law setsKeepContracts and signed agreementsLife of the contract plus the legal claim windowTierDatabase and file backupsWeeks hot, months in archive storage, then expireDeleteCustomer data past its purposeAnonymise what has value, then purge the personal fieldsShortenRaw access and debug logsRotate after a few weeks and compress automatically
How common business data types map to keep, tier or delete decisions, with the disposal method each one needs.

When does a business actually need a formal retention policy?

You need a written policy the day you hold personal data about customers, employees or users — for most businesses, that is day one. You can reasonably defer it only if you hold almost nothing personal, your accountant already owns the statutory archive rules, and every backup expires automatically. Otherwise write the table this month.

There is such a thing as overkill. A two-person consultancy living in cloud accounting and a shared drive does not need a twenty-page document; sensible defaults plus auto-expiring backups cover them. What nobody can outsource is the decision itself — the provider's defaults don't know your legal periods, so a named owner signs the table.

How do you build a retention schedule in a day?

Build the schedule as a one-page table with five columns: data type, retention period, storage location, owner and disposal method. Walk through every system that holds records, assign each type a lifetime and a name, then enforce those lifetimes in the systems themselves rather than in someone's memory.

  1. Inventory every store: production databases, each website, object buckets, email, code repositories, any old server or laptop that still boots.
  2. Classify what each holds: financial records, contracts, customer personal data, operational logs, credentials.
  3. Assign a lifetime per type: statutory minimums for tax and company records; the shortest defensible period for everything else.
  4. Choose disposal: hard delete, anonymise, or archive-then-delete. Deleting production rows is destructive — back up first and test the condition with a count.
  5. Name one owner per store. A person, not a department.
  6. Automate and review: enforce the lifetimes using the settings below, then revisit the table annually and whenever a new system arrives.

A filled-in example of the table itself:

Data typeKeep forDisposal
Accounting records, tax filingsStatutory period — confirm with your accountantArchive, never touch
Contracts and agreementsContract life plus the claim windowArchive, then delete
Active customer dataRelationship plus a grace periodAnonymise, then delete
BackupsWeeks hot, up to a year archivedAutomatic expiry
Application and access logsA few weeksRotation and overwrite

Where does retention actually get enforced?

Enforce retention in the systems that hold the data, because a policy in a drawer deletes nothing. Object storage expires through lifecycle rules, Linux logs rotate under logrotate, and databases shed old rows through a scheduled job. Three short, real examples cover most small-business setups.

{
  "Rules": [
    {
      "ID": "backups-archive-then-expire",
      "Status": "Enabled",
      "Filter": { "Prefix": "backups/" },
      "Transitions": [
        { "Days": 90, "StorageClass": "GLACIER" }
      ],
      "Expiration": { "Days": 365 }
    }
  ]
}

Applied with aws s3api put-bucket-lifecycle-configuration --bucket your-bucket --lifecycle-configuration file://lifecycle.json, this tiers anything under backups/ to archive storage after 90 days and deletes it at one year. It schedules real deletion, so a wrong prefix or a fat-fingered day count expires data you meant to keep. Start on a test prefix and read the Amazon S3 lifecycle documentation before applying.

/var/log/myapp/*.log {
    weekly
    rotate 8
    compress
    missingok
    notifempty
}

That keeps about two months of application logs, compressed, then reuses the space. The worst case if you get it wrong is shorter logs — which is why log rotation is a good first automation.

Database purges deserve more care. A job deleting expired sessions or stale accounts should be tested with a SELECT COUNT(*) on the same condition, run inside a transaction you can roll back, and preceded by a fresh backup. If you run PostgreSQL, its backup documentation explains how dumps, archives and expiry interact before you automate anything.

How do you verify a retention policy is working?

Verify two things every quarter: that what you must keep still restores, and that what you should have deleted is actually gone. Restore a random backup to a scratch server, run a count of records past their retention date, and review the storage expiry report. Log each check with a date and a name.

One honest caveat: you cannot surgically remove a row from last Tuesday's backup. You delete it in production and let backup expiry retire the old copies. Write that exemption into the policy so an auditor sees intent, not neglect. If nobody in-house owns the servers, this quarterly check is exactly what an ongoing maintenance arrangement should include.

The life of a record under a retention scheduleTimeline showing a record created, used in production, archived, held for the legal period, then deleted and logged.The life of a record under a retention scheduleCreatedEnters a system1Day 0Months 0–242Active useLive databaseWarm archiveCheaper, restorable3Years 2–6After expiry4DeletedPurged on scheduleDisposal loggedThe audit trail5Every purgeDeleted means gone from production; backup copies age out on their own expiry schedule.The disposal log is itself a record — keep it longer than the data it describes.
The life of a business record under a retention schedule, from creation through active use and warm archive to scheduled deletion, with the disposal log kept as proof.

What goes wrong when retention is only a document?

Failure looks quiet: a lifecycle rule applied to a test bucket but never to production, a logrotate file that shipped in the package but never landed on the server, or a purge cron that died months ago. Debug it by listing the actual expiry rules on each store and checking the last run time of every deletion job.

  • Backups taken before the policy keep expired data alive for another full cycle. Inventory them separately.
  • Soft-deleted rows still sit in the database until something purges them explicitly.
  • Anonymisation that misses the email address pasted into a free-text notes field. Search for identifiers after your first purge.
  • An orphaned copy of the database on a forgotten server, still billing and still leaking. These copies are the most common finding in a cleanup review.

What does this look like in practice?

Here is the pattern we meet most often: a site three years old, everything retained since launch, nobody able to say why. The cleanup takes about a week of part-time work and needs no new software — only a table, two scripts and a lifecycle rule.

Picture the e-commerce shop: every order, abandoned cart and newsletter signup since launch, most of those people long gone. Orders feed the books, so they move to encrypted archive storage with delete rights limited to one admin. Accounts dormant for two years go through an anonymisation script, dry-run counted first. Raw logs get 60-day rotation; the backup bucket gets the lifecycle rule above. Storage and restore times fall together, and the next breach reaches a year of customers instead of three. Staff deleting files by hand, the common alternative, lasts exactly as long as that person remembers. Doing nothing is the expensive option — the bill arrives as a breach, not a storage line.

In short

Write the one-page table: what you hold, how long, who deletes it. Keep the statutory records and the tested backups. Expire everything else automatically, inside the systems themselves. Verify quarterly with a restore and a count. That is a data retention policy a business can actually run — a few hours a quarter, and a far smaller breach when one comes.

People also search for

If this table does not exist for your business yet, our team can help you build it: a review of what you hold, a written retention schedule, and the expiry jobs and backup lifecycles that keep it running — alongside our web, WordPress and hosting work or as a standalone cleanup. Contact IT Gurkha and we will walk your systems with you.

Frequently asked questions

  • A written schedule that maps each class of data the business holds — invoices, contracts, application logs, customer records, backups — to a maximum retention period and a disposal action. It names an owner for each class, states where the data lives, and defines what happens at expiry: deletion, anonymisation or transfer to archive storage.

  • Financial records commonly need six to seven years in many jurisdictions because of tax and company-law limitation periods, but the exact figure depends on where you are incorporated and who regulates you. Confirm each period with your accountant or counsel, then record it per data class rather than applying one number to everything.

  • A backup is a short-window recovery copy that is overwritten as the rotation advances; its job is restoring yesterday, not preserving history. An archive is a separate copy kept for years, usually indexed, often in cheaper cold storage and sometimes immutable. Backups used as archives fail the moment a point-in-time restore falls off the rotation.

  • Records past their retention period, personal data you no longer have a lawful basis or business need for, and anything payment security rules forbid, such as card verification values, which must never be stored at all. Verbose debug logs containing customer data and uncontrolled production copies on laptops also belong on the disposal list.

  • Yes. Privacy regimes such as GDPR treat keeping personal data beyond its stated purpose or period as a breach in its own right, separate from any leak. Over-retention also widens breach notification scope and legal discovery, and the storage bill grows quietly, so the cost usually appears long before a regulator does.

  • A legal hold suspends scheduled deletion for data relevant to litigation, audit or investigation, overriding the retention policy until it is lifted. Operationally, the deletion job must check a hold list before purging, and you record when the hold began. Deleting held data automatically is the expensive failure mode, so test the override.

  • You rarely edit backups directly; you bound the problem instead. Keep backup retention short enough that erasures converge as backups expire, delete the data from live systems immediately, and document that backups are purged or expired within a stated window. If a backup is ever restored, rerun the purge job on the restored copy.

  • Use the mechanisms the platform provides: object storage lifecycle rules or write-once retention locks for immutable records, database purge jobs that delete on a dated column and log rows affected, and mailbox retention policies in your email suite. Rehearse any purge against a staging copy or dry run first, because a mistyped date predicate deletes the wrong decade.

  • Sample and reconcile. Check that objects due to expire actually left the bucket, that row counts for expired records trend toward zero, and that the deletion job's log shows runs and non-zero purges where expected. Retention fails silently in production: the job errors once, the log rotates, and the archive quietly doubles.

  • No. Unlimited retention inflates storage and retrieval costs, makes legal discovery slower and broader, and multiplies the personal data exposed in a breach. The workable middle path is tiering: move aged data into cheap archive storage with lifecycle rules, then delete at the period's end instead of letting keep-everything become the policy by default.

0 comments

Be the first to share your thoughts.

Leave a comment

Chat on WhatsApp