Search This Blog

Thursday, September 17, 2026

SQL Server Transaction Log Forensics: Preserving Evidence During an Incident

SQL Server Transaction Log Forensics: Preserving Evidence During an Incident SQL Server Transaction Log Forensics: Preserving Evidence During an Incident
This is the companion to Why sys.fn_dblog Is Still Undocumented And Still There. That one was about why the transaction log stays undocumented. This one is about how:  What to do in the first fifteen minutes after someone says "the data's gone," when the log still holds the answer and almost everything your instance does routinely is quietly destroying it.

A note on scope: this is a preservation and investigation guide, not a full disaster-recovery playbook. If you are facing a real production data-loss incident, contact Microsoft Support right away, alongside your internal incident-response team. Do not wait until you have tried everything here before escalating. The goal is to preserve your options while help is being brought in, not to replace support-led recovery guidance.

The log is designed to forget

Here's the the problem. The transaction log is the only place that remembers what actually happened, and the log is designed to forget. Truncation, checkpoints, log backups, shrink jobs, and instance restarts are all normal, healthy behaviors and every one of them can take your evidence with it.

Worse, several of the instincts that feel most responsible during an incident are the ones that do the damage:

  • "Let me take a log backup to be safe."  alears the active log.
  • "Let me restart the instance and see if it clears up." triggers recovery and a checkpoint.
  • "The log's grown huge, let me shrink it." actively reclaims the space holding your evidence.
  • "Let me detach and copy the files somewhere safe." worst case, you can't reattach cleanly.

So the first rule is unglamorous: stop touching things. The second rule is that preservation comes before investigation, and investigation comes before recovery. Get those in the wrong order and you spend the rest of the night wishing you hadn't.

First: Stop The Bleeding

Before you query anything, do these.

Announce a change freeze on the affected database. Application deploys, maintenance jobs, ad-hoc cleanup scripts all paused. Every write pushes your evidence closer to the edge of the active log.

Disable the jobs that destroy evidence, in this order:

  1. Any log shrink job. Shrink reclaims exactly the space your evidence is sitting in. If the volume is genuinely full you may have no choice but to shrink, but exhaust the alternatives first: free space elsewhere on the drive, or add a second log file on another volume to buy room. Shrink last, knowing you're trading evidence for space.
  1. Any index rebuild or maintenance job. These generate enormous log volume and will push your records out.
  1. Hold off on the log backup schedule, but read the disk-space warning below before you disable it.

Do not, under any circumstances yet: restart the instance, detach the database, run CHECKPOINT manually, take a non-copy-only log backup, fail over the AG, or start a restore over the top of the live database.

Write down the time. Every step from here should be timestamped in a notes file. You will need it, either for the postmortem or because someone will eventually ask you to prove what you did.

Step 1: Find out whether you have a game to play

Your recovery model determines everything that follows.

SELECT name, recovery_model_desc, log_reuse_wait_desc
FROM sys.databases
WHERE database_id = DB_ID('YourDatabase');

FULL: You're in the best position. Log records survive until a log backup truncates them, and your backup chain lets you restore to a specific LSN.

BULK_LOGGED: Mostly like FULL, with a caveat covered later.

SIMPLE: The news is bad. A checkpoint clears the log, and checkpoints happen constantly. There is no point-in-time recovery. Your realistic options are the last full/differential backup and whatever non-log evidence you can find. Don't spend twenty minutes on fn_dblog hoping; check it just once, and if it's empty, move on.

That log_reuse_wait_desc column is worth a second look, because for once it may be working in your favor. If it shows LOG_BACKUP, ACTIVE_TRANSACTION, AVAILABILITY_REPLICA, or a replication reason, something is preventing truncation right now, meaning your evidence is being held in place by the very thing that normally annoys you (Troubleshoot a full transaction log). The same value is exposed as log_truncation_holdup_reason in sys.dm_db_log_stats.

On an Availability Group, AVAILABILITY_REPLICA means the primary is waiting to ship log to a secondary that's lagging or down. Annoying on any other day, it's a gift during an incident.

Step 2: Get the evidence out of the log

This is the single most important step, and the one most people skip. Don't investigate in the log. Copy the log's contents into a table you control, then investigate the copy.

USE YourDatabase;
GO
SELECT * INTO Forensics.dbo.LogDump_20260916
FROM sys.fn_dblog(NULL, NULL);

Persisting the output into another database is a well-worn technique (Filter Results of fn_dblog function). Two reasons it matters enormously here:

  • Truncation can no longer hurt you. The records are now rows in a table with their own backup and recovery path.
  • The investigation gets dramatically faster. Reading the log is slow, and you will query this data dozens of times as you narrow the search. The same advice applies when reading backups with fn_dump_dblog.

Put the target table in a different database, ideally on a different instance. Writing your evidence into the database you're investigating adds log volume to the very log you're trying to preserve.

On a busy database the active log can hold millions of records, so if the unfiltered SELECT INTO is too heavy, bound it by LSN range or filter on Operation, but try to capture it all first if you possibly can. You can always narrow a table you already have.

Step 3: Capture a file, without truncating anything

A table copy is perfect for the active log. But if the records you want have already rolled out, you need backup files and getting one without destroying the active log is where most people go wrong.

A routine log backup truncates. Microsoft's BACKUP reference says it outright: "After a typical log backup, some transaction log records become inactive, unless you specify WITH NO_TRUNCATE or COPY_ONLY".

On a healthy, online database use a copy-only log backup:

BACKUP LOG YourDatabase
TO DISK = 'E:\Forensics\YourDatabase_incident.trn'
WITH COPY_ONLY, INIT, CHECKSUM;

A copy-only log backup preserves the existing log archive point and doesn't affect the sequencing of your regular log backups (Copy-only backups). You get a readable .trn file, your active log stays intact, and your backup chain is undisturbed.

For a damaged database you intend to restore, take a tail-log backup:

BACKUP LOG YourDatabase
TO DISK = 'E:\Forensics\YourDatabase_tail.trn'
WITH NORECOVERY, NO_TRUNCATE;

If the database is online and you plan to restore it, back up the tail of the log first, using WITH NORECOVERY to avoid an error on an online database (Tail-log backups), and the point-of-failure restore procedure uses exactly this NORECOVERY, NO_TRUNCATE form. Note what NORECOVERY does: it leaves the database in RESTORING state. That's correct when you're committing to a restore, and completely wrong when you're still investigating a database that users are hitting. Know which situation you're in.

If the log is damaged badly enough that NO_TRUNCATE fails, you can attempt the tail-log backup with CONTINUE_AFTER_ERROR instead. And if the log files are damaged and no tail-log backup is possible at all, you restore without one and accept losing everything committed since the last log backup.

Also grab a copy-only full backup if you don't have a recent one. Full backups don't truncate the log, and having a known-good starting point makes every later decision reversible.

Collect the existing log backup chain, too. Copy the .trn files from the incident window somewhere your retention cleanup can't reach. Your own retention job is a real threat here as it deletes old backups on schedule and doesn't know today is special.

Step 4: The log backup schedule question

Now the trade-off I got wrong the first time I thought this through, and which deserves stating plainly.

Pausing the log backup schedule protects the active log. It also means:

  • The log grows unchecked. On a busy system it can fill the drive and take the database offline. You'd convert a data-loss incident into an outage,  a strictly worse incident.
  • Your recovery position stops advancing. From the moment you stop taking log backups, you can no longer restore to any point after the last one. If the investigation goes sideways and you do need a point-in-time restore, you've narrowed your own options.

So the answer isn't "pause the schedule." It's "make the schedule irrelevant first." Steps 2 and 3 do exactly that: once the records are in a table and in a copy-only backup file, a routine log backup truncating the live log costs you nothing.

If you genuinely need to pause it and you haven't captured anything yet, and you need minutes to get organized then pause it, watch free disk space actively, and set yourself a hard time limit. Don't let a paused schedule survive the incident.

Shrink jobs sit differently. A log backup at least buys you something, it advances your recovery position while it truncates. A scheduled shrink buys you nothing but disk space you weren't short of, which is why it should be off during an incident and, honestly, off the rest of the time too. The exception is the one case that isn't a schedule at all: the volume is actually filling and you're out of alternatives. That's a deliberate decision to trade evidence for staying online.

Step 5: Corroborate from outside the log

The log tells you what changed. It's poor at telling you what statement ran or who was connected. Several sources in SQL Server can help with that, and they're all fragile, by that I mean mean the default trace will roll over, ring buffers will wrap etc...

The default trace captures object created, altered, and dropped events, and it's running right now unless someone disabled it. This is the fastest path to "who dropped that table", Pinal Dave's walkthrough and the DallasDBAs version both read the trace files directly (SQL SERVER – Who Dropped Table or Database, see also DallasDBAs). Copy the .trc files out immediately; the default trace uses rollover files and will overwrite itself.

The system_health Extended Events session starts automatically with the Database Engine and runs with no noticeable overhead, collecting diagnostic data continuously. It's the diagnostic log almost nobody reads. Copy its files too.

Also worth grabbing while you're at it: the SQL Server error log files, output from any SQL Server Audit or third-party auditing you have running, application logs for the same window, and if you're lucky enough to have it , Query Store, which survives restarts and may show you the offending statement text.

Copy all of it to your forensics folder before you do anything else invasive. Files are cheap; a rolled-over trace is gone.

Step 6: Work out what actually happened

Now query your copied table. What you're looking for depends on the operation, and the operations don't look alike.

A DELETE logs one record per row. Delete ten rows and you get ten LOP_DELETE_ROWS records. Good news: row-level detail.

A TRUNCATE TABLE does not. It removes rows without logging the individual row deletions, it deallocates pages wholesale rather than deleting records one by one. It is still fully logged and still rolls back; there's no such thing as a non-logged operation in a user database. Veteran DBAs have been asking for a no-logging option for user databases for as long as there have been DBAs, and the answer has always been no because logging isn't a bookkeeping tax, it's the mechanism that makes rollback and recovery possible at all.

But you will not find your rows enumerated in the log. You'll find deallocation, which tells you that it happened and when, not what was in it. For a truncate, restore-based recovery is essentially your only path.

An UPDATE may log only the changed fragment rather than the full before-and-after row image. This is the single most common source of disappointment: "I can see the update, why can't I reconstruct the old value?"

To find who, pull the [Transaction SID] from the LOP_BEGIN_XACT record for the transaction and pass it to SUSER_SNAME() (See Paul Randal's SQLskills). Then join back on [Transaction ID] to see everything that transaction did. The LOP_BEGIN_XACT record is also where you get the transaction's start time and name which is what turns a pile of records into a timeline. The general pattern of filtering by Operation and AllocUnitName to isolate the damage is well documented.

What you want out of this step is three specific things: the LSN where the bad transaction began, the user, and the blast radius which objects, how many rows.

Step 7: Reading older evidence from backups

If the records aren't in the active log, they're in your log backups, and fn_dump_dblog reads those. It's the same idea as fn_dblog pointed at a file, with a long parameter list that's mostly NULL.

Two things to plan for. First, it's slow, and it can chew through entire backup files and dump its output into a table the same way, then query the table. Second, you may need to walk backwards through several backups to find the right window, which is exactly the tedium Paul Randal's walkthrough exists to spare you.

One consolation on timing: truncation only marks VLFs as reusable rather than erasing them (Transaction log Truncate vs Shrink vs VLF number), and a VLF can only be marked reusable when nothing still needs its records. So records sometimes survive longer in VLFs than the truncation point suggests, and trace flag 2536 can expose the inactive portion. Treat that as luck, never as a plan.

Step 8: Choose the shape of the recovery

You have your LSN. Now the decision that people get wrong under pressure: restore beside the database, not over it.

SQL Server has no native single-object restore, there is no RESTORE TABLE, and it's been a standing feature request for years. Third-party log readers do offer table-level recovery through a GUI, and it can be a genuine time-saver but note what they're actually doing. They read the log and generate compensating DML to undo the change; they are not restoring an object out of a backup, because the engine gives them no way to. That means they inherit every limitation in this article: they need the relevant log records to still exist, and they're subject to the same undocumented-format caveats. Without one of those tools, the standard move is a side-by-side restore… bring the backup up under a different database name, extract what you need, and copy the rows back into the live database while it keeps serving. The restore sequence is identical either way: most recent full, then the most recent differential based on it, then every log backup after that in order.

Restoring in place turns a data-loss incident into an outage, and if you get the stop point wrong you have to start over from the full backup. Side-by-side costs disk and buys you unlimited attempts.

For the stop point, if you have a clean timestamp, STOPAT is simplest:

RESTORE LOG YourDatabase_Copy
FROM DISK = 'E:\Backups\YourDatabase_log.trn'
WITH STOPAT = '2026-09-16 01:18:00', NORECOVERY;

If you need LSN precision and after Step 6 you have it, use the mark options. STOPATMARK = 'lsn:<lsn_number>' makes the record containing that LSN the recovery point and rolls forward through it; STOPBEFOREMARK stops immediately before it. For undoing a bad transaction you almost always want STOPBEFOREMARK against the LSN of its LOP_BEGIN_XACT. Watch the format conversion, the LSN in log output uses colon-delimited hex and needs converting for the restore syntax (Coeo).

Then reconcile. Everything that happened after your stop point is also absent from the restored copy, so copying rows back means thinking about which changes were legitimate. This is where a narrow blast radius from Step 6 pays for itself.

What the log will not give you

Set expectations early, with yourself and with whoever is asking for hourly updates.

  • Fragment-only updates. As above, an UPDATE may log only the changed portion, so clean before-image reconstruction is the exception rather than the rule.
  • TRUNCATE and DROP give you deallocation, not row contents.
  • Minimally logged operations. Under BULK_LOGGED, bulk imports, SELECT INTO, and similar are minimally logged, and log backups covering them can't support point-in-time recovery within that window. If a minimally logged operation ran since the last log backup and a data file is damaged and offline, a tail-of-the-log backup isn't possible at all.
  • TDE. Encrypted log content limits what any reader can hand back.
  • Schema drift. Decoding an old log record requires knowing the table's schema as it was then. If columns changed since, reconstruction gets shaky fast.
  • In-Memory OLTP. Memory-optimized tables merge multiple row changes into single log records and don't log index modifications at all. Row-level reconstruction assumptions don't hold.
  • PaaS. Azure SQL Database doesn't expose the transaction log at all. There, your answer is point-in-time restore and whatever auditing you enabled in advance.

And the standing caveat: everything here rests on undocumented functions that Microsoft can change or remove in any version, with no support recourse. Record the exact build you validated your scripts against, and re-verify after every upgrade.

If it's an Availability Group

A few extra moving parts.

Don't fail over during the investigation unless you have to. You'll change which replica's log you're reading and complicate your own timeline.

Run fn_dblog on the primary. Read-only secondaries will fight you, and the primary is the authoritative copy.

Check log_reuse_wait_desc first. AVAILABILITY_REPLICA means truncation is already blocked, which is working in your favor. Don't "fix" it mid-incident.

Remember log backups may be running on a secondary. If your backup job lives elsewhere in the AG, disabling the job on the primary accomplishes nothing. Find where it actually runs.

Replication and CDC change the picture too. Both hold log records until their readers catch up, so they may be preserving evidence for you and their own metadata tables are an independent record of what changed.

Quick version

If you only remember one thing, remember the order:

Freeze. Pause shrink and maintenance jobs unless you absolutely have to. Don't restart, detach, CHECKPOINT, fail over, or take a normal log backup.


Check the recovery model. SIMPLE means go straight to backups.

Copy the log out. SELECT * INTO OtherDB.dbo.LogDump FROM sys.fn_dblog(NULL, NULL);

Capture files without truncating. BACKUP LOG … WITH COPY_ONLY plus a copy-only full, plus the existing .trn chain, moved beyond reach of retention cleanup.

Grab the outside evidence. Default trace .trc files, system_health files, error logs, audit output. They roll over.

Then investigate the copies, unhurried: find the LOP_BEGIN_XACT LSN, the SID, the blast radius.

Restore side-by-side with STOPBEFOREMARK, never over the top.

Preservation, then investigation, then recovery. The whole discipline is refusing to do them out of order.

Agatha Christie's Hercule Poirot solved his cases with two things: order and method, and the little grey cells. He never once ran to the scene and started moving the furniture. Neither should you, the log is your crime scene, and every tempting quick fix is a footprint in the flowerbed.

Prepare for the Next Incident: Auditing, Recovery, and Runbooks

The reason this article has to exist is that the log is a lousy audit trail being pressed into service as one. Fix that. Once the immediate incident is resolved, address the gaps that made the investigation difficult. The goal is to have purpose-built records, tested recovery procedures, and a clear runbook ready next time, rather than having to reconstruct events from transaction-log internals.

  • SQL Server Audit for who-did-what, with a real retention story.
  • Temporal tables on anything where "what did this row look like before" is a question you'll ask more than once. This is the single highest-value change for accidental-update incidents.
  • Change Data Capture or change tracking where you need the changes themselves and these are documented, supported interfaces, which is the whole point.
  • Extended Events sessions for DDL, since the default trace is small and rolls over fast.
  • Verify your restore chain regularly, including a real side-by-side restore drill. Almost everything in Step 8 goes better if you've done it once when nothing was on fire.
  • Sort out the sysadmin question in advance. If reading the log requires sysadmin and your on-call DBA doesn't have it, you'll spend your best twenty minutes on an access request.
  • Write the runbook. Fill in your own paths, job names, instance names, and where the forensics folder lives. A checklist you wrote calmly is worth more than anything you'll reason out under pressure.

One last thing worth saying out loud: log forensics is the tool of last resort, and its best use is buying you information, not restoring your data. The restore is what restores your data. The log is what tells you where to stop.

Know when to bring in Microsoft Support

The log reading commands discussed here are undocumented and unsupported. Their usefulness does not make them a supported recovery procedure. Despite everything this article covers and links to, contact Microsoft Support early, especially if your backups cannot cover the potential data loss or cannot be restored. That advice applies even if you have decades of experience and have handled similar incidents before.

Use this information to understand the internals, preserve evidence, and organize your findings while Microsoft Support investigates. The goal is to save time and keep recovery options open, not to replace expert assistance or guarantee that lost data can be recovered.

Try it for yourself with this self-contained T-SQL demo. It creates a sample database, simulates data changes, and walks you through investigating the evidence using sys.fn_dblog. It includes setup instructions and optional cleanup. Run it only on a disposable, non-production SQL Server instance.

Further reading

Backup and restore mechanics

Reading the log

Corroborating evidence

Limits and gotchas


Tuesday, September 15, 2026

Why sys.fn_dblog Is Undocumented And Why It's Still There

Why sys.fn_dblog Is Undocumented And Why It's Still There Why sys.fn_dblog Is Still Undocumented — And Still There
This article is about why things are the way they are, not how to do it. There are no step-by-step examples here on purpose, the further reading at the end covers the hands-on side.

sys.fn_dblog is an undocumented SQL Server function that reads the active portion of the transaction log. Every DBA eventually ends up looking for it. Maybe you're chasing a mystery delete. Maybe you're just curious about internal structures, what SQL Server actually writes when you update a single column. Either way, someone on a forum suggests this line. Or these days, more likely, your favorite AI tool does:

SELECT * FROM sys.fn_dblog(NULL, NULL);

You run it. Out comes a wall of columns with names like Current LSN, Operation, Context, AllocUnitName, Page ID, Log Record Fixed Length. It feels like you just pried the lid off the engine. And then you go looking for the documentation, and there isn't any.

That's not an oversight. It's been that way for two decades, on purpose, and the reasons are interesting.

First, why DBAs keep wanting this

The desire to read the transaction log is almost universal among DBAs, and it comes from a handful of very human places:

  • Forensics. "Who deleted those 1734 rows at 1:19 PM, and can I prove it?" If auditing wasn't enabled and there's no trigger, the log is the only remaining witness.
  • Recovery without a full restore. Sometimes you need to reconstruct a handful of rows, not restore the entire database to a scratch server. If the lost change is still in the active log, you may be able to inspect the log records and manually reconstruct some deleted or updated rows. Keep your expectations in check, though this rarely works out as cleanly as it sounds.
  • Point-in-time precision. Finding the exact LSN just before the bad transaction so you can RESTORE ... WITH STOPBEFOREMARK at the right boundary.
  • Understanding replication, CDC, and Availability Groups. All of them are log readers under the hood. Watching the log makes their behavior stop feeling like magic.
  • Log growth mysteries. Something is holding the log hostage and log_reuse_wait_desc is only telling you what, not who.
  • And honestly: education. A huge slice of fn_dblog usage is pure curiosity. Seeing that a single-row update produces LOP_MODIFY_ROW against a specific slot on a specific page, and that a page split fans out into a whole cluster of log records, teaches you more about SQL Server in ten minutes than a week of reading. That's a legitimate reason to poke at it. Just not on production.

A short history

The original way in was DBCC LOG, and it dates back to the SQL Server 6.x/7.0/2000 era. The syntax was as terse as everything else in the DBCC family:

DBCC LOG (dbid | 'DBName', 3)

Example:




The second parameter controlled verbosity,  0 for the bare minimum (operation, context, transaction ID), rising through 1, 2, 3 up to 4 for the full dump, as documented across community write-ups of the command. Later variants accepted extra arguments to filter by LSN, transaction ID, page ID, object ID, or record count (DBCC command reference list).

It is useful but awkward. It returned a fixed result set you couldn't join to, filter properly, or aggregate. 

Paul Randal, who worked on the storage engine team, explained why so much of this lives under DBCC in the first place: adding a DBCC command is far easier than building a proper, supported T-SQL surface, so DBCC became the natural home for "reach in and touch an internal data structure" features built for the dev team's own use. That's the key insight: these things were never designed as user features. We're borrowing the engineers' tools!

Then SQL Server 2005 arrived with table-valued functions, and the internal log dump got a much nicer wrapper: sys.fn_dblog. Same idea, but now it's a relational rowset. You can WHERE, JOIN, GROUP BY, and dump it into a temp table.

A small family grew up around it, each solving a different limitation:

  • sys.fn_dblog(start_lsn, end_lsn):  Reads only the active  portion of the online log for the current database, with optional LSN bounds (NULL, NULL for everything available).
  • sys.fn_dump_dblog:Reads log backups and detached .ldf files, which is what you need once the records you want have already been truncated out of the live log. It's slower, takes a long list of mostly-NULL parameters, and is the tool Paul Randal walks through for locating a dropped object's LSN and then restoring with STOPBEFOREMARK (SQLskills).
  • sys.fn_full_dblog: First arrived in the SQL Server 2017 timeframe as a more capable alternative: eight parameters instead of two, adding database ID, page targeting, and backup account/container, which lets you query across databases with a CROSS APPLY against sys.databases. It returns the same ~130 columns. And it is also undocumented, nobody publicly documents what those backup parameters actually do.
  • Trace flag 2536 is the classic companion, used to make the inactive portion of the log visible too.

Notice what never arrived, for any of them: a documentation page.

Why: The log's on-disk format is an implementation detail

Microsoft's public documentation describes the transaction log logically, a serial stream of log records, each stamped with an ever-increasing LSN, physically divided into virtual log files (VLFs). Community internals work adds the next layer: a three-level hierarchy of VLFs containing log blocks containing the actual log records.

What's documented is the architecture. What's never documented is the byte layout, the log block header fields, the log record header, the per-operation payload encoding, how a LOP_MODIFY_ROW packs its before/after fragments, how the VLF header stores parity and sequence numbers.

And that's exactly the stuff that shifts between major versions. Every release brings storage-engine work that touches the log: new operation types for new features, changes to what gets logged and how, compression and encryption of what's on disk, and adjustments driven by the AG and CDC log readers. The log format is one of the least frozen structures in the SQL Server, because the log is where nearly every new engine feature has to leave its footprint.

fn_dblog isn't a translation layer that hides all this. It's a thin projection of internal structures. So when the internals move, its output moves with it:

  • The column list changes. It's roughly 116 columns on SQL Server 2008 R2, 129 on later builds, and 130+ depending on version.
  • Operation and context values evolve as features are added.
  • Payload semantics vary. An UPDATE doesn't necessarily record the whole before-and-after row; it can record just the changed fragment,  which is why "reconstructing the old rows" is harder than it looks.

Documenting fn_dblog would mean committing to a contract Microsoft has no intention of freezing. The moment it's documented, it's supported; the moment it's supported, the storage engine team loses the freedom to reshape the log. Given the choice between "publish a stable log format" and "keep improving the log," the engine team picks the second one every time. Community consensus on the DBA side says the same thing plainly: it's undocumented, unsupported, can change or disappear at any version, and you can't get an official answer about what the columns mean. Microsoft's own forum guidance is blunt: the log exists for internal use, and reading it directly is not officially supported.

The transaction log file is not intended for direct reading by users but for internal use. What you are asking is a level 500 actions (internals) and it is not officially supported.  Microsoft Q&A.

If you want another indicator that this is policy rather than neglect, look at sys.fn_full_dblog. In 2017 Microsoft shipped a newer, more capable log reader, more parameters, cross-database reach and documented exactly as much of it as its predecessor: nothing. Twelve years after fn_dblog appeared, with a clean opportunity to draw the line somewhere else, the answer was the same. The log's contents are not a public interface, and adding better internal tools doesn't change that.

So why not remove it? Because Microsoft still needs it. Support engineers, the product group, and internal recovery scenarios all rely on being able to dump the log. It stays because it's useful to them,  we're just allowed to look. That's the whole bargain: available, never promised.

Feature by feature, the log format kept evolving

If you want concrete evidence that the log format isn't a fixed target, look at what's been bolted into it over the last decade. Each of these changed what gets written, or what a log reader sees:

  • In-Memory OLTP (2014). Memory-optimized tables share the same log file but log very differently: no write-ahead logging in the traditional sense, multiple row changes merged into a single log record, and no log records at all for index modifications since indexes are rebuilt at recovery (sqlserver-help.com). A tool that assumes one log record per row modification is already wrong.
  • Accelerated Database Recovery (2019). ADR versions physical modifications into a Persistent Version Store and only undoes non-versioned operations, which lets recovery skip the traditional undo phase, and because the PVS itself must be recoverable, all operations against it are logged, increasing log volume, New record types, new semantics, same function signature.
  • Columnstore, TDE, log compression for AGs, minimally logged bulk operations each one either adds record shapes or removes information from the log entirely.

None of these arrived with a "here's what changed in the log format".

What Microsoft does document and why the difference matters

Here's the tell that this isn't laziness. Microsoft has been steadily adding documented, supported views over the log, they just stop at metadata and aggregates, never record contents:

  • sys.dm_db_log_info (SQL Server 2016 SP2+) returns VLF-level information, the supported replacement for DBCC LOGINFO.
  • sys.dm_db_log_stats returns summary-level log health attributes including log_backup_time, which is genuinely useful on AG secondaries and needs only VIEW DATABASE STATE rather than sysadmin.
  • log_reuse_wait_desc in sys.databases tells you what's preventing truncation.

Microsoft's intention is clear: how much log, how many VLFs, what's blocking reuse, and when it was last backed up are all fair game and will be kept stable. What the individual records say is not, and never will be.

Meanwhile, Microsoft does ship fully supported ways to consume log content, they just don't let you read it raw. The Replication Log Reader Agent monitors the log and moves marked transactions into the distribution database , and CDC, change tracking, and replication are all supported on Always On Availability Groups . Those are the sanctioned log readers. fn_dblog is the unsanctioned one.

Restrictions and Limitations

Things that bite people, and that no documentation page will warn you about:

  • It's sysadmin-gated. Querying it fails with "User does not have permission to query the virtual table, DBLog" (Msg 9010) for anyone who isn't sysadmin; plain GRANT SELECT on the function isn't enough (DBA Stack Exchange). Certificate-signed module signing is the usual workaround when a tool genuinely needs it which is exactly why ETL vendors document db_owner plus SELECT on master.sys.fn_dblog as an alternative to full sysadmin. Sysadmin-for-forensics is a real governance conversation.

  • You're racing truncation.  fn_dblog  only sees the active portion. In SIMPLE recovery a CHECKPOINT clears it; in FULL a log backup does. Once the records are gone, only fn_dump_dblog against backups can help which is why a huge .ldf can still return almost nothing.  Note: Preserving log evidence during an actual incident has enough moving parts to deserve its own post, I'll cover it separately someday.

  • Volume and cost. On a busy database the active log can hold millions of records. Filter on LSN ranges, Operation, and AllocUnitName; don't SELECT * and hope.
  • fn_dump_dblog is not free. It's markedly slower than fn_dblog, and it's well known in the field for holding onto resources within the session, so treat it as something you run deliberately on a scratch instance, not casually on a production box.
  • TDE and encryption cut you off. Encrypted log content limits what any log reader, Microsoft's or a vendor's, can hand back.
  • PaaS closes the door. Azure SQL Database does not expose the transaction log, and log access is restricted on managed platforms generally, which is why log-based CDC tools fall back to other capture methods there. As workloads move to PaaS, the log-reading skill gets less portable, not more.
  • Names are not stable either. Even the "friendly" columns aren't a contract. Reading AllocUnitName and joining out to system metadata works, until an internal name format shifts.

The third-party angle

If Microsoft won't decode the log for you, vendors will try. There's a long lineage of commercial log readers: ApexSQL Log (later a Quest product), Lumigent Log Explorer, Red Gate's SQL Log Rescue, and others offering graphical row-level audit trails and undo/redo script generation from online logs and log backups (Quest/ApexSQL).

How do they do it? Broadly, two approaches, usually combined:

  1. Read the .ldf and backup files directly and parse the binary structures themselves. ApexSQL Log, for instance, doesn't install anything on the SQL Server engine; it installs a Windows service to enable remote reading of the online log files and analyzes native or compressed log and backup content (ApexSQL FAQ).
  1. Lean on the same undocumented surfaces we have  fn_dblog and fn_dump_dblog then enrich the raw records by joining against system metadata to turn page IDs, slot IDs, and allocation unit names back into recognizable tables, columns, and values.

Both paths hit the same wall: the format is proprietary and undocumented, so everything is reverse-engineered. That has consequences worth knowing before you buy:

  • Version lag. Every new major release means re-reverse-engineering. Support ships months late, or not at all.
  • Partial reconstruction. Because the log records deltas rather than full row images in many cases, and because some operations are minimally logged, a complete audit trail isn't always achievable. The tools do impressively well, then hit gaps they can't fill.
  • Feature blind spots. In-Memory OLTP's merged log records, ADR's version-store records, columnstore, and encryption each degrade what a reverse-engineered parser can reconstruct.
  • No schema time machine. Decoding an old log record requires knowing the table's schema as it was then. If columns were added or dropped since, reconstruction gets shaky fast which is a limitation shared by every tool in the category.

And the commercial risk is real. ApexSQL Log hasn't added SQL Server 2022 support, and the product line has been headed for discontinuation (r/SQLServer discussion). Note where the surviving change-capture ecosystem went instead: modern pipelines like Debezium For SQL Server and most cloud connectors consume CDC or change tracking, the documented interfaces, rather than parsing .ldf bytes.

Practical Guidance

If you want to use fn_dblog, use it the way it deserves to be used:

  • Play on a scratch instance, not production. Restore a copy and investigate there whenever you can.

  • Treat it as a lens, not a source of truth. Never build a report, application, or automated job on its column list. It will break on your next upgrade, with no support recourse.

  • Preserve evidence first. Before you investigate a suspected data loss: take a log backup to a safe location, then pause routine log backups and log-shrink jobs so the active log stops rolling over.
Run: SELECT * INTO OtherDB.dbo.LogDump FROM sys.fn_dblog(NULL, NULL);
  • For real auditing, use documented features. SQL Server Audit, Extended Events, temporal tables, or CDC/change tracking. They exist precisely so you don't have to read the log.
  • For real recovery, use backups. Log backups plus STOPAT / STOPBEFOREMARK is the supported path. fn_dblog is great for finding the LSN to stop before; it's a poor substitute for the restore itself.
  • Write down your version. If you keep a runbook that queries the log, record the exact build it was validated on, and re-verify after every upgrade. That single habit turns an unsupported query from a liability into a managed risk.
  • Learn from it freely. Run an insert, an update, a delete, a page split, a rollback, and watch what appears. Best mental model of logging you'll ever build, and it costs nothing on a test database.

Conclusion

sys.fn_dblog sits in a peculiar, permanent middle ground: too useful for Microsoft to remove, too volatile for Microsoft to document. It survives because the on-disk log format is an implementation detail that changes with major versions, log block headers, record headers, operation encodings, feature-driven additions from In-Memory OLTP to ADR and publishing a stable interface over it would freeze a structure the engine team needs to keep changing.

DBCC LOG was the first crack in that wall. fn_dblog made the view a lot clearer. Third-party tools spent twenty years reverse-engineering the rest with real skill and real limits. And Microsoft's answer, consistently, has been to document the log's shape while keeping its contents private, and to hand you CDC and replication when you need the contents for real.

So go look inside the log. Just don't build anything load-bearing on the view.

Further reading: 

The deep dives (from Paul Randal)


Reading and interpreting the output


Official documentation worth reading alongside


Permissions and gotchas

Thursday, September 10, 2026

How to Check TDE Progress and Elapsed Time in SQL Server

How to Check TDE Progress and Elapsed Time in SQL Server
Transparent Data Encryption (TDE) protects SQL Server data and log files while they are stored on disk. SQL Server handles encryption and decryption automatically, allowing applications and users to continue working normally. Database backups are also encrypted and cannot be restored without the correct certificate or key.

What does TDE protect against?


TDE protects data while it is at rest in database files, transaction log files, backups, disks, and storage snapshots. If someone steals or copies those files, they cannot simply restore or attach the database and read its contents without the correct certificate or key.

When SQL Server reads an encrypted page from disk into the buffer cache, it automatically decrypts the page so the database engine can use it. This means TDE does not protect data already in memory, nor does it prevent SQL injection, compromised administrator accounts, misuse by authorized users, unencrypted exports, or interception of data travelling across the network. 

Who should use TDE?


If your database contains sensitive or personal information, TDE is worth considering. It is especially useful when encryption at rest is required by standards such as HIPAA or PCI DSS, or by your organization’s security policies.

However, don’t enable TDE everywhere without a plan. It introduces some performance overhead and requires careful management of encryption keys and certificates. Use it where the security and data-protection requirements justify the extra work.

Why can a TDE scan take so long?


TDE encryption and decryption scans can take hours, even on a moderately sized SQL Server database.

The scan runs in the background and must read and rewrite every database page. Its speed depends heavily on storage performance, and SQL Server provides no supported MAXDOP, priority, or speed setting to make it finish faster. TDE may be an online operation, but it is not always a “start it and forget it” job. On a large database, the scan can keep storage busy for hours. 

Does Enterprise Edition make TDE faster?


Not directly and usually not by much. TDE works the same way in SQL Server Standard and Enterprise editions. Enterprise can use greater CPU, memory, and high-availability capabilities, but it does not provide a different form of TDE or a special setting that makes the scan faster.

Why should you monitor TDE progress?


The initial TDE scan, which encrypts the existing data pages at rest, can take much longer than expected. Even a modest 300–400 GB database may take hours when server activity is high or storage is struggling to keep up. Watching the percentage creep forward can feel endless, especially during your first TDE implementation.

The wait becomes more stressful when the next step in your change plan depends on the scan finishing. For example, you may be waiting to take the final encrypted backup, add the database to an Availability Group, or run post-implementation scripts and validation checks. A delay can affect both your schedule and your blood pressure!


Naturally, you want answers: How much is complete? How long has it been running? Is it making steady progress? Is the scan running normally, suspended, or aborted? Without that information, your carefully planned change window can quickly turn into a guessing game.

How can you check TDE progress and elapsed time?


The following practical query helps answer those questions. By viewing the percentage complete alongside the elapsed time, you can get a useful sense of its average pace and whether it is moving normally. It is not a precise finish-time prediction because workloads and storage activity can change, but it helps you see whether the scan is making steady progress or may need attention.


/*
    PURPOSE
    -------
    Shows the progress, status, and duration of current TDE encryption,
    decryption, key-change, and protection-change operations.

Why this query is different
    ---------------------------
Most TDE monitoring queries show only the current status and
percentage complete from sys.dm_database_encryption_keys. This
query also matches the DMV results with the most recent
scan-start entry in the SQL Server error log, allowing it to show
the start time and elapsed duration without requiring a monitoring
table, SQL Agent job, or external tool. HOW IT WORKS ------------ 1. Reads TDE scan-start messages from the current SQL Server
error log
and stores them in a temporary table. 2. Queries sys.dm_database_encryption_keys for databases with an
active,
suspended, or aborted TDE operation. 3. Matches each database to its most recent "Beginning database encryption scan" error-log entry. 4. Calculates the duration between that entry and the current
server
time. Duration is displayed in seconds, minutes, and
HH:MM:SS format.
IMPORTANT LIMITATIONS --------------------- - Only the current SQL Server error log is searched. - If the error log rolled over after the scan started, the start
time
and duration will be NULL. - If a scan was suspended and resumed, duration begins with the
most
recent scan-start entry. It does not include time from
earlier runs.
- encryption_scan_modify_date is reported in UTC, while the
error-log
and collection times normally use the SQL Server
host's local time.
REQUIREMENTS ------------ SQL Server 2019 or later because the encryption scan state
columns were
introduced with SQL Server 2019. */ USE master; GO SET NOCOUNT ON; DROP TABLE IF EXISTS #TdeErrorLog; CREATE TABLE #TdeErrorLog ( LogDate datetime, ProcessInfo nvarchar(50), [Text] nvarchar(max) ); INSERT INTO #TdeErrorLog EXEC sys.xp_readerrorlog 0, -- Current error log 1, -- SQL Server error log N'Beginning database encryption scan', NULL, NULL, NULL, N'desc'; SELECT d.name AS database_name, dek.encryption_state, dek.encryption_state_desc, dek.percent_complete, dek.encryption_scan_state, dek.encryption_scan_state_desc, scan_start.operation_start_time, GETDATE() AS collection_time, duration.duration_seconds, CAST(duration.duration_seconds / 60.0 AS decimal(18,2)) AS duration_minutes, CASE WHEN duration.duration_seconds IS NULL THEN NULL ELSE CONCAT ( duration.duration_seconds / 86400, N'd ', RIGHT(N'00' + CONVERT(nvarchar(2), (duration.duration_seconds % 86400) / 3600), 2), N':', RIGHT(N'00' + CONVERT(nvarchar(2), (duration.duration_seconds % 3600) / 60), 2), N':', RIGHT(N'00' + CONVERT(nvarchar(2), duration.duration_seconds % 60), 2) ) END AS formatted_duration, -- This DMV value is documented as UTC dek.encryption_scan_modify_date AS scan_state_modified_utc, dek.key_algorithm, dek.key_length, dek.encryptor_type FROM sys.dm_database_encryption_keys AS dek INNER JOIN sys.databases AS d ON d.database_id = dek.database_id OUTER APPLY ( SELECT TOP (1) el.LogDate AS operation_start_time FROM #TdeErrorLog AS el WHERE CHARINDEX ( N'''' + d.name + N'''', el.[Text] ) > 0 ORDER BY el.LogDate DESC ) AS scan_start OUTER APPLY ( SELECT CASE WHEN scan_start.operation_start_time IS NOT NULL THEN DATEDIFF_BIG ( SECOND, scan_start.operation_start_time, GETDATE() ) END AS duration_seconds ) AS duration -- Only active, suspended, or aborted TDE operations WHERE dek.encryption_state IN ( 2, -- Encryption in progress 4, -- Key change in progress 5, -- Decryption in progress 6 -- Protection change in progress ) ORDER BY d.name; DROP TABLE IF EXISTS #TdeErrorLog; GO


What happens during a TDE scan?


Now that we can monitor the scan, it helps to understand what SQL Server is doing behind the scenes.

Even though it may appear so, TDE encryption and decryption scans are not necessarily single-threaded. When you enable TDE, SQL Server performs a few checks and starts a background encryption worker. This allows the original command to finish while the real work continues in the background.

The encryption worker creates disk workers to scan the database files. Each worker processes 8 KB database pages in batches of 32. The pages are loaded into memory, marked as changed, and logged so the operation can also be replayed on an Availability Group secondary.

After the scan finishes, SQL Server performs a checkpoint. During encryption, the changed pages are encrypted and written back to disk. Decryption and encryption-key rotation use the same basic scanning process, although the pages are handled according to the requested operation.

SQL Server generally creates one disk worker per storage volume, not one worker per processor. Ten data files on one volume may still use only one worker, while files spread across ten volumes could use ten workers. This helps control the impact on storage, but it also explains why some TDE scans appear almost single-threaded.

TDE scan parallelism is managed internally by SQL Server and is not controlled by MAXDOP. Even when multiple workers are active, the scan can still be limited by storage performance.

Because every page must be read, processed, and written back, a TDE scan consumes storage, memory, and CPU resources. It may therefore compete with the database’s normal workload.

A 2019 Microsoft article about TDE scan internals describes SQL Server creating approximately one disk worker per storage volume. The core scan process remains relevant to SQL Server 2025, but worker allocation and batching are internal implementation details that could change in future versions or cumulative updates. 

Can you make a TDE scan run faster?


TDE scans depend heavily on storage speed. If several databases on the same storage volume are being processed at once, their scans will compete for I/O and may all take longer.

Adding more CPU or changing MAXDOP is unlikely to help. Adding extra data files on the same storage volume will not necessarily make the scan more parallel either.

What should you avoid while the scan is running?


While a scan is running, try to reduce other storage-heavy work such as backups, CHECKDB, index maintenance, and ETL processes. Avoid moving database files or changing the storage layout because some file operations are restricted during a TDE scan.

Should you redesign your storage for TDE?


That depends. For future large databases, placing files on separate physical storage volumes may allow SQL Server to use additional workers. However, this only helps when the volumes provide genuinely independent throughput, rather than separate drive letters backed by the same storage pool. In the year 2026, with shared SAN and cloud storage now common and IT shifting toward cloud and AI platforms, a dedicated physical SQL Server with its own direct-attached storage is typically reserved for rare or specialized requirements.

That said, there is no need to redesign every database just for an occasional TDE scan. A more practical approach is to provide sufficient storage performance and run only one major TDE scan per shared volume at a time.

What Triggers a Full TDE Scan?


It helps to know which operations trigger a full TDE scan, so you know when to keep an eye on its progress. A full scan occurs when SQL Server must encrypt, decrypt, or re-encrypt every page in the database. Three main operations cause this:

  • Enabling TDE: SQL Server reads every database page, encrypts it, and writes it back to storage.
  • Disabling TDE: SQL Server performs the journey in reverse, reading and rewriting every page without TDE encryption.
  • Regenerating the database encryption key (DEK): SQL Server creates a new DEK and re-encrypts every page with it. This includes changing the DEK’s encryption algorithm 

There is an important difference between regenerating the DEK and changing the certificate that protects it. During the initial TDE setup, the certificate is created and stored in the instance’s master database before the DEK is created in the user database. Changing this certificate or asymmetric key normally re-encrypts only the DEK, not every database page. This is much less work and does not require a full database scan.

Pausing, resuming, or restarting SQL Server during a scan does not start a brand-new scan. SQL Server saves the progress and continues the existing operation when it resumes 



See also