Search This Blog

Thursday, May 16, 2019

Find identical duplicate indexes

Unlike some other RDBMSs, SQL Server does not stop you from creating duplicate indexes. Often, the developer or DBA adding a new index doesn’t check whether an index with the same key columns and order already exists. This situation is more common than many realize.

Duplicate indexes are not only unnecessary, they are bad. They don’t make any query faster, but they still have to be maintained. Every INSERT, UPDATE, and DELETE has to update all copies of the index, that increases write latency and transaction log usage. They also consume extra disk space and memory, and they lengthen index maintenance and backup operations without providing any additional benefit.

When I started looking for identical duplicate nonclustered indexes, same table, same columns, same order, I wanted something flexible enough to handle a few different scenarios. That’s what this script is for: it helps you spot indexes that are essentially copies of each other so you can decide whether to keep or drop them.

There are two knobs you can tweak:

  • If you want to treat indexes as duplicates even when the key columns are in a different order, set @disregard_column_order = 1. Leave it at 0 if you only care about truly identical definitions, including column order.

  • By default, the script ignores cases where one index is clustered and the other is nonclustered. If you also want to see those pairs, set @include_clustered_indexes = 1.


-- See blog post: https://sqlpal.blogspot.com/2019/05/find-identical-duplicate-indexes-revised.html
SET NOCOUNT ON;
USE [Your Database Name;
/*==========================================================================================
  Script Name : Find_Identical_Duplicate_Indexes.sql
  Purpose:
      Find identical duplicate indexes on the same table (same key columns, same order
      unless configured otherwise), so you can evaluate and potentially drop redundant ones.
  High-level approach:
      - Build a comma-separated list of key columns for every index in the database.
      - Optionally:
          * Include clustered vs nonclustered index pairs.
          * Ignore column order when comparing indexes.
      - Group by (schema, object, index_columns) and return those with COUNT(*) > 1.

  Variables/Parameters:
      @include_clustered_indexes (bit)
          0 = Ignore duplicates where one index is clustered and the other is nonclustered
          1 = Include those cases as duplicates too.

      @disregard_column_order (bit)
          0 = Only consider indexes duplicates if key columns match AND are in the same order.
          1 = Consider indexes duplicates even when the same key columns are in a different order.

  Notes / Caveats:
      - This script does NOT:
          * Consider ASC/DESC sort order differences.
          * Consider included columns.
          * Distinguish filtered vs non-filtered indexes beyond exposing filter_definition.
      - Do NOT blindly drop indexes; review each case in the context of workload and plans.
==========================================================================================*/


/*
    Whether to include identical indexes where one is clustered and 
    the other one is non-clustered
*/
DECLARE @include_clustered_indexes bit = 0;


/*
    Whether to find duplicate indexes where all key columns are the same,
    but not necessarily in the same order. Typical usage:
      - 0 (default): require same column order (more strict, closer to “identical”).
      - 1: ignore key column order, treat any permutation of the same columns as duplicates.
*/
DECLARE @disregard_column_order bit = 0;


;WITH cte AS
(
    SELECT
        o.schema_id,
        o.type_desc,
        o.object_id,
        i.index_id,
        i.name AS index_name,

        /*
            Build a comma-separated list of key column names for this index.
              - We need a stable string representation of the key columns to compare indexes.
              - Ordering of columns in the list is controlled by @disregard_column_order:
                    @disregard_column_order = 0 → order by key_ordinal (true index order)
                    @disregard_column_order = 1 → order by column_id (logical column order)
        */
        index_columns =
            COALESCE(
                STUFF
                (
                    (
                        SELECT
                            CAST(',' + COL_NAME(object_id, column_id) AS varchar(max))
                        FROM sys.index_columns
                        WHERE object_id = i.object_id
                          AND index_id  = i.index_id
                          AND is_included_column = 0          -- Only key columns, not INCLUDE
                        ORDER BY 
                            object_id, 
                            index_id,
                            CASE 
                                WHEN @disregard_column_order = 1 
                                    THEN column_id           -- Ignore index key order
                                ELSE key_ordinal            -- Respect index key order
                            END
                        FOR XML PATH(''), TYPE
                    ).value('.', 'varchar(max)')
                    , 1, 1, ''
                )
            , ''),

        -- Index metadata for review and decision making
        i.type_desc       AS index_type,
        i.is_unique,
        i.data_space_id,
        i.ignore_dup_key,
        i.is_primary_key,
        i.is_unique_constraint,
        i.fill_factor,
        i.is_padded,
        i.is_disabled,
        i.is_hypothetical,
        i.allow_row_locks,
        i.allow_page_locks,
        i.has_filter,
        i.filter_definition

    FROM sys.indexes AS i
    INNER JOIN sys.objects AS o 
        ON o.object_id = i.object_id
    WHERE 
        OBJECTPROPERTY(o.object_id, 'IsMsShipped') = 0   -- Skip system objects
        AND i.index_id <> 0                              -- Skip the heap "index"
        /*
            Control whether clustered indexes participate:

            - If @include_clustered_indexes = 0:
                  i.index_id > 1 → only nonclustered indexes.
            - If @include_clustered_indexes = 1:
                  i.index_id > 0 → clustered + nonclustered.
        */
        AND i.index_id > CASE WHEN @include_clustered_indexes = 1 THEN 0 ELSE 1 END
)

-- Find indexes with identical index_columns on the same object and same type
SELECT
    SCHEMA_NAME(i1.schema_id) AS schema_name,
    i1.type_desc,
    OBJECT_NAME(i1.object_id) AS object_name,
    i1.index_name,
    i1.*  -- Includes index_columns + metadata for review
FROM cte AS i1
INNER JOIN 
(
    /*
        Identify combinations of (schema_id, type_desc, object_id, index_columns)
        that occur more than once → those represent duplicate index definitions.
    */
    SELECT 
        schema_id, 
        type_desc, 
        object_id, 
        index_columns
    FROM cte
    GROUP BY 
        schema_id, 
        type_desc, 
        object_id, 
        index_columns
    HAVING COUNT(*) > 1
) AS i2
    ON  i1.schema_id     = i2.schema_id
    AND i1.type_desc     = i2.type_desc
    AND i1.object_id     = i2.object_id
    AND i1.index_columns = i2.index_columns
ORDER BY 
    schema_name, 
    i1.type_desc, 
    object_name, 
    i1.index_name;



This query does not take the ASC or DESC sort direction into account. You might have two indexes with the same key columns where one is ASC and the other is DESC, and there can be perfectly valid reasons for that. It also doesn’t distinguish whether one or both indexes are filtered. I’d love to hear your feedback before I invest more effort covering every edge case. 

Download the SQL script from GitHub:




Wednesday, May 1, 2019

Capture Trend in Dead Lock Occurances

Capture Trend in Dead Lock Occurances
Deadlocks are usually detected and resolved automatically by SQL Server by killing one of the SPIDs involved in it.  You may have an alert setup to notify you and/or users may approach you with the issue.

But this post is not about finding the root cause and fix the issue for good. 

Ideally, you don't want to see dead locks occur at all and especially not on a regular basis.  But situations could arise where a database starts experiencing deadlocks. In this post I would like to share a code I have used to capture the daily count of deadlocks for a trending report.

Note: I am using the DMV sys.dm_os_performance_counters in this code so it will not work if the SQL Server performance counters are disabled for some reason.

You can use this query to check if the performance counters are enabled or disabled.

SELECT COUNT(*) FROM sys.dm_os_performance_counters;  

If the return value is 0 rows, more likely than not, it indicates that 
the performance counters have been disabled.



Step 1: Create empty table to store the deadlock counts


USE [AdminDBA]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [dbo].[deadlock_counter](
       [ID] [int] IDENTITY(1,1) NOT NULL,
       [record_created_date] [datetime] NOT NULL,
       [SQLStartedOn] [datetime] NOT NULL,
       [object_name] [nchar](128) NULL,
       [counter_name] [nchar](128) NULL,
       [instance_name] [nchar](128) NULL,
       [cntr_value] [bigint] NULL,
       [cntr_type] [int] NULL,
       [AveragePerDay] [bigint] NULL,
PRIMARY KEY CLUSTERED
(
       [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

 Step 2: Create a SQL Server Agent Job


/*

Please update email operator value for @notify_email_operator_name parameter

Please feel free to update any of the settings including the schedule you would like to use.

*/


USE [msdb]
GO
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

END

DECLARE @jobId BINARY(16)
EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'DBA - Deadlock Counter',
             @enabled=1,
             @notify_level_eventlog=0,
             @notify_level_email=2,
             @notify_level_netsend=0,
             @notify_level_page=0,
             @delete_level=0,
             @description=N'Populate deadlock performance counter values',
             @category_name=N'[Uncategorized (Local)]',
             @owner_login_name=N'sa',
             @notify_email_operator_name=N'DBA', @job_id = @jobId OUTPUT

IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'deadlock performance counter values',
             @step_id=1,
             @cmdexec_success_code=0,
             @on_success_action=1,
             @on_success_step_id=0,
             @on_fail_action=2,
             @on_fail_step_id=0,
             @retry_attempts=0,
             @retry_interval=0,
             @os_run_priority=0, @subsystem=N'TSQL',
             @command=N'set nocount on
set transaction isolation level read uncommitted
go
use AdminDBA
go
-- the counter values are cumulative since SQL Server service was started, not per second
insert into deadlock_counter
SELECT getdate() record_created_date,
       d.create_date SQLStartedOn, p.*, AveragePerDay = CONVERT(BIGINT, (( 1.0 * p.cntr_value / NULLIF(Datediff(dd, d.create_date,CURRENT_TIMESTAMP), 0 ))))
-- INTO deadlock_counter
FROM   sys.dm_os_performance_counters p
       INNER JOIN sys.databases d ON d.NAME = ''tempdb''
WHERE  Rtrim(p.counter_name) = ''Number of Deadlocks/sec''   
--       AND cntr_value > 0
      AND Rtrim(p.instance_name) = ''_Total''
ORDER  BY cntr_value DESC

-- select * from deadlock_counter',
             @database_name=N'master',
             @flags=0
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'Daily',
             @enabled=1,
             @freq_type=4,
             @freq_interval=1,
             @freq_subday_type=1,
             @freq_subday_interval=0,
             @freq_relative_interval=0,
             @freq_recurrence_factor=0,
             @active_start_date=20190228,
             @active_end_date=99991231,
             @active_start_time=80000,
             @active_end_time=235959
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
    IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:
GO


Step 3: Query to extract deadlock counts for each day/interval


-- Note: the counter values are cumulative since SQL Server service was started, not per second

set nocount on
set transaction isolation level read uncommitted
select * ,cntr_value - coalesce(lag(cntr_value) over (order by id), cntr_value) new_deadlocks
from AdminDBA..deadlock_counter
order by ID desc