As you can see, there are two categories of extended stored procedures you can use, with complimentary procedures between them. The “Instance-Aware” category procedures utilize key word substitution in order to navigate to the appropriate registry keys for the SQL Server instance that is running. I’ll show you an example shortly.

A lot of companies spin their wheels trying to diagnose a SQL problem.

With SQL Server, we’ve seen it all and have done it all. Save time. Call us first.

Learn More

Registry in SQL Server Basics

Before we go too much further, let’s explain a few basic key phrases when dealing with the registry:

Registry: The registry is a hierarchical database used by Windows. It is organized in a tree view manner.

Registry Hive: A hive is a logical group of keys, subkeys, and values in the registry that has a set of supporting files containing backups of its data.

Examples of Registry Hives are:

Of all of these hives, you will probably use HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER the most.

Registry Key: A node within a registry hive is called a key. A key can contain both sub-keys and values. Sometimes an application will only be looking for the presence of a key; at other times it will need to look at the value.

Registry Value: A registry value is simply a piece of data, of any data type.

The following image shows the tree view layout of the registry, and identifies the various items within it:

Querying the registry from SQL Server

Using xp_regread / xp_instance_regread

Let’s start off by querying some data within SQL Server. I have an instance of SQL Server named “SQL2014” (would you care to take a guess as to what version of SQL Server this is?). One of the items stored in the registry is the location of the SQL Agent working directory. These procedures can query the registry and return the specified values. For example:

Source code

EXECUTE master.sys.xp_regread 
    'HKEY_LOCAL_MACHINE', 
    'Software\Microsoft\Microsoft SQL Server\MSSQL12.SQL2014\SQLServerAgent', 
    'WorkingDirectory';

When I execute this statement, SQL returns the following result:

Source code

VALUE            DATA
---------------- -----------------------------------
WorkingDirectory D:\MSSQL\MSSQL12.SQL2014\MSSQL\JOBS

In order for you to run this, you may need to change the key as appropriate for the version and instance on your server. The result includes both the value, and the data for the specified path.

xp_instance_regread

In this example, I used xp_regread to read the direct registry path. If you remember from earlier, there are SQL Server instance-aware versions of each registry procedure. A comparable statement using the instance-aware procedure would be:

Source code

EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent',
    'WorkingDirectory';

This statement returns the exact same information. Let’s look at the difference between these – in the first query, the registry path is the exact registry path needed. It includes “\Microsoft SQL Server\MSSQL12.SQL2014\”. In the latter query, this string is replaced with “\MSSQLSERVER\”. Since the latter function is instance aware, it replaces the “MSSQLSERVER” with the exact registry path necessary for this instance of SQL Server. Pretty neat, isn’t it? This allows you to have a script that will run properly regardless of instance. The rest of the examples in this post utilize the instance-aware procedures to make it easier for you to follow along and run these yourself.

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regread 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’
    [, [@value_name=]’value_name’]
    [, [@VALUE=]@VALUE OUTPUT]

The first parameter is the registry hive that you want to query, the second parameter is the key path, and the third is the value name. The third parameter is optional – if provided, then the procedure will return the data from the specified value item; if not provided, then the procedure only returns whether the specified key exists. There is also an optional fourth parameter, which is an output parameter, and the data output goes into that. The parameters are positional, and while you can specify a name for the parameter, any name will work. Since the parameters are positional, my recommendation is to not use the parameter names. The following example utilizes the optional fourth parameter, and it will return just the specified path into the variable:

Source code

DECLARE @SQLAgentDirectory VARCHAR(255);
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent',
    'WorkingDirectory',
    @SQLAgentDirectory OUTPUT;
SELECT @SQLAgentDirectory;

You would think that if you specify either NULL or an empty string for the third parameter, that you could send whether the key exists to an output variable. However, I have not found a way to accomplish this. Specifying either of these values results in an error when running this statement. If you know how to do this, please leave a remark so that this post can be updated with that information.

Using xp_regenumvalues / xp_instance_regenumvalues

These procedures enumerate through all of the values of the specified key, returning a separate result set for each value. For instance, the following statement will return all of the values in the above SQLServerAgent key:

Source code

EXECUTE master.sys.xp_instance_regenumvalues
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent';

Having all of these result sets makes it difficult to work with this procedure. Thankfully, you can put these all into one result set. Just create a temporary table (or table variable) to hold the output, and then use INSERT / EXECUTE to fill it, like the following example does:

Source code

DECLARE @Registry TABLE (VALUE VARCHAR(255), DATA VARCHAR(255));
INSERT INTO @Registry
EXECUTE master.sys.xp_instance_regenumvalues
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent';
SELECT * FROM @Registry;

Now that all of the results are in one result set, you can work with it a bit easier.

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regenumvalues 
    [@rootkey=]’rootkey’,
    [@key=]’key’

Using xp_regenumkeys / xp_instance_regenumkeys

These procedures enumerate through all of the keys in a specified path, and return all of the keys in that path. Unlike xp_instance_regenumvalues, all of the keys are returned in one result set, though you will probably want to use INSERT / EXECUTE to put this into temporary storage so that you can work with it. An example of using these procedures is:

Source code

EXECUTE master.sys.xp_instance_regenumkeys
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER';

The syntax for these procedures is:

Source code

EXECUTE xp_regenumkeys 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’

Modifying the registry from within SQL Server

Up to this point, we have focused on retrieving data from the registry. What if you want to modify the registry? Read on… be forewarned that the following procedures are modifying the registry, which means that they can also damage the registry, possibly rendering the server unusable. Use at your own risk!

xp_regwrite / xp_instance_regwrite

Use these procedures to create keys and write data into the registry. You can create up to 32 sub-keys at a time. In the following example, a new key “MyNewKey” will be added to the SQLServerAgent key, and the value “MyNewValue” will be added to this new key with the data “Now you see me!”. It will then read this value from the registry.

Source code

EXECUTE master.sys.xp_instance_regwrite
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue',
    'REG_SZ',
    'Now you see me!';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue';

Syntax

Source code

EXECUTE xp_regwrite 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’,
    [@value_name=]’value_name’,
    [@type=]’type’,
    [@VALUE=]’VALUE’

xp_regdeletevalue / xp_instance_regdeletevalue

These procedures are used to delete a specified value from the registry. In this example, the “MyNewValue” value will be deleted. The example first enumerates through all of the values in this key (just the one), deletes the value, enumerates through them again (since there are no values, there will be no result set), and then finally shows that the key is still present.

Source code

EXECUTE master.sys.xp_instance_regenumvalues
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';
EXECUTE master.sys.xp_instance_regdeletevalue
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue';
EXECUTE master.sys.xp_instance_regenumvalues
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regdeletevalue 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’,
    [@value_name=]’value_name’

xp_regdeletekey / xp_instance_regdeletekey

Use these procedures to delete an entire key from the registry. In this example, the script first adds another new key under “MyNewKey”, and a value in that new key. The script then deletes both the “AnotherNewKey” (which deletes the value just added also) and “MyNewKey” keys and finally shows that both keys have been deleted. Note that in order to delete a key, it cannot have any sub-keys, which is why the script deletes “AnotherNewKey” first (try running the script first by commenting out the first xp_instance_regdeletekey to see that it must be empty).

Source code

EXECUTE master.sys.xp_instance_regwrite
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey\AnotherNewKey',
    'MyNewValue',
    'REG_SZ',
    'Another new value!';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey\AnotherNewKey';
EXECUTE master.sys.xp_instance_regdeletekey
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey\AnotherNewKey';
EXECUTE master.sys.xp_instance_regdeletekey
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey\AnotherNewKey';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regdeletekey 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’

xp_regaddmultistring / xp_instance_regaddmultistring

Use these procedures to add a string to a multi-string entry in the registry, or to create a multi-string registry entry. In this example, I’ll call the procedure twice. The first time will create the entry with one string in it, and the second time will add a second string to it. Then the example will show the results of this value.

Source code

EXECUTE master.sys.xp_instance_regaddmultistring
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue',
    'A multi-string value!';
EXECUTE master.sys.xp_instance_regaddmultistring
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue',
    'Yet Another new string added to this multi-string value!';
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue'

Did you notice that the regread procedure got a little confused here? It has two value columns (which is the name of the value), but the second one has the data for the first one. Then the data column is null. If a third string is added, it still returns just these three columns. To see that we actually added these strings, we’ll have to use regedit.exe:

Source code

-- add a third string
EXECUTE master.sys.xp_instance_regaddmultistring
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue',
    'How about a third string?';
-- only shows the first string
EXECUTE master.sys.xp_instance_regread
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue'

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regaddmultistring 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’,
    [@value_name=]’value_name’,
    [@VALUE=]’VALUE’

xp_regremovemultistring / xp_instance_regremovemultistring

These procedures are used to remove a string from a multi-string entry. This example will remove the middle string.

Source code

EXECUTE master.sys.xp_instance_regremovemultistring
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey',
    'MyNewValue',
    'Yet Another new string added to this multi-string value!';

Examining the string now in regedit.exe, we can see it removed all of the strings starting with the specified string… in other words, it removed the second and third strings. If we add the strings back in, and then remove the third string, we can see that it also removes the second and third string. In testing with a fourth string, it appears that if you are deleting a string that is not the first string, then all of the remainder of the strings after the first string are removed. Deleting the first string deletes all of the strings. Well, these are undocumented procedures, so it’s not likely that this bug will ever be fixed.

Syntax

The syntax for these procedures is:

Source code

EXECUTE xp_regremovemultistring 
    [@rootkey=]’rootkey’,
    [@KEY=]’KEY’,
    [@value_name=]’value_name’,
    [@VALUE=]’VALUE’

And finally, let’s ensure that things are cleaned up:

Source code

EXECUTE master.sys.xp_instance_regdeletekey
    'HKEY_LOCAL_MACHINE',
    'Software\Microsoft\MSSQLSERVER\SQLServerAgent\MyNewKey';

Conclusion

With the exception of removing and reading the multi-string values, these extended stored procedures all work pretty well. We’ve been able to read registry values, and to enumerate through a list of keys and values. We can create and delete keys and values.

If you try to use the procedures in sections outside of SQL Server keys, you may run into registry security problems where the SQL Server service account doesn’t have permissions to accomplish the task. On the internet, you will find articles telling you to add the service account to the local administrators group (and to restart the service). This is a VERY BAD IDEA – by doing this, you can completely compromise the server. You could do anything that you desire on the server. Instead, use regedt32.exe, which will allow you to modify the permissions on the key that where you are having access issues.

I close by repeating my warning from above – be very careful when modifying the registry. It is possible to corrupt the registry to the point where the server will no longer function. To recover, an operating system reinstall will be necessary.

As a footnote on working with the instance-aware installation. I recall seeing on the internet other keywords that could be substituted instead of MSSQLSERVER. One of them is SQLServerAgent. However, I can’t find any others right now. If you happen to know of other keywords that work, please reply to this post so that it can be added to this post.

The post Working with the registry in SQL Server appeared first on SQL Solutions Group.

Some of the SQL Server Best Practices

Instant File Initialization

The first change appears during the installation of SQL Server 2016 itself. One of the best practice items that we look for in our Health Check is that the SQL Server Database Engine service has the “Perform Volume Maintenance Task” security privilege. SQL Server 2016 now allows the setting of this privilege right in the install screen.

The benefit of having this privilege set is that it allows SQL Server to utilize “Instant File Initialization” (IFI). When IFI is not set, then any growth of a database’s data file will be zero-initialized… that is, the entire growth portion of the file is overwritten with zeros (note that database log files are always zero-initialized… this is needed for the crash recovery process). During the time that the database data file is being zero-initialized, no other activity can occur in the database. This setting not only affects growths of data files, but also affects database restores and adding new files to a filegroup. Kimberly Tripp has an excellent post that explains this in more detail, and the kind of performance impact that can be obtained by having IFI enabled.

However, configuring the instance to utilize IFI has a potential negative security side-effect. If you have just dropped a database, and a growth occurs on another database, it could use the part of the drive that the other database was using. And although it is extremely unlikely, the pages could possibly line up. With advanced techniques, one could then read the data that was on those pages. Obviously, zero-initializing the pages would eliminate this problem. It is also obvious that there are a lot of circumstances that have to line up to allow this potential security issue…but the chance of it occurring will never be zero with IFI enabled.

Because of the performance impact, and extremely low security risk, most SQL Server professionals recommend having this option set.

tempdb

Ahh, tempdb. In SQL Server, that database does so much… it’s where temporary tables and table variables are stored. It’s where row version activity is stored. It’s where running queries will spill out temporary workspace needed for the query. And on, and on, and on. On busy systems, having this database running optimally is of vital importance. And there are several best practices that need to be observed here.

First up is the number of data files needed for tempdb. On a busy system that is creating a lot of temporary objects (specifically temporary tables and table variables), the creation of these objects creates a hot-spot in one of the database file’s internal page types that stores information about the allocation of these objects. However, each database file has these internal pages. Frequently, this contention can be greatly reduced by simply adding additional database files to this database. The specific advice for how many files to have has changed over time, and there is disagreement over the exact number of files to have. What isn’t disagreed over is that additional files are needed. And once again, during the SQL Server 2016 installation, this can be set, as seen in the following screen shot:

This screen shot shows several of the changes that are now available in the installation process. First off is the number of data files. Note that the installation limits the number of data files to the lower of 8 or the number of CPUs presented to SQL Server (one of the methods of determining how many files to have).

The next settings that this screen shot shows is the new default size and autogrowth values. Technically, these changes are made to the model database. However, they apply to all new databases, including tempdb when it is created. The initial size of the data and log files is now 8 MB. The default auto-growth of data and log files is now 64MB.

The next setting available in this screen is the directory to place the data and log files in. Specifically, you can have tempdb data files in multiple directories. During the installation, these files are created in the directories in a round-robin fashion.

Trace Flags

There are three specific trace flags that are commonly set in SQL Server. If trace flag (TF) 1117 is enabled, then whenever a data file grows, all of the files in that database will also grow. In SQL Server 2016, this trace flag has been replaced with the AUTOGROW_ALL_FLAGS or AUTOGROW_SINGLE_FILE option of the ALTER DATABASE command, and setting TF 1117 no longer has any effect. Since it is set as part of the ALTER DATABASE command, it can be controlled at the database level, where TF 1117 was a setting across the entire instance. Furthermore, because this setting is so valuable to have set in tempdb, by default tempdb is set with this feature enabled.

TF 1118 controls whether an extent can have pages belonging to multiple objects (known as a mixed extent), or to a single object (known as a dedicated extent). Using mixed extents allows having small tables use the absolute minimum amount of storage space…which isn’t really that much of an issue as it was in SQL 7.0 when it was added. Using dedicated extents can also help in the contention when creating lots of objects really fast. In SQL Server 2016, this trace flag has been replaced with the SET MIXED_PAGE_ALLOCATION option of the ALTER DATABASE command (and like TF 1117, TF1118 no longer has any effect). Furthermore, in the tempdb database, this setting is also enabled by default.

The next trace flag to talk about is TF 4199. This trace flag controls whether changes to how the query optimizer functions are used or not. In prior versions of SQL Server, the changes were not used unless this TF had been enabled. Starting in SQL Server 2016, most of these behaviors are enabled if the database is in the SQL Server 2016 compatibility level.

The post SQL Server ’16 helps you meet Best Practice appeared first on SQL Solutions Group.

The day started off with Ben Miller talking about PowerShell.

Next up, SSG founder Randy Knight addressed Isolation Levels, Locking, and Deadlocking.

Following Randy, my subject was SQL Injection, Referential Integrity (including optimizations that SQL Server can make when referential integrity is in place), and demonstrating the performance aspects of the various types of functions.

Jason Brimhall concluded the day with the topic of Extended Events.

As one attendee shared, “Great presentations! I learned a lot. Especially that NOLOCK is bad!”

At the end of the day, SSG gave away some prizes. Kevin Barnett and Delfin Esposo each walked away with a copy of the book SQL Server T-SQL Recipes, co-authored by SSG MCMs Jason Brimhall and me. James Rzepka won the coveted year of Amazon Prime/Amazon Fire tablet combo.

All of us at SQL Solutions Group had a wonderful time doing this FreeCon, and we hope that we can make this an annual event.

Pictures are courtesy of Slava Murygin (@SlavaSQL), who spent the entire week running around taking pictures of everything SQL related during this week of the PASS Summit. He described our FreeCon as a “world class SQL learning event.” Thanks, Slava!

The post SQL Solutions Group’s 5th Anniversary FreeCon appeared first on SQL Solutions Group.

The workshops that we will be presenting are:

Code Smells for the Consultant (which I’ll be presenting)

Throughout my career, I’ve seen developers do some pretty crazy things to databases ( I know because I come from a developer background). Come to this session to learn both what I (and SSG) look for and why it’s bad for the database (or your career), and alternatives that can be used. Some of the topics that I will discuss include; how coding mistakes open up the database for SQL Injection attacks, how coding choices can slow down the server, and how design choices keep SQL Server dumb ( if SQL Server was allowed to be smart, it would be faster!). Trust me, your DBA will love you for identifying and fixing these code smells.

A Masters Passport to Extended Events (presented by Jason Brimhall)

As is commonly the case, all good things come to an end.  And now is as good a time as any for the use of SQL Trace and Profiler to come to an end.  Let’s face it, Trace was a good tool and had some wonderful uses.  Profiler for that matter was a good tool and was useful at times.

It is time to let those old tools retire gracefully and move into the world of XE.  This workshop will provide you the means to let Profiler and Trace be retired from your toolset as you discover all that XE has to offer.

This focused session on Extended Events will help prepare you to put this tool to immediate use as you walk back to your daily duties.  This workshop will teach you about Extended Events starting with the basics and moving through to some specific XE sessions that I would use to troubleshoot in a client environment – while doing so with minimal impact.

You will be exposed to advanced troubleshooting techniques as we work through complex issues that are made easier through the use of XE.  Take advantage of this opportunity to dive into the world of Extended Events and learn how you can make best use of this tool in your SQL 2008+ environment.

You can read the rest of this article over here.

The post FreeCon, October 27, 2015 Seattle appeared first on Wayne Sheffield.

The post FreeCon, October 27, 2015 Seattle appeared first on SQL Solutions Group.

The first step is to grab the trace and XE scripts from the prior post at https://sqlsolutionsgroup.com/wean-off-sql-profiler-part-1/. Modify both scripts to put the output files in an appropriate place on your system. Run both of the scripts to start the trace and to create the XE session. Next, start the XE session with the following script:

Source code

ALTER EVENT SESSION Deadlocks
ON SERVER
STATE = START;

The next step is to create a deadlock. Open up a new query window, and run the following. Leave this query window open.

Source code

USE tempdb;
GO
IF OBJECT_ID('dbo.Test1') IS NOT NULL DROP TABLE dbo.Test1;
IF OBJECT_ID('dbo.Test2') IS NOT NULL DROP TABLE dbo.Test2;
CREATE TABLE dbo.Test1 (col1 INT);
CREATE TABLE dbo.Test2 (col2 INT);
INSERT INTO dbo.Test1 VALUES (1),(2),(3),(4),(5);
INSERT INTO dbo.Test2 VALUES (1),(2),(3),(4),(5);
GO
BEGIN TRANSACTION
UPDATE dbo.Test1 SET col1 = col1*10 WHERE col1=3;

Next, open up a second query window, and run the following code in that window:

Source code

USE tempdb;
BEGIN TRANSACTION;
UPDATE dbo.Test2 SET col2 = col2*20 WHERE col2 = 4;
UPDATE dbo.Test1 SET col1 = col1*20 WHERE col1 = 3;
COMMIT TRANSACTION;

Finally, return to the first query window and run the following code, at which point one of the statements in one of the query windows will be deadlocked:

Source code

UPDATE dbo.Test2 SET col2 = col2*10 WHERE col2 = 4;
COMMIT TRANSACTION;

Transaction (Process ID 57) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Now that we’ve created a deadlock, let’s compare the trace output data to the XE output data. First, let’s grab the data from the trace file with this script, which selects all of the non-null columns from the table valued function (remember to change the filename/path as appropriate):

Source code

SELECT t2.TextData, t2.BinaryData, t2.NTUserName, t2.ClientProcessID, t2.ApplicationName,
       t2.LoginName, t2.SPID, t2.Duration, t2.StartTime, t2.EndTime, t2.ObjectID,
       t2.ServerName, t2.EventClass, t2.Mode, t2.DatabaseName, t2.Type
FROM   sys.traces t1
CROSS APPLY sys.fn_trace_gettable (path, NULL) t2
WHERE t1.path LIKE 'C:\SQL\Traces\%'

We now want to compare this to the data that was collected by the Extended Event session. In SSMS, expand the Management Tree, then the Extended Events tree, Session tree, and the Deadlocks tree:

The event file target can now be seen. Double-click the file target to view the data, and you can see the events that fired for the deadlock. Note that you can right-click on the grid header, and choose additional columns to put into the grid – for this XE session, I like to add the database_name, resource_description, resource_owner, resource_type and xml_report columns.

By selecting one of the rows, all of the data will be available in the lower grid in a Name/Value format.

Let’s compare the XE output to the trace output, and ensure that the data captured by the trace is present in the XE output. The first trace column is TextData. In that, we see multiple bits of data: SPID, lock resource, and the deadlock graph in XML format. In the XE, the resource_description column has the lock resource, the xml_report column has the deadlock graph in XML format, and the session_id column has the SPID. Skipping the BinaryData column, the next column is the NTUserName. Here we can see that this is collected in the XE nt_username column. A minor difference is that the XE includes the domain name, and the trace doesn’t.

The ClientProcessID column maps to the client_pid column. Likewise, the ApplicationName maps to client_app_name, LoginName maps to server_principal_name, SPID maps to session_id, Duration maps to duration, StartTime maps to timestamp, ObjectID maps to object_id. The EventClass, Mode and Type columns maps via friendly names to the name, mode and resource_type columns. The only columns that aren’t explicitly in the XE results are the BinaryData, EndDate and ServerName columns. The EndDate can be calculated from the timestamp and duration columns, and we could have selected the server_instance_name in “Global Fields (Actions)” tab when configuring the XE session. So here, we can see that everything needed from the trace is available in the XE session.

For the next step, one might want to load the data into a table, and viewing it in the viewer doesn’t allow for this. Or maybe you are working on a server prior to SQL Server 2012, and the GUI isn’t available for you to use. The following script can be used to query the data from the file target (again, remember to change the file name/path as necessary):

Source code

SELECT OBJECT_NAME,
       CONVERT(XML, event_data) AS event_data
FROM   sys.fn_xe_file_target_read_file('C:\SQL\XE_Out\Deadlocks*.xel', NULL, NULL, NULL);

As can be seen from the output, the data that we most want to see is stored as XML (when working with Extended Events, all of the data is stored as XML) in the event_data column, so I have converted this column to XML. At this point, you can click the XML output, and it will be opened up in text, allowing you to see all of the data as it is stored in the XML output.

From this point, it’s just a matter of modifying the query to return the columns that you are interested in.

Source code

WITH cte AS
(
SELECT  t2.event_data.VALUE('(event/@name)[1]','varchar(50)') AS event_name,
        t2.event_data.VALUE('(event/@timestamp)[1]', 'datetime2') AS StartTime,
        t2.event_data.VALUE('(event/data[@name="duration"]/value)[1]', 'bigint') AS duration,
        t2.event_data.VALUE('(event/data[@name="database_name"]/value)[1]', 'sysname') AS DBName,
        t2.event_data.VALUE('(event/action[@name="nt_username"]/value)[1]', 'varchar(500)') AS nt_username,
        t2.event_data.VALUE('(event/data[@name="mode"]/value)[1]', 'varchar(15)')  + ' (' + 
        t2.event_data.VALUE('(event/data[@name="mode"]/text)[1]', 'varchar(50)') + ')' AS mode,
        t2.event_data.VALUE('(event/data[@name="object_id"]/value)[1]', 'integer') AS OBJECT_ID,
        t2.event_data.VALUE('(event/data[@name="resource_description"]/value)[1]', 'varchar(max)') AS resource_description,
        t2.event_data.VALUE('(event/data[@name="resource_owner_type"]/text)[1]', 'varchar(max)')  AS resource_owner_type,
        t2.event_data.VALUE('(event/data[@name="resource_type"]/text)[1]', 'varchar(max)') + ' (' +
        t2.event_data.VALUE('(event/data[@name="resource_type"]/value)[1]', 'varchar(max)') + ')' AS resource_type,
        t2.event_data.VALUE('(event/action[@name="server_principal_name"]/value)[1]', 'varchar(max)') AS server_principal_name,
        t2.event_data.VALUE('(event/action[@name="session_id"]/value)[1]', 'varchar(max)') AS session_id,
        t2.event_data.VALUE('(event/action[@name="client_pid"]/value)[1]', 'integer') AS client_pid,
        t2.event_data.VALUE('(event/action[@name="client_app_name"]/value)[1]', 'varchar(max)') AS client_app_name,
        t2.event_data
FROM    sys.fn_xe_file_target_read_file('C:\SQL\XE_Out\Deadlocks*.xel', NULL, NULL, NULL) t1
CROSS APPLY (SELECT CONVERT(XML, t1.event_data)) t2(event_data)
)
SELECT  cte.event_name,
        cte.StartTime,
        DATEADD(MICROSECOND, duration, CONVERT(DATETIME2, [cte].StartTime)) AS EndDate,
        cte.duration,
        cte.DBName,
        cte.nt_username,
        cte.server_principal_name,
        cte.mode,
        cte.OBJECT_ID,
        cte.resource_description,
        cte.resource_owner_type,
        cte.resource_type,
        cte.session_id,
        cte.client_pid,
        cte.client_app_name,
        cte.event_data 
FROM    cte;

This query extracts the data from XE file target. It also calculates the end date, and displays both the internal and user-friendly names for the resource_type and mode columns – the internal values are what the trace was returning.

For a quick recap: in Part 1, you learned how to convert an existing trace into an XE session, identifying the columns and stepping through the GUI for creating the XE session. In part 2 you learned how to query both the trace and XE file target outputs, and then compared the two outputs and learned that all of the data in the trace output is available in the XE output.

The post XE Output: Weaning yourself off of SQL Profiler, Part 2 appeared first on SQL Solutions Group.

It just so happens that today is the 67^th installment of the monthly T-SQL Tuesday blogging event. T-SQL Tuesday, that wonderful monthly blogging party started by Adam Machanic where a selected host challenges the SQL Server universe to have a blog post about a specific topic. This wild frenzy of SQL Server blog posting occurs on the second Tuesday of each month. This month, it is being hosted by my friend (and the person with the highest amount of energy known to mankind) Jes Borland (b / t), and the topic that she has chosen is Extended Events. Her specific challenge to the SQL Server universe is:

I want to know (and others do, too) how you’ve solved problems with Extended Events. What sessions have you created? What unique way have you used predicates or targets? What challenges have you overcome?

The biggest challenge that I have is not a technical challenge… it’s a personal challenge: actually getting started with XE. I have so many scripts for doing traces, that I just immediately use them instead of the better XE system. I really need to wean myself off of using Profiler / traces. Therefore, I’ve decided to start converting my trace scripts into XE scripts, and I’ll share with you how I go about doing it. Today, I’m going to look at my favorite trace script – a trace to capture deadlock information.

First off, let’s start with the trace script:

Source code

-- Create a trace to capture deadlocks
DECLARE @rc INTEGER,
        @TraceID INTEGER,
        @maxfilesize BIGINT,
        @OutputFileName NVARCHAR(256),
        @TraceStopTime DATETIME,
        @ON BIT,
        @intfilter INT,
        @bigintfilter BIGINT;
 
SET @ON = 1;
SET @maxfilesize = 10; --mb 
DECLARE @FileName NVARCHAR(256);
-- added InstanceName for when server is running multiple instances
SET @FileName = N'C:\SQL\Traces\' + CONVERT(sysname, SERVERPROPERTY('InstanceName')) + '\' +
               CONVERT(CHAR(8), GETDATE(), 112) + '_' +
               REPLACE(CONVERT(CHAR(8), GETDATE(), 114), ':', '');
EXECUTE xp_create_subdir @FileName;               
SET @FileName = @FileName + N'\' + REPLACE(@@SERVERNAME, N'\', N'_') + '_DeadlockTrace_' + CONVERT(CHAR(8), GETDATE(), 112) + N'_' + REPLACE(CONVERT(CHAR(8), GETDATE(), 114), N':', N'') + N'Z';
PRINT @FileName;
SET @TraceStopTime = DATEADD(HOUR, 6, GETDATE());
EXEC @rc = sp_trace_create @TraceID output, 2, @FileName, @maxfilesize, @TraceStopTime;
 
IF (@rc != 0) GOTO error
 
-- Set the events
-- Event 148: Deadlock Graph
EXEC sp_trace_setevent @TraceID, 148, 1, @on;   -- TextData
EXEC sp_trace_setevent @TraceID, 148, 11, @on;  -- LoginName
EXEC sp_trace_setevent @TraceID, 148, 12, @on;  -- SPID
EXEC sp_trace_setevent @TraceID, 148, 14, @on;  -- StartTime
-- Event 25: Lock:Deadlock
EXEC sp_trace_setevent @TraceID, 25, 1, @on;    -- TextData
EXEC sp_trace_setevent @TraceID, 25, 2, @on;    -- BinaryData
EXEC sp_trace_setevent @TraceID, 25, 6, @on;    -- NTUserName
EXEC sp_trace_setevent @TraceID, 25, 9, @on;    -- ClientProcessID
EXEC sp_trace_setevent @TraceID, 25, 10, @on;   -- ApplicationName
EXEC sp_trace_setevent @TraceID, 25, 11, @on;   -- LoginName
EXEC sp_trace_setevent @TraceID, 25, 12, @on;   -- SPID
EXEC sp_trace_setevent @TraceID, 25, 13, @on;   -- Duration
EXEC sp_trace_setevent @TraceID, 25, 14, @on;   -- StartTime
EXEC sp_trace_setevent @TraceID, 25, 15, @on;   -- EndTime
EXEC sp_trace_setevent @TraceID, 25, 22, @on;   -- ObjectID
EXEC sp_trace_setevent @TraceID, 25, 32, @on;   -- Mode
EXEC sp_trace_setevent @TraceID, 25, 35, @on;   -- DatabaseName
EXEC sp_trace_setevent @TraceID, 25, 57, @on;   -- Type
-- Event 59: Lock:Deadlock Chain
EXEC sp_trace_setevent @TraceID, 59, 1, @on;    -- TextData
EXEC sp_trace_setevent @TraceID, 59, 2, @on;    -- BinaryData
EXEC sp_trace_setevent @TraceID, 59, 12, @on;   -- SPID
EXEC sp_trace_setevent @TraceID, 59, 14, @on;   -- StartTime
EXEC sp_trace_setevent @TraceID, 59, 22, @on;   -- ObjectID
EXEC sp_trace_setevent @TraceID, 59, 32, @on;   -- Mode
EXEC sp_trace_setevent @TraceID, 59, 35, @on;   -- DatabaseName
EXEC sp_trace_setevent @TraceID, 59, 57, @on;   -- Type
 
 
-- Set the Filters
-- Set the trace status to start
EXEC sp_trace_setstatus @TraceID, 1;
 
-- display trace id for future references
SELECT TraceID=@TraceID;
GOTO finish;
 
error: 
SELECT ErrorCode=@rc;
 
finish: 
GO

From this script, you can see that the trace is collecting data from three events (Deadlock Graph (148), Lock:Deadlock (25) and Lock:Deadlock Chain (59)) and several columns for each event. The next step is to convert all of this information into XE events / actions. For this, I’ll modify the script in BOL (at https://msdn.microsoft.com/en-us/library/ff878264.aspx) to the following to return the XE events / actions for a specified running trace. Since the trace needs to be running, execute the above trace script to start the trace, then run the following code:

Source code

-- change the following value to the trace_id for the running trace to be converted to XE
DECLARE @trace_id INTEGER = 2;
 
SELECT DISTINCT
        tb.trace_event_id,
        te.name AS 'Event Class',
        em.package_name AS 'Package',
        em.xe_event_name AS 'XEvent Name',
        tb.trace_column_id,
        tc.name AS 'SQL Trace Column',
        am.xe_action_name AS 'Extended Events action'
FROM    sys.FN_TRACE_GETEVENTINFO(@trace_id) ei
        JOIN sys.trace_events te
            ON te.trace_event_id = ei.eventid
        JOIN sys.trace_columns tc  -- all available trace columns
            ON tc.trace_column_id = ei.columnid
        LEFT OUTER JOIN sys.trace_xe_event_map em
            ON em.trace_event_id = te.trace_event_id
        LEFT OUTER JOIN sys.trace_event_bindings tb
            ON tb.trace_event_id = em.trace_event_id
            AND tc.trace_column_id = tb.trace_column_id
        LEFT OUTER JOIN sys.trace_xe_action_map am
            ON am.trace_column_id = tc.trace_column_id
ORDER BY tb.trace_event_id, tb.trace_column_id;

These results may contain NULL values in the “Extended Events action” column. A NULL value here means that there is not a corresponding event action for that column. For my deadlock trace, I get the following results:

With this information, I can now jump into SSMS (2014) and use the New Session wizard to create a new XE session (Expand the Management and Extended Events nodes. Right click on Sessions and select New Session). I name the session “Deadlocks”, and click the “Events” page to select the events that I want to use.

Since all of the events that I’m looking for contain “Deadlock”, I search for this and 5 events are displayed. I double-click each of the three listed above (or select them and click the “>” button) to move them to the “Selected Events” grid.

The next step is to configure the columns, so click the Configure button.

Select the lock_deadlock event, and on the “Global Fields (Actions)” tab select the client_app_name, client_pid, database_name, nt_username, server_principal_name and session_id columns.

Click on the “Event Fields” tab, and check the checkboxes for database_name and resource_description. Fields that show up in this tab with checkboxes have customizable actions, and don’t actually collect the information unless checked. Frequently this will be because these fields require an additional level of resource usage to gather this information.

Select the lock_deadlock_chain event, and select the session_id and database_name columns on the “Global Fields (Actions)” tab. Click on the “Event Fields” tab and check the checkbox for database_name and resource_description. Finally, select the xml_deadlock_report event, and select the server_principal_name and session_id columns on the “Global Fields (Actions)” tab.

Finally, click on the Data Storage page, and add an event_file target type. The file name defaults to the name that you gave the session earlier. I set the maximum size to 250mb and 5 rollover files. If you select to script to a new query window (HIGHLY recommended), you will have a script with all of the DDL necessary to create this XE.

At this point, there are two changes that I like to make. First, I like to make my scripts so that they won’t have an error, and if the XE session already exists then one will be generated. To make it bullet-proof, I add an IF EXISTS check at the top and drop the XE session if it already exists. Secondly, I like to call xp_create_subdir to create the directory that the script points to, just in case the directory doesn’t exist. Note that xp_create_subdir will be successful if the directory already exists, so I don’t check to ensure that the directory doesn’t exist before executing this procedure.

My final XE Deadlock script looks like:

Source code

IF EXISTS (SELECT 1 FROM sys.server_event_sessions WHERE name = 'Deadlocks')
	DROP EVENT SESSION [Deadlocks] ON SERVER;
GO
EXECUTE xp_create_subdir 'C:\SQL\XE_Out';
GO
CREATE EVENT SESSION [Deadlocks]
ON SERVER
ADD EVENT sqlserver.lock_deadlock(
    SET collect_database_name=(1),collect_resource_description=(1)
	ACTION 
	(
			  sqlserver.client_app_name	-- ApplicationName from SQLTrace
			, sqlserver.client_pid	-- ClientProcessID from SQLTrace
			, sqlserver.nt_username	-- NTUserName from SQLTrace
			, sqlserver.server_principal_name	-- LoginName from SQLTrace
			, sqlserver.session_id	-- SPID from SQLTrace
	)
),
ADD EVENT sqlserver.lock_deadlock_chain(
    SET collect_database_name=(1),collect_resource_description=(1)
	ACTION 
	(
			  sqlserver.session_id	-- SPID from SQLTrace
	)
),
ADD EVENT sqlserver.xml_deadlock_report(
	ACTION 
	(
			  sqlserver.server_principal_name	-- LoginName from SQLTrace
			, sqlserver.session_id	-- SPID from SQLTrace
	)
)
ADD TARGET package0.event_file
(
	SET filename = 'C:\SQL\XE_Out\Deadlocks.xel',
		max_file_size = 250,
		max_rollover_files = 5
)
WITH (STARTUP_STATE=OFF)
;

Now that you have this XE session scripted out, it can be easily installed on multiple servers. If you encounter a deadlock problem, you can easily start the XE session and let it run to trap your deadlocks. They will be persisted to a file dedicated for the deadlocks. You can use my Deadlock Shredder script at https://bit.ly/ShredDL to read the deadlocks from the file and shred the deadlock XML into a tabular output.

Note that the default system_health XE session also captures deadlocks. I like to have a dedicated session for just deadlocks. As lightweight as XE is, sometimes it may benefit a server to turn off the system_health session. Additionally, Jonathan Kehayias has a script that will take a running trace and completely script out an XE session for it. This script can be found at https://www.sqlskills.com/blogs/jonathan/converting-sql-trace-to-extended-events-in-sql-server-2012/. Even though this script is available, I like to figure things out for myself so that I can learn what is actually going on.

Did you catch the note above where I mentioned that the default system_health XE captures deadlocks? This means that if you are still enabling trace flags 1204 and/or 1222 to capture deadlock information in your error logs, you don’t need to do that anymore. Furthermore, by using this Deadlock XE, you can have the deadlocks persisted to a file where, in combination with my Deadlock Shredder script, it will be even easier to analyze the deadlocks than trying to figure it out from the captured information in the error logs.

I hope that this post will help someone with the conversion from using SQL Profiler to using XE instead. The process for converting other traces would be the same, so it can be easily adapted. I also hope that these deadlock scripts will be helpful to you!

In my next post, I will be comparing the results of the deadlock trace and the deadlock XE session.

And Jes – thanks for hosting this month! I’m really looking forward to seeing the roundup for this month.

The post Weaning yourself off of SQL Profiler (Part 1) appeared first on SQL Solutions Group.

On a nice wintry day, your city ended up being covered in several feet of snow. During the course of the night, your own house had several power outages. Being concerned about your databases, you shoveled your way into your office, so that you could check on things. (Okay… actually you would just VPN in, but this is my story after all…)

Once you get into your server, you check the jobs and find that your job that runs DBCC CHECKDB has failed. Let’s assume that a power glitch has caused corruption in your database. In order to find out what all is affected, you run DBCC CHECKDB WITH NO_INFOMSGS, ALL_ERRORMSGS. But, this happens to be on your 2TB database, and it will take a while for CHECKDB to finish so that you can find the scope of corruption.

You’d really like to know what tables are affected without having to wait. Luckily(?), this corruption was recorded in msdb.dbo.suspect_pages, and having just recently read Paul Randal’s post at here, we know we can use DBCC PAGE to determine this information. And, after having read my last blog post, you know that we can automate DBCC PAGE, so we can use our new friend “WITH TABLERESULTS” to find out what objects have been corrupted.

The suspect_pages table, documented here, has three particular columns of interest: database_id, file_id and page_id. These correspond nicely to the first three parameters needed for DBCC PAGE. To automate this, we need to know what information we need to return off of the page – and from Paul’s post, we know that this is the field “METADATA: ObjectId”. For this code example, let’s assume that this corruption is on page 11 of the master database (just change “master” to the name of your 2TB database).

Source code

IF OBJECT_ID('tempdb.dbo.#DBCCPAGE') IS NOT NULL DROP TABLE #DBCCPAGE;
CREATE TABLE #DBCCPAGE (
       ParentObject VARCHAR(255),
       [OBJECT]     VARCHAR(255),
       Field       VARCHAR(255),
       [VALUE]     VARCHAR(255));
INSERT INTO #DBCCPAGE
EXECUTE ('DBCC PAGE (''master'', 1, 11, 3) WITH TABLERESULTS;');
 
SELECT schema_name = OBJECT_SCHEMA_NAME(ca.OBJECT_ID),
       OBJECT_NAME = OBJECT_NAME(ca.OBJECT_ID)
FROM   #DBCCPAGE
       CROSS APPLY (SELECT CONVERT(INTEGER, VALUE)) ca(OBJECT_ID)
WHERE   Field = 'Metadata: ObjectId';

And there you go… you now know which object it is that has the corruption. In the same fashion, another interesting field that is returned is the IndexId – the Field value is “Metadata: IndexId”. It would be a similar exercise to grab that from this page also, an exercise that I’ll leave to you.

An automated method for getting the object from all suspect pages would entail encapsulating this logic into a cursor to spin through each row in the suspect_pages table (and I’ll even throw in getting the index_id also):

Source code

IF OBJECT_ID('tempdb.dbo.#DBCCPAGE') IS NOT NULL DROP TABLE #DBCCPAGE;
IF OBJECT_ID('tempdb.dbo.#SuspectObjects') IS NOT NULL DROP TABLE #SuspectObjects;
CREATE TABLE #DBCCPAGE (
        ParentObject    VARCHAR(255),
        [OBJECT]        VARCHAR(255),
        Field           VARCHAR(255),
        [VALUE]         VARCHAR(255));
CREATE TABLE #SuspectObjects (
        database_id     INTEGER,
        FILE_ID         INTEGER,
        page_id         INTEGER,
        OBJECT_ID       INTEGER,
        index_id        INTEGER);
DECLARE @database_id    INTEGER,
        @FILE_ID        INTEGER,
        @page_id        INTEGER,
        @SQLCMD         NVARCHAR(MAX);
 
DECLARE cCrackSuspectPages CURSOR LOCAL FAST_FORWARD FOR
SELECT  'EXECUTE (''DBCC PAGE (' +
            CONVERT(VARCHAR(15), database_id) + ', ' +
            CONVERT(VARCHAR(15), FILE_ID) + ', ' +
            CONVERT(VARCHAR(15), page_id) + ') WITH TABLERESULTS;'');',
        database_id, FILE_ID, page_id
FROM    msdb.dbo.suspect_pages;
 
OPEN cCrackSuspectPages;
FETCH NEXT FROM cCrackSuspectPages INTO @SQLCMD, @database_id, @FILE_ID, @page_id;
WHILE @@FETCH_STATUS = 0
BEGIN
    TRUNCATE TABLE #DBCCPAGE;
    INSERT INTO #DBCCPAGE EXECUTE (@SQLCMD);
 
    INSERT INTO #SuspectObjects
            (database_id,
             FILE_ID,
             page_id,
             OBJECT_ID,
             index_id
            )
    SELECT  @database_id, @FILE_ID, @page_id,
            (SELECT CONVERT(INTEGER, VALUE)
             FROM   #DBCCPAGE dp
             WHERE  dp.Field = 'Metadata: ObjectId'),
            (SELECT CONVERT(INTEGER, VALUE)
             FROM   #DBCCPAGE dp
             WHERE  dp.Field = 'Metadata: IndexId');
 
    FETCH NEXT FROM cCrackSuspectPages INTO @SQLCMD, @database_id, @FILE_ID, @page_id;
END
CLOSE cCrackSuspectPages;
DEALLOCATE cCrackSuspectPages;
 
SELECT  database_name = DB_NAME(database_id),
        database_id,
        FILE_ID,
        page_id,
        schema_name = OBJECT_SCHEMA_NAME(OBJECT_ID, database_id),
        OBJECT_NAME = OBJECT_NAME(OBJECT_ID, database_id),
        index_id
FROM    #SuspectObjects;

If you happen to be on SQL 2012 or higher, this can be greatly simplified by using the new (undocumented) DMO function sys.dm_db_database_page_allocations (and it also takes away the need to crack the page using DBCC PAGE).

Source code

SELECT  database_name = DB_NAME(sp.database_id),
        sp.database_id,
        sp.FILE_ID,
        sp.page_id,
        schema_name = OBJECT_SCHEMA_NAME(dpa.OBJECT_ID, sp.database_id),
        OBJECT_NAME = OBJECT_NAME(dpa.OBJECT_ID, sp.database_id),
        dpa.index_id
FROM    msdb.dbo.suspect_pages sp
CROSS APPLY sys.dm_db_database_page_allocations(sp.database_id, NULL, NULL, NULL, 'LIMITED') dpa
WHERE   sp.FILE_ID = dpa.allocated_page_file_id
AND     sp.page_id = dpa.allocated_page_page_id;

And there we go – yet another time when you might want to automate using DBCC PAGE. By now you should be able to see other uses for it – as long as you can get the database_id, file_id and page_id, you can automate the usage of it to retrieve the information that you are looking for.

The post Automating DBCC PAGE, part 2: Determining object names on pages appeared first on SQL Solutions Group.

If we were to examine Books Online (BOL) for DBCC in 2000 (https://technet.microsoft.com/en-us/library/aa258281%28v=sql.80%29.aspx) and 2005 (https://msdn.microsoft.com/en-us/library/ms188796%28v=sql.90%29.aspx), we would notice a section titled “Using DBCC Result Set Output”, with the phrase: “Many DBCC commands can produce output in tabular form by using the WITH TABLERESULTS option. This information can be loaded into a table for additional use.” This section has been removed from more recent versions, most likely because the results returned are not documented for the individual DBCC commands and are thus subject to change without notice.

Okay, let’s try this out. At the above BOL links, there are two DBCC commands that are documented to use the TABLERESULTS option: OPENTRAN and SHOWCONTIG. Testing all of the other DBCC commands shows that this option can also be used on the CHECKALLOC, CHECKDB, CHECKFILEGROUP and CHECKTABLE commands. I’m going to continue this post with using DBCC CHECKDB. If you check BOL, it does not mention the TABLERESULTS option. Let’s first see the results we get without the TABLERESULTS option by running:

Source code

DBCC CHECKDB (master);

For the consistency check that is performed on the master database, I end up with over 300 lines of output. The key lines in the output are:

Source code

…
CHECKDB found 0 allocation errors and 0 consistency errors in database 'master'.
…

Source code

CHECKDB found 0 allocation errors and 0 consistency errors in database 'mssqlsystemresource'. 
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

If we were to modify the above statement to use the TABLERESULTS option:

Source code

DBCC CHECKDB (master) WITH TABLERESULTS;

we would actually get two result sets – one for the master database, and one for the hidden mssqlsystemresource database. Notice that the “DBCC execution completed…” line is not in either of the result sets – it is still displayed on the Messages tab.

Now, most people usually modify this command to suppress informational messages and to show all error messages (it defaults to “only” the first 200 error messages per object), so the command that is normally used would be:

Source code

DBCC CHECKDB (master) WITH ALL_ERRORMSGS, NO_INFOMSGS;

And, hopefully, this only returns the message “Command completed successfully”. Let’s modify this command to use the TABLERESULTS option:

Source code

DBCC CHECKDB (master) WITH ALL_ERRORMSGS, NO_INFOMSGS, TABLERESULTS;

If there is no corruption in the database, it still returns only the message “Command completed successfully”. However, if there is corruption, you will get a result set back. So, I’m now going to run this against a database (Lab) where I have engineered some corruption. First off, let’s run CHECKDB without the TABLERESULTS option to check the initial output:

Source code

DBCC CHECKDB (Lab) WITH ALL_ERRORMSGS, NO_INFOMSGS;

Source code

Msg 8939, LEVEL 16, STATE 98, Line 1
TABLE error: OBJECT ID 885578193, INDEX ID 0, partition ID 72057594039042048, alloc unit ID 72057594043432960 (type In-ROW DATA), page (1:328). Test (IS_OFF (BUF_IOERR, pBUF->bstat)) failed. VALUES ARE 133129 and -4.
Msg 8928, LEVEL 16, STATE 1, Line 1
OBJECT ID 885578193, INDEX ID 0, partition ID 72057594039042048, alloc unit ID 72057594043432960 (type In-ROW DATA): Page (1:328) could not be processed. See other errors FOR details.
CHECKDB FOUND 0 allocation errors and 2 consistency errors in TABLE 'Tally' (OBJECT ID 885578193).
CHECKDB FOUND 0 allocation errors and 2 consistency errors in DATABASE 'Lab'.
repair_allow_data_loss IS the minimum repair LEVEL FOR the errors FOUND BY DBCC CHECKDB (Lab).

Here we can see that we indeed have corruption. Quickly now… is any of this corruption in a non-clustered index?

Running this statement with the TABLERESULTS option, we get a grid of the results:

Source code

DBCC CHECKDB (Lab) WITH ALL_ERRORMSGS, NO_INFOMSGS, TABLERESULTS;

(This screen shot is only showing some of the columns. Note that the message of the previous output for the minimum repair level is still on the Messages tab.)

As you can see, this already makes examining your errors somewhat easier. For instance, you can easily scroll through this output to see if the corruption is in non-clustered indexes (IndexId > 1), where the corruption could easily be fixed by scripting out the index definition, dropping and then re-creating the non-clustered index. Suppose you had over 100 errors… you can see how much faster this would be.

If you recall, the BOL description says that this data can be loaded into a table for further processing. Furthermore, this post is about persisting DBCC output data, which implies storing it into a table. So, let’s make a table, put these results into the table, and then run a query against it. First off, let’s make a local temporary table to hold the results (you could put this into a permanent table if you so desire):

Source code

IF OBJECT_ID('tempdb.dbo.#DBCCCHECKDB') IS NOT NULL
   DROP TABLE #DBCCCHECKDB;
CREATE TABLE #DBCCCHECKDB (
   Error       INTEGER,
   [LEVEL]     INTEGER,
   [STATE]     INTEGER,
   MessageText VARCHAR(MAX),
   RepairLevel VARCHAR(MAX),
   [Status]   INTEGER,
   [DbId]     INTEGER,
   DbFragId   INTEGER,
   ObjectId   INTEGER,
   IndexId     INTEGER,
   PartitionId BIGINT,
   AllocUnitId BIGINT,
   RidDbId     INTEGER,
   RidPruId   INTEGER,
   [FILE]     INTEGER,
   [Page]     INTEGER,
   Slot       INTEGER,
   RefDbId     INTEGER,
   RefPruId   INTEGER,
   RefFile     INTEGER,
   RefPage     INTEGER,
   RefSlot     INTEGER,
   Allocation INTEGER);

Since the output from using TABLERESULTS isn’t documented, I’ve had to make some assumptions about the data types for these columns. To actually insert the output, we need to use the INSERT INTO … EXECUTE statement:

Source code

Transact-SQL
INSERT INTO #DBCCCHECKDB
EXECUTE ('DBCC CHECKDB (Lab) WITH ALL_ERRORMSGS, NO_INFOMSGS, TABLERESULTS;');

Now that we have the data stored in a temporary table, let’s run a query against the table to return some aggregated data.

— Get the objects, # of errors, and repair level

Source code

SELECT [DbId],
       database_name = DB_NAME([DbId]),
       ObjectId,
       [OBJECT] = OBJECT_SCHEMA_NAME(ObjectId, [DbId])
                   + '.' +
                   OBJECT_NAME(ObjectId, [DbId]),
       ErrorQty = COUNT(*),
       RepairLvl = MAX(RepairLevel)
FROM   #DBCCCHECKDB
WHERE   Error NOT IN (8989,8990)
GROUP BY [DbId], ObjectId;

With which I get the following results:

So, there we go. Easy-peasy. By utilizing the TABLERESULTS option, the output of the DBCC CHECKDB command has been persisted into a table, and we are now able to run our own queries against that data. In the event that there is corruption in multiple indexes in a table, this query could easily be extended to get the number of errors in each index.

The post Persisting DBCC output data appeared first on SQL Solutions Group.

In a CPU universe far, far away, there existed a particular series known as the 80486 processor, commonly called just 486. One particular issue that this series of CPUs had was that they were too fast for many games of that era (games that were coded for running off of CPU ticks). If you can get hold of one of these games and try to run it on a modern system, I wish you luck in even being able to press a single key before the game ends! But I digress… in order to counteract this issue, the processors had a feature where they could be slowed down. There were different ways that the CPU could be slowed down, but to interact with the computer user, there was a button on the case, known as the Turbo button, that would cycle the system between the low and high speeds of the CPU. Accidentally leaving this button in the low speed would slow down everything else on that computer; merely putting the system into the high speed would fix your performance problem.

So what does this mean for SQL Server?

In a CPU universe very, very close to you, today CPUs have a similar feature known as CPU Throttling. This feature, otherwise known as Dynamic Frequency Scaling, dynamically adjusts the CPU frequency on the fly. There are several benefits from doing this: less heat is generated, less power is consumed, and it can even make the systems quieter by also reducing the fan speed necessary to cool the computer. In a large server farm, reducing the heat means that cooling costs are also reduced. By operating the systems in a reduced state, the lower cooling costs coupled with the lower power requirements of the computers themselves can mount up to quite a savings. The dynamic portion of this feature is based upon system monitoring of CPU usage, and when it is low the CPU frequency is reduced, typically by 50% or more.

For most systems, the computer’s BIOS will monitor certain communication from the operating system, so the operating system can also send instructions for setting the CPU Throttling. In the Windows operating system, this is performed through the “Power Plan” setting. Since Windows Server 2008, the default power plan is “Balanced”, which allows Windows to reduce the CPU speed in an effort to balance power and performance.

How does this affect SQL Server?

SQL Server is one application that doesn’t typically use CPU resources so much as to make the CPU Throttling disengage and the CPUs to run at full power. This manifests itself in SQL Server as queries just taking longer to run. You might even have a new server that the application runs slower on than the server it is replacing. The solution is to simply disengage the CPU Throttling feature so that the CPUs will run at full speed constantly.

Detecting if CPU Throttling is engaged

CPU Throttling is implemented at both the hardware and software level. The Windows operating system can control CPU Throttling, and this can also be controlled from the BIOS. On some systems, the BIOS will ignore signals sent from Windows, so we will cover how to test and set both methods.

To see if your CPUs are running at rated speed, there are two methods. The first is the third-party tool CPU-Z, available at https://www.cpuid.com/softwares/cpu-z.html. In this screen shot (from the above link), you can see that the CPU is rated to run at 3.2GHz, but is actually running at almost 1.2GHz. This system is having its CPU throttled.

Additionally, you can check out the WMI performance counters from the WIN32_Processor class. This DOS command returns the current and max clock speeds every second for 30 seconds; if the CurrentClockSpeed and the MaxClockSpeed differ significantly, then the CPU is being throttled.

Source code

WMIC CPU GET CurrentClockSpeed, MaxClockSpeed /VALUE /EVERY:1 /REPEAT:30

In Windows, CPU Throttling is controlled from the Power Settings control panel applet. On a server running SQL Server, the desired setting is to be using the the “High Performance” power plan. You can do this from either the Power Settings control panel applet, or from the command line (starting with Windows Server 2008R2).

To start the Power Settings control panel applet, press the windows key + r key combination. In the run dialog box, enter powercfg.cpl. In the applet, you are looking for:

From a command prompt, you can simply run this DOS command:

Source code

powercfg -GETACTIVESCHEME

Disabling CPU Throttling

To change the power plan, in the control panel applet simply select the High Performance power plan (as shown above). From the DOS command prompt:

Source code

powercfg -SETACTIVE SCHEME_MIN

Changing the power plan takes effect immediately, so you should immediately see your CPUs running at full speed. This means that you can change the power plan on your SQL Server without requiring an outage. If, however, this does not return your CPUs to full speed, then the BIOS is overriding the Windows setting, and you will need to reboot to go into the BIOS to disable it, and once changed reboot again. Different BIOS manufacturers call CPU Throttling / Dynamic Frequency Scaling by different names, so you will need to investigate your server manufacturer’s web site to determine what it is called. This will obviously cause an outage during this time, so this needs to be a scheduled maintenance action.

What kind of difference does the SQL “go fast” button make?

This isn’t the fix for all queries, but it should help out pretty dramatically. For a system replacing another, it should return the application to pre-update performance, if not surpassing it.

Recently, I had the opportunity to test out the performance impact of changing the power plan from Balanced to High Performance. Prior to changing the power plan, a set of queries was running in 7 hours, 45 minutes. After changing just the power plan to High Performance, this set of queries is now running in 3 hours, 55 minutes. That’s almost ½ the time, from this simple setting change.

Just remember… SQL Server is not “green”. It needs full power.

And now you know about SQL Server’s hidden “Go Fast” button.

The post SQL Go Fast button appeared first on SQL Solutions Group.