An indexed view is a view where the result set from the query (the view definition) becomes materialized in lieu of the virtual table result set of a standard (non-indexed) view. Many times we see that an indexed view would be created to help improve performance. Far too often, an indexed view is created without consideration for the costs of the indexed view. In this article, I hope to cover some of the more important costs that are frequently overlooked when considering an indexed view as a performance panacea.

The Setup

To demonstrate the overlooked costs of indexed views, I have decided to use the AdventureWorks2014 database (if you don’t have this sample database, you can get your copy here). I will create a view in that database and then I will add a few indexes to that view. Prior to sharing the entire setup, and like all good DBAs, I need to first grab a baseline. For this baseline, I am doing nothing terribly complex. I will just grab table sizes and storage information for the entire database.

Here are my results for the size of the objects within the AdventureWorks2014 database:

These results show that the largest object in the database happens to be the Person.Person table at just about 30MB. Not terribly large, but the database is not terribly large as a whole. Let’s see what happens when I add a materialized view, based on the Person.Person table, to the mix. Here is the definition of that view along with the indexes that I will add to help the performance of some essential production related queries.

Source code

USE [AdventureWorks2014];
GO
 
CREATE VIEW [Person].[vPerson]
WITH SCHEMABINDING
AS
	SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
			, pp.AdditionalContactInfo
			, pp.Demographics
		FROM [Person].[StateProvince] sp
			INNER JOIN [Person].[CountryRegion] cr
				ON sp.[CountryRegionCode] = cr.[CountryRegionCode]
			INNER JOIN Person.Address pa
				ON sp.StateProvinceID = pa.StateProvinceID
			INNER JOIN Person.BusinessEntityAddress pbe
				ON pbe.AddressID = pa.AddressID
			INNER JOIN Person.Person pp
				ON pp.BusinessEntityID = pbe.BusinessEntityID
		WHERE pbe.AddressTypeID = 2;
 
GO
 
 
CREATE UNIQUE CLUSTERED INDEX CI_vPersonID ON Person.vPerson(BusinessEntityID);
 
CREATE INDEX IX_vPersonName ON Person.vPerson(FirstName,LastName);
CREATE INDEX IX_vPersonState ON Person.vPerson(FirstName,LastName,StateProvinceID,StateProvinceCode,StateProvinceName);

After executing all of that code to create this new view with its indexes, I have the following size results:

The creation of this view has chewed up a bunch of storage. It has jumped right up to the number two spot on the biggest objects list within this database. You can see that differences by comparing the highlighted rows to the previous image. The vPerson view is highlighted in red in this second image to help point it out quickly.

Surely this must be a contrived example and people don’t really do this in the real world, right? The answer to that is simply: NO! It DOES happen. I see situations like this all too often. Far too often, large text fields are added to an indexed view to make retrieval faster. I have mimicked that by adding in two XML columns from the Person.Person table. This is definitely overkill because a simple join back to the table based on the BusinessEntityID would get me those two columns. All I have effectively done is duplicated data being stored and I have achieved that at the low low cost of increased storage of 25% for this small database. If you are curious, the column count between the Person.Person table and this new view is 13 columns each.

I call the 25% increase storage cost a significant increase. An increase of 25% in storage for a single materialized view is not an uncommon occurrence. I have seen multi-terabyte databases with 25% of the total data storage being caused by a single view. If you are tight on storage, any unexpected data volume increase can cause a non-linear growth in data requirements to occur just because of the presence of indexed views. Take that into consideration when presented with an option to either performance tune the code or to take a short-cut and create an indexed view.

Band-aid Performance

I alluded to the use of indexed views as a cog to help improve performance. This is done by taking the query that sucks and turning it into a view. The thought is that it will run faster because an index is created specifically for that data set. Generally the performance gains from using an indexed view, to camouflage the bad query, are seen when the query is complex. Unfortunately, the query I provided for this article is not terribly complex. Due to the simplicity of the first example, let’s look first at the execution plan for another query that runs terribly slow but is also not terribly complex:

There are a few table spools and index scans. The most costly operator of the bunch appears to be a merge join with a many-to-many join operation. The indexed view in this case does give me a very significant improvement on this query (look at that estimated cost and then the estimated number of rows). What if I could tune the query a little and avoid the all of storage cost? Let’s see!

Consider the base query for this new view:

Source code

SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
			, pp.AdditionalContactInfo
			, pp.Demographics
		FROM [Person].[StateProvince] sp 
			INNER JOIN [Person].[CountryRegion] cr
				ON RTRIM(LTRIM(sp.[CountryRegionCode])) = RTRIM(LTRIM(cr.[CountryRegionCode]))
			INNER JOIN Person.Address pa
				ON RTRIM(LTRIM(sp.StateProvinceID)) = RTRIM(LTRIM(pa.StateProvinceID))
			INNER JOIN Person.BusinessEntityAddress pbe
				ON RTRIM(LTRIM(pbe.AddressID)) = RTRIM(LTRIM(pa.AddressID))
			INNER JOIN Person.Person pp
				ON RTRIM(LTRIM(pp.BusinessEntityID)) = RTRIM(LTRIM(pbe.BusinessEntityID))
		WHERE RTRIM(LTRIM(pbe.AddressTypeID)) = 2;

Looking at the query, you may be asking yourself why use so many trim functions on each of the joins? This is an extreme that can be easily fixed. None of the fields in the joins should need to be trimmed since they are all numeric in this case. Starting there and removing all of the function use in the joins should provide significant improvements in query speed and without the use of a view. Testing it out would yield similar performance results to the first view in this article. This small change would cause the query to complete in 1 second compared to about 90 seconds. That is a huge gain, and without the added cost of increased storage use.

Do I really see stuff like this in the field? Yes! Spending a little time to fix the root of the problem rather than looking for a quick fix and performance can be spectacular. There are times, though, that an indexed view may be an absolute requirement. That would be perfectly fine. There are times when it is warranted and there is no way around it. If an indexed view is an absolute must, then there are a couple more considerations to be taken into account.

Disappearing Act

I showed previously that the indexed view requires storage space and showed that the new view created quickly claimed its place as the second largest object within the database. Now, to stage the next segment, I will show once again that the indexes are present and consuming disk space. This can be shown via the following query:

Source code

SELECT o.name AS ObjName
		, i.name AS IdxName
		, SUM(ps.page_count) AS page_count
		, SUM(ps.record_count) AS record_count
		, SUM(ps.page_count) / 128.0 AS SizeMB
	FROM sys.indexes i
		INNER JOIN sys.objects o
			ON i.OBJECT_ID = o.OBJECT_ID
		INNER JOIN sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL,
													'detailed') ps
			ON ps.OBJECT_ID = o.OBJECT_ID
	WHERE o.name LIKE 'vperson%'
	GROUP BY o.name
		, i.name;

And the results of that query should look something like the following (quick reminder that I created a new indexed view for the second demo and these results will be based on that second view):

Now that we know the index is present and consuming space, time to make a change to the view and see what happens. I am proceeding under the premise that I have determined that including the two XML columns in the view are completely unnecessary and are the cause of too much space consumption. I can reap the benefits of the view at a fraction of the cost if I ALTER the view and remove those columns. So I will proceed by issuing the following ALTER statement:

Source code

ALTER VIEW [Person].[vPerson2]
WITH SCHEMABINDING
AS
	SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
		FROM [Person].[StateProvince] sp 
			INNER JOIN [Person].[CountryRegion] cr
				ON RTRIM(LTRIM(sp.[CountryRegionCode])) = RTRIM(LTRIM(cr.[CountryRegionCode]))
			INNER JOIN Person.Address pa
				ON RTRIM(LTRIM(sp.StateProvinceID)) = RTRIM(LTRIM(pa.StateProvinceID))
			INNER JOIN Person.BusinessEntityAddress pbe
				ON RTRIM(LTRIM(pbe.AddressID)) = RTRIM(LTRIM(pa.AddressID))
			INNER JOIN Person.Person pp
				ON RTRIM(LTRIM(pp.BusinessEntityID)) = RTRIM(LTRIM(pbe.BusinessEntityID))
		WHERE RTRIM(LTRIM(pbe.AddressTypeID)) = 2
;
 
GO

Take note of how long this ALTER command takes to complete – nearly instantaneous.  That rocks! Right? Let’s look at the indexes to make sure they are still in working order and don’t require a defrag. Based on that, I will re-run that query to check the indexes and sizes – for giggles.

Source code

SELECT o.name AS ObjName
		, i.name AS IdxName
		, SUM(ps.page_count) AS page_count
		, SUM(ps.record_count) AS record_count
		, SUM(ps.page_count) / 128.0 AS SizeMB
	FROM sys.indexes i
		INNER JOIN sys.objects o
			ON i.OBJECT_ID = o.OBJECT_ID
		INNER JOIN sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL,
													'detailed') ps
			ON ps.OBJECT_ID = o.OBJECT_ID
	WHERE o.name LIKE 'vperson%'
	GROUP BY o.name
		, i.name;

This time when the query completes, I get a nice little surprise: I no longer have any indexes on that view. Changing the view definition instantly drops any indexes that were built on that view. If this is done in a production system, imagine the implications and problems that could arise.

Conclusion

In this article, I have covered indexed views and three different considerations that are often overlooked when looking to implement a materialized view. The three considerations for indexed views, as discussed, are: storage, changes to the view, and the band-aid effect. Overlooking the cost of storage can have serious repercussions on the production environment. Implementing an indexed view without attempting to tune the code first could lead to overlooking the storage cost and could end up just being a waste of resources. The last tidbit is that any change to an indexed view will drop the indexes. That is an easily overlooked feature of indexed views. If you forget to recreate the indexes on the view after making changes, you could be looking at a production outage.

There are a couple more critical considerations for indexed views that are often overlooked as well. These additional considerations are blocking, deadlocking, and maintenance. These considerations will be discussed further in a future article.

The post Indexed Views – Performance Panacea or Plight appeared first on SQL Solutions Group.

One of the frequently required job functions of the database administrator is to track disk space consumption. Whether this requirement comes from management or from a learning opportunity after a production outage, the need exists.

As a hard working DBA, you want to make sure you hit all of the notes to make management sing your praises. Knowing just when the database may fill the drives and prevent a production outage just happens to be one of those sharp notes that could result in a raise and management singing hallelujah. The problem is, how do you do it from within SQL Server? You are just a DBA after all and the disk system is not your domain, right?

Trying to figure it out, you come across a pretty cool function within SQL Server. The name of the function is sys.dm_os_volume_stats. Bonus! This is an excellent discovery, right? Let’s see just how it might work. First a sample query:

Source code

SELECT DISTINCT  ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM sys.master_files mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

If I run that on my local system, I might end up with something that looks like the following:

Looking at the image you may be wondering to yourself right now why I have highlighted a couple of things. You may also be wondering why I used the word “might” in the previous paragraph as well. The reasoning will become more evident as we progress. For now, you have resolved to continue testing the script so execute it again and end up with something that may look like the following (for the same server):

Whoa! What just happened there? Why are there two listings for the C: drive? Why does each register a different value for the FreeSpace column? In addition without any additional usage on the C drive (as verified through other tools) the FreeSpace is changing between executions as well as within the same execution. This is problematic, so you continue testing:

And yet again!

This can’t be correct, can it? Just for giggles let’s modify it just a bit to see if there are any additional clues. Using the following changed script, hopefully a clue will help shed some light on this:

Source code

SELECT ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM sys.master_files mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

This script yields the following potential results:

Look at the different highlighted areas! There are three different values for FreeSpace for the C: drive in this particular execution. The case of the C: drive plays no role in whether the value is recorded differently or not. This seems to be more of a bug within the dynamic management function. From execution to execution, using this particular method, one could end up with duplicate entries but distinct values. The sort of the execution could be returned differently (though we could fix that).

All of these tests were run on my local machine and I really do only have one C: drive. I should never receive multiple entries back for any drive. If using this particular DMF to track space usage, it could be somewhat problematic if the duplicate drive data pops up. How do we get around it, you ask? Here is another example that I have that has not yet produced this duplication:

Source code

WITH presel AS (SELECT database_id, FILE_ID,LEFT(mf1.physical_name,3) AS Volume, ROW_NUMBER() OVER (PARTITION BY LEFT(mf1.physical_name,3) ORDER BY mf1.database_id) AS RowNum
					FROM sys.master_files mf1)
,roundtwo AS (SELECT DISTINCT pr.database_id, pr.FILE_ID
				FROM presel pr
				WHERE pr.RowNum = 1)
 
SELECT ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM roundtwo mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

Using this version of the script is not terribly more complex, but it will prove to be more reliable. You can see I used some CTEs to provide a little trickery and ensure that I limit my results. What if it is a mount point or a non-standard drive letter? I have not tested that. Let me know how that goes. As you can see, I am restricting the drive selection by using the row_number function against the drive letter.

For alternative reliable methods to find your disk space consumption, I would recommend something different because it is tried and tested. I would recommend using a wmi call to fetch the data. Samples are provided as follows:

Source code

/* for mount points, something like this */
EXECUTE sys.xp_cmdshell 'wmic volume get name, freespace, capacity, label'
/* the base wmi query that does not support mount points */
EXECUTE xp_cmdshell 'wmic logicaldisk get name,freespace,size,volumename,blocksize'

Easy peasy, right? Have at it and try tracking your disk space.

The post Disk Space and SQL Server appeared first on SQL Solutions Group.

We often pontificate about the what-ifs for a black box recorder within SQL Server, right? We wish and dream and go about creating our own little recorder to suit the requirements we deem necessary for successful monitoring of the server for the just-in-case scenario. Well, the truth of the matter is that we really don’t need to go to such great lengths to create a “black box recorder” because Microsoft has already done all of that for us.

Wait, what? Yes, that’s right! Truth be told this shouldn’t be much news for most Senior Level data professionals working with SQL Server for the past few years. But if you are new to the product, this might be good news for you. Then again, it might just be bad news depending on your take.

Very much like what you may find with the airline industry, the black box recorder in SQL Server is not just a single method (device) implemented to capture all of the desired data points. On a passenger jet, you may find that there are three or more such devices that contribute to the capture and recording of the in-flight data. In SQL Server, there are three major processes that help capture our in-flight data. Over the next few articles I will discuss each of these processes. These processes include:

• Default Trace
• system_health Extended Event Session
• sp_server_diagnostics procedure

This multi-tiered approach does provide a sort of fail-safe system. If one should be disabled, there still remain up to two more processes that may be running. That said, each of these can be disabled but it does take a bit of an overt effort. And since it does require somebody to put out effort to try and disable each of the black box components, you could potentially capture the culprit via audits from the other components or via an explicit audit that you create for these types of purposes.

Since this will be something to discuss over a few articles, I will break out each process into an individual article. For today, I will discuss the default trace.

Default Trace

The default trace by itself is something that can be turned off via configuration option. There may be good reason to disable the default trace. Before disabling the default trace, please consider the following that can be captured via the default trace. I will use a query to demonstrate the events and categories that are configured for capture in the default trace.

Source code

/* events */
SELECT te.name AS EventName
		,tca.name AS CategoryName
		, CASE tca.type WHEN '0' THEN 'Normal'
				WHEN '1' THEN 'Connection'
				WHEN '2' THEN 'ERROR' END AS CategoryType
		, t.PATH AS TracePath
		, oa.logical_operator,oa.comparison_operator, oa.VALUE AS FilteredValue
	FROM sys.traces t
		CROSS APPLY (SELECT DISTINCT gei.eventid FROM sys.FN_TRACE_GETEVENTINFO(t.id) gei) ca
		INNER JOIN sys.trace_events te
			ON te.trace_event_id = ca.eventid
		INNER JOIN sys.trace_categories tca
			ON te.category_id = tca.category_id
		OUTER APPLY (SELECT gfi.columnid,gfi.logical_operator,gfi.comparison_operator,gfi.VALUE FROM sys.FN_TRACE_GETFILTERINFO(t.id) gfi) oa
	WHERE t.is_default = 1

In this query, I have requested a few more data points than necessary to illustrate the point. That is to help illustrate an additional point that the default trace isn’t filtering out any data for these events. If the event fires (in this trace), it is recorded. Let’s divert right back to the events and categories for now. The results of that query will produce the following sample list of events for me on SQL Server 2014:

That is quite a range of events covered by this particular recording device. From changes to objects down to security related events and even errors and warnings. If somebody drops an object, the default trace can catch it. If one of the various DBCC statements is executed, it will be trapped in this trace.

There is one thing that is captured by this trace that is not overly obvious. In fact, it is rather annoying in how it is recorded in my opinion. Server configuration settings such as “cost threshold of parallelism” or the lesser utilized “user options” are not registered as an “Object:Alter” event but rather as an ErrorLog event. I wouldn’t call this type of “change” an error and don’t necessarily like seeing these registered with other legitimate errors. That said, it would be nice to see these logged differently (they are currently logged the same way with Extended Events). So, if somebody is dorking around with server configurations, it becomes a bit more convoluted to figure it out, but we can get there. Let’s see how that works:

Source code

-- server configs Audited via Def Trace
SELECT 	T.StartTime
		, T.SPID
		, T.LoginName
		, T.HostName
		, T.ApplicationName
		, T.DatabaseName
		--, ObjectName,sv.number AS ObjTypeVal, sv.name [ObjectType]
		--, T.TextData
		, ConfigOption = SUBSTRING(T.TextData,CHARINDEX('''',T.TextData)+1,CHARINDEX(' changed from ',T.TextData)-CHARINDEX('''',T.TextData)-2)
		, PrevValue = SUBSTRING(T.TextData,CHARINDEX('from ',T.TextData)+5,CHARINDEX('to ',T.TextData)-CHARINDEX('from ',T.TextData)-5)
		, NewValue = SUBSTRING(T.TextData,CHARINDEX('to ',T.TextData)+3,CHARINDEX('. Run',T.TextData)-CHARINDEX('to ',T.TextData)-3)
		, EventName = te.name
		, T.EventClass
	FROM sys.FN_TRACE_GETTABLE(CONVERT(VARCHAR(150), 
			( SELECT REVERSE(SUBSTRING(REVERSE(PATH),
					CHARINDEX('\',REVERSE(path)),256)) + 'LOG.trc'
				FROM    sys.traces
				WHERE   is_default = 1)), DEFAULT) AS T  
		INNER JOIN sys.trace_events AS te
			ON T.EventClass = te.trace_event_id
	WHERE T.EventClass = 22
		AND T.TextData LIKE '%config%'
	ORDER BY T.StartTime DESC;

Looking at my system, I get the following sample results:

You can see there are plenty of settings that I have been playing with captured by the default trace. If you happen to have a junior DBA or maybe a cowboy DBA that loves to shoot from the hip and make changes, this will help you find all the necessary details to bring to that DBA for discussion.

The default trace comprises one third of the black box recorder. I have shown a quick use for the default trace and have also shown some of the various events that are captured from this trace. I recommend getting to know your default trace just a little bit better. You never know when you may need to resort to the flight recorder data held in the default trace. Being familiar with the default trace before you need to use it will help improve your comfort level when under the stress of trying to figure out what happened just before the server went belly up!

The post SQL Black Box Recorder appeared first on SQL Solutions Group.

Not only may it be very commonplace to experience this due to the nature of the job, but it can also get to feel that way when a new product is released. Suddenly, that icy island is further north into the Arctic Ocean and you are struggling to gasp for a bit of air in the frigid cold.

I am certain this may be some of what you feel as you try to take a look at Extended Events. It is a fancy, new technology and there is not a ton of time to get up to speed while trying to warm yourself against the development chill. As you stew about the situation, you begin to ponder, “How am I even supposed to know what an XE is, let alone what XEs are running on my server?” Never mind the confusion on what an XE is and that it is actually running or deployed sessions on the server. Don’t worry about the terminology just yet. What can we do to figure out what might be out there?

Well, that is where the trusty Google has come to help you. Thanks to a better than adequate ability to search the Internet, you landed on this page that has a script to help you explore what might be running as far as Extended Events on your server. You may be surprised to learn that you have more Extended Events Sessions running than you fathomed. Yes, SQL Server comes with some default sessions. Yes, some of these default sessions are more public than others. And yes, some of these

Come on in out of the cold. Get off that arctic island and warm up with a little XE fun. This script is quite simple but will show you what you may have out there on your servers. Once you see what is there, hopefully your curiosity will get a little piqued and you will want to learn a little more (there is more of that here on this site too).

And now, for that script:

Source code

DECLARE @SessionName VARCHAR(128) = NULL --'sp_server_diagnostics session' --NULL for all
;
 
SELECT ISNULL(ses.name,xse.name) AS SessionName
		, CASE
			WHEN ISNULL(ses.name,'') = ''
			THEN 'Private'
			ELSE 'Public'
			END AS SessionVisibility
		, CASE
			WHEN ISNULL(xse.name,'') = ''
			THEN 'NO'
			ELSE 'YES'
			END AS SessionRunning
		, CASE
			WHEN ISNULL(xse.name,'') = ''
				AND ISNULL(ses.name,'') = ''
			THEN 'NO'
			ELSE 'YES'
			END AS IsDeployed
	FROM sys.server_event_sessions ses
	FULL OUTER JOIN sys.dm_xe_sessions xse
		ON xse.name =ses.name
	WHERE COALESCE(@SessionName, ses.name, xse.name) = ISNULL(ses.name, xse.name)
	ORDER BY ses.event_session_id;
 
GO

Quick and simple, and full of that warming XE sensation. This script just shows you what is a private session versus public. What is a private session, you ask? Well, that happens to be all of those database audits you have deployed to your server. It also includes those little things like the “sp_server_diagnostics session” session (yes it sounds redundant but that is because session is also part of the name for that session).

This script will also show you what is deployed to the server versus what is running on the server (not all sessions have to be running). And to top it all off, I set it up so you can query for a specific session or for all sessions.

Try it out on one or more of your servers and let me know how it goes.

The post What Sessions are Deployed? appeared first on SQL Solutions Group.

Well, for whatever reason, there seems to be a myth out there that when deadlocks come a-knocking, then just throw this little directive at it and all will be well. Today, we get to test that and show what will continue to happen.

First, lets look at what the ROWLOCK actually means:

Specifies that row locks are taken when page or table locks are ordinarily taken.

This seems like a fabulous idea if the deadlocks are occurring against a table involving a massive update. Let’s take a look at a small update scenario involving just a handful of records.

Source code

USE tempdb;
GO
IF OBJECT_ID('dbo.Test1') IS NOT NULL
	DROP TABLE dbo.Test1;
IF OBJECT_ID('dbo.Test2') IS NOT NULL
	DROP TABLE dbo.Test2;
CREATE TABLE dbo.Test1 ( col1 INT );
CREATE TABLE dbo.Test2 ( col2 INT );
INSERT INTO dbo.Test1
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
INSERT INTO dbo.Test2
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
GO
BEGIN TRANSACTION;
UPDATE dbo.Test1
	SET	col1 = col1 * 10
	WHERE col1 = 3;
 
/* query window 2*/
 
USE tempdb;
BEGIN TRANSACTION;
UPDATE dbo.Test2
	SET	col2 = col2 * 20
	WHERE col2 = 4;
UPDATE dbo.Test1
	SET	col1 = col1 * 20
	WHERE col1 = 3;
COMMIT TRANSACTION;
 
/* back to the first query window*/
UPDATE dbo.Test2
	SET	col2 = col2 * 10
	WHERE col2 = 4;
COMMIT TRANSACTION;

Looking at the code, we can see there are only five rows in each of the tables. In addition, an update will be performed to both col1 and col2 in each table for one specific row. So we are keeping this to a singleton type of update, and we are able to force a deadlock by using this setup. Not only do we see that a deadlock will occur consistently, we would see the following in the sys.dm_tran_locks DMV as well as in the deadlock graphs.

In my environment, I used session 51 and 54 consistently for the deadlock repro. In each of the tests, each spid did obtain page locks as well as RID locks (row locks) that were waiting on each other in order to perform an Update. This is what we saw when I ran the setup without the ROWLOCK directive. What if I used the ROWLOCK directive (are you guessing already that there would be no change because the deadlock occurs on the lock held for the update that is waiting on the other update to complete?)? Let’s take a look at that too!

Source code

/* what about using rowlock hint? let's try it */
USE tempdb;
GO
IF OBJECT_ID('dbo.Test1') IS NOT NULL
	DROP TABLE dbo.Test1;
IF OBJECT_ID('dbo.Test2') IS NOT NULL
	DROP TABLE dbo.Test2;
CREATE TABLE dbo.Test1 ( col1 INT );
CREATE TABLE dbo.Test2 ( col2 INT );
INSERT INTO dbo.Test1
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
INSERT INTO dbo.Test2
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
GO
BEGIN TRANSACTION;
UPDATE dbo.Test1 WITH (ROWLOCK)
	SET	col1 = col1 * 10
	WHERE col1 = 3;
 
/* query window 2*/
 
USE tempdb;
BEGIN TRANSACTION;
UPDATE dbo.Test2 WITH (ROWLOCK)
	SET	col2 = col2 * 20
	WHERE col2 = 4;
UPDATE dbo.Test1 WITH (ROWLOCK)
	SET	col1 = col1 * 20
	WHERE col1 = 3;
COMMIT TRANSACTION;
 
/* back to the first query window*/
UPDATE dbo.Test2 WITH (ROWLOCK)
	SET	col2 = col2 * 10
	WHERE col2 = 4;
COMMIT TRANSACTION;

The only change to this setup is that the ROWLOCK directive has been added to the update statements. Examining the sys.dm_tran_locks DMV reveals the same locks being held as was seen without the directive. This shouldn’t be too big of a surprise since the updates are against a single row.

In addition to the same locks being held, we continue to experience the same deadlock problem. Using an Extended Events session to trap deadlock information (similar to the article previously discussed), we can pull out some pretty good info. Let’s examine some of the deadlock data trapped by an XE session.

Source code

WITH	cte	AS ( SELECT t2.event_data.VALUE('(event/@name)[1]', 'varchar(50)') AS event_name
							, t2.event_data.VALUE('(event/@timestamp)[1]',
													'datetime2') AS StartTime
							, t2.event_data.VALUE('(event/data[@name="duration"]/value)[1]',
													'bigint') AS duration
							, t2.event_data.VALUE('(event/data[@name="database_name"]/value)[1]',
													'sysname') AS DBName
							, t2.event_data.VALUE('(event/action[@name="sql_text"]/value)[1]',
													'varchar(max)') AS sql_text
							, t2.event_data.VALUE('(event/action[@name="nt_username"]/value)[1]',
													'varchar(500)') AS nt_username
							, t2.event_data.VALUE('(event/data[@name="mode"]/value)[1]',
													'varchar(15)') + ' ('
							+ t2.event_data.VALUE('(event/data[@name="mode"]/text)[1]',
													'varchar(50)') + ')' AS mode
							, t2.event_data.VALUE('(event/data[@name="object_id"]/value)[1]',
													'integer') AS OBJECT_ID
							, t2.event_data.VALUE('(event/data[@name="resource_description"]/value)[1]',
													'varchar(max)') AS resource_description
							, t2.event_data.VALUE('(event/data[@name="resource_owner_type"]/text)[1]',
													'varchar(max)') AS resource_owner_type
							, t2.event_data.VALUE('(event/data[@name="resource_type"]/text)[1]',
													'varchar(max)') + ' ('
							+ t2.event_data.VALUE('(event/data[@name="resource_type"]/value)[1]',
													'varchar(max)') + ')' AS resource_type
							, t2.event_data.VALUE('(event/action[@name="server_principal_name"]/value)[1]',
													'varchar(max)') AS server_principal_name
							, t2.event_data.VALUE('(event/action[@name="session_id"]/value)[1]',
													'varchar(max)') AS session_id
							, t2.event_data.VALUE('(event/action[@name="client_pid"]/value)[1]',
													'integer') AS client_pid
							, t2.event_data.VALUE('(event/action[@name="client_app_name"]/value)[1]',
													'varchar(max)') AS client_app_name
							, t2.event_data
							,ROW_NUMBER() OVER (ORDER BY t2.event_data.VALUE('(event/@timestamp)[1]',
													'datetime2')
													,t2.event_data.VALUE('(event/@name)[1]', 'varchar(50)')) AS Event_Seq
						FROM sys.fn_xe_file_target_read_file('C:\Database\XE\Deadlock*.xel',
																NULL, NULL, NULL) t1
							CROSS APPLY ( SELECT CONVERT(XML, t1.event_data)
										) t2 ( event_data )
				)
	SELECT c1.event_name AS LockEvent, c2.event_name AS GraphEvent
			, c1.StartTime
			, DATEADD(MICROSECOND, c1.duration, CONVERT(DATETIME2, [c1].StartTime)) AS EndDate
			, c1.duration
			, c1.DBName
			, c1.sql_text
			, c1.nt_username
			, c1.server_principal_name
			, c1.mode
			, c1.OBJECT_ID
			, c1.resource_description
			, c1.resource_owner_type
			, c1.resource_type
			, c1.session_id
			, c1.client_pid
			, c1.client_app_name
			, c1.event_data
			, graph.query('.') AS EventDeadlockGraph
		FROM cte c1
			INNER JOIN cte c2
				ON c1.Event_Seq = c2.Event_Seq - 1
			CROSS APPLY c2.event_data.nodes('(event/data[@name="xml_report"]/value)[last()]/*')
				AS deadlock ( graph )
		WHERE c1.event_name = 'lock_deadlock'
			AND c2.event_name = 'xml_deadlock_report'
		ORDER BY c1.StartTime;

The results from this query will show us the deadlock graph, the event data, as well as several other pieces of data already parsed from the session data for you. And looking at the session data, one can see that the sql_text from each of the queries will demonstrate both the ROWLOCK directive and the directive-free versions of the query. In this query you can also see that I did a little black magic to match up the two event types from the event session (lock_deadlock and xml_deadlock_report). Then I was able to join the two together to produce one row per deadlock event and to see the sql_text with the deadlock graph on one row. Otherwise, the sql_text does not produce with the deadlock_report event. I leave the rest of the query to the reader to discover and question.

From the EventDeadlockGraph column, we could click the cell and take a close look at the XML generated for the deadlock event. Further, if I choose to save the xml as an XDL file and then reopen it in SSMS, I can see the deadlock graphical report as shown in the following.

We see that row locks are still in effect for the update coming from both sides. This further supports that the directive really is just a waste of time in trying to combat this type of deadlock. This is one of those cases where the best option would be to optimize the code and work things out without trying to take a shortcut.

Wrapping Up

Look to optimize the code instead of trying to take a shortcut. In addition, take a look at the deadlocks, the locks held, and the code to get a better understanding of what is truly happening.

The post An Experiment with Deadlocks appeared first on SQL Solutions Group.

In that article, I showed one common use-case for SQL Server Profiler and how to achieve the same result via Extended Events. What I didn’t show in that article was how to correlate all of your favorite trace/profiler options into the shiny new XE equivalents. Is there even a way to do that?

As luck would have it, there is a means to correlate trace/profiler events to Extended Event events (yeah that sounds weird).  With the release of SQL Server 2012, Microsoft introduced a couple of catalog views to help with this correlation effort. Those views have a pretty straightforward naming convention. For example, one of the views is named sys.trace_xe_event_map while the other is sys.trace_xe_action_map. For this article, I will be focusing on the former.

Making the Upgrade

When looking to finally make the upgrade away from trace/profiler, a big key is to figure out if your favorite trace events are even an option at the next level. I have talked about finding events in XE in previous articles by trying to search for a specific topic that might apply to your current condition.

But if you are already familiar with specific trace events, you may just want/need to know what the new name is in XE. This is where those catalog views come into play. And in support of that, here is a query that can help in that upgrade effort:

Source code

SELECT te.trace_event_id,tc.name AS CategoryName
	, CASE tc.type
		WHEN 0 THEN 'Normal'
		WHEN 1 THEN 'Connection'
		WHEN 2 THEN 'ERROR'
		END AS TraceEventType
	,te.name AS TraceEventName
	, xem.package_name, xem.xe_event_name
FROM sys.trace_xe_event_map xem
	RIGHT OUTER JOIN sys.trace_events te
		ON te.trace_event_id = xem.trace_event_id
	INNER JOIN sys.trace_categories tc
		ON te.category_id = tc.category_id
ORDER BY tc.category_id,te.trace_event_id ASC;

Looking through the results, one would notice that not every trace event maps to an Extended Event event. The events that don’t map, in general, deal with audits (which is actually driven by Extended Events).

This is a really good start to getting you on your way to that much needed upgrade away from profiler/trace. What if there are several traces that are already in use or scripted for various reasons in the environment?

Decision Time

If you happen to have traces that are already deployed in the environment or in script form, it can be a bit of a tedious pain to convert those to Extended Events. Do you manually recreate the traces as XE sessions? Do you abandon the upgrade due to the effort and annoyance it will create? Or do you find some automated means of performing the conversion?

Of those three options, only two are valid options. Those options involve performing the conversion of the traces to XE sessions. There are pros and cons for each. You may opt to take your time and learn more about Extended Events by performing the manual upgrade, or you may choose to save time by using an automated routine.

Should you decide to try the automated routine, there is one already out and available to help you on your way. Jonathan Kehayias wrote the script and you can download it here.

The post Upgrading From SQL Server Profiler appeared first on SQL Solutions Group.

What do you do when you run into that missing database situation and the inevitable denial that will ensue?  This is when an audit can save the day.  Through an audit, you can discover who dropped the database and when it happened.  Then you have hard data to take back to the team to again ask what happened.  Taking the info from a previous article of mine, we can alter the script I published there and re-use it for our needs here.

Source code

DECLARE @DBName sysname = 'AdventureWorks2014'
		,@d1 DATETIME
		,@diff INT;
 
SELECT ObjectName
		  , ObjectID
		  , DatabaseName
		  , StartTime
		  , EventClass
		  , EventSubClass
		  , ObjectType
		  , ServerName
		  , LoginName
		  , NTUserName
		  , ApplicationName
		  , CASE EventClass
				WHEN 46
					THEN 'CREATE'
				WHEN 47
					THEN 'DROP'
				--WHEN 164
				--	THEN 'ALTER'
			END AS DDLOperation
		INTO #temp_trace  
	FROM sys.FN_TRACE_GETTABLE(CONVERT(VARCHAR(150), 
			( SELECT REVERSE(SUBSTRING(REVERSE(PATH),
					CHARINDEX('\',REVERSE(path)),256)) + 'LOG.trc'
				FROM    sys.traces
				WHERE   is_default = 1)), DEFAULT) T  
  WHERE EventClass in (46,47) 
			AND EventSubclass = 0
			AND ObjectType = 16964-- i just want database related events   
			AND DatabaseName = ISNULL(@DBName,DatabaseName);
 
 
SELECT @d1 = MIN(StartTime) 
	FROM #temp_trace;
 
SET @diff= DATEDIFF(hh,@d1,GETDATE());
 
 
SELECT @diff AS HrsSinceFirstChange    
		, @d1 AS FirstChangeDate    
		, sv.name AS obj_type_desc
		, tt.ObjectType
		, tt.DDLOperation
		, tt.DatabaseName,tt.ObjectName,tt.StartTime
		, tt.EventClass,tt.EventSubClass
		, tt.ServerName,tt.LoginName, tt.NTUserName
		, tt.ApplicationName
		, (dense_rank() OVER (ORDER BY ObjectName,ObjectType ) )%2 AS l1     
		, (dense_rank() OVER (ORDER BY ObjectName,ObjectType,StartTime ))%2 AS l2    
	FROM #temp_trace tt
		INNER JOIN sys.trace_events AS te 
			ON tt.EventClass = te.trace_event_id
		INNER JOIN sys.trace_subclass_values tsv
			ON tt.EventClass = tsv.trace_event_id
			AND tt.ObjectType = tsv.subclass_value
		INNER JOIN master.dbo.spt_values sv 
			ON tsv.subclass_value = sv.number
			AND sv.type = 'EOD'
	ORDER BY StartTime DESC;
 
DROP TABLE #temp_trace;

This script will now query the default trace to determine when a database was dropped or created.  I am limiting this result set through the use of this filter: ObjectType = 16964.  In addition to that, I have also trimmed the result-set down to just look for drop or create events.

This is the type of information that is already available within the default trace.  What if you wished to not be entirely dependent on the default trace for that information?  As luck would have it, you don’t need to be solely dependent on the default trace.  Instead you can use the robust tool called extended events.

The post Where did the Database Go? appeared first on SQL Solutions Group.

Today, I would like to share a short tutorial on how to use extended events as if it were a profiling session.  I am only going to show one of the many possible means to profile via extended events.  Just understand that this means you have flexibility and a way to do more with this simple example than what I am going to share.

Requirement

You need to capture T-SQL queries that are occurring against your instance in order to try and determine the code that is being passed to SQL Server from the application.

XEvents How-To

Rather than jump to Profiler, you are going to need to jump to a fresh, clean query window in SSMS.  Just like in Profiler, you will want to capture certain “statement” oriented events.

For this session, I want to start with sp_statement_starting and sql_statement_starting.  I am not going to use the _completed forms of those events because I want to capture as much as I can – without capturing too much noise.  Sometimes, there may be a _starting without a coordinated _completed and that could throw a wrench in the works.

With the basic information in hand, we are ready to implement some of the things learned in previous XEvents posts.  Let’s go ahead and start setting up the session that will capture the information we seek.

Source code

USE master;
GO
-- Create the Event Session
IF EXISTS(SELECT * 
          FROM sys.server_event_sessions 
          WHERE name='TraceIncomingQueries')
    DROP EVENT SESSION TraceIncomingQueries 
    ON SERVER;
GO
CREATE EVENT SESSION TraceIncomingQueries
ON SERVER
ADD EVENT sqlserver.sp_statement_starting (
	SET collect_object_name=(1)
	ACTION(sqlserver.database_name,sqlserver.nt_username,sqlserver.session_id,sqlserver.client_hostname,sqlserver.client_app_name)
WHERE sqlserver.client_app_name <> 'Microsoft SQL Server Management Studio - Transact-SQL IntelliSense'
	),
ADD EVENT sqlserver.sql_statement_starting(
    ACTION(sqlserver.database_name,sqlserver.nt_username,sqlserver.session_id,sqlserver.client_hostname,sqlserver.client_app_name)
WHERE sqlserver.database_name='AdventureWorks2014'
	AND sqlserver.client_app_name <> 'Microsoft SQL Server Management Studio - Transact-SQL IntelliSense'
	)
ADD TARGET package0.event_file(SET filename=N'C:\Database\XE\TraceIncomingQueries.xel')
 
/* start the session */
ALTER EVENT SESSION TraceIncomingQueries 
ON SERVER 
STATE = START;
GO

One caveat to consider is in the sp_statement_starting event, I have specified a specific set option be used.

Source code

SET collect_object_name=(1)

This particular option is disabled by default.  If you want to know the object_name of code that is being used (triggers, stored procedures, etc), then this option must be enabled.

I have specified the same actions to be used for each of the events I am trapping, but this is purely up to you.  If you do not wish to capture these actions, then you can remove them or add different actions as necessary.  I have also specified a different predicate for each of the events.  This is something that works so much better in XEvents than Profiler – the filtering that can be applied is much more flexible.

Once the session is started, then it is merely a matter of figuring out how to look at the data.  I generally use T-SQL to parse the data from the file, but you could also fiddle with the GUI as well.  You can access that by right-clicking the event session in management studio and then selecting “Watch Live Data”.  If everything is configured properly, then you will start to see data after the first event is triggered.  From within the GUI, you can pick and choose which columns to display in your view.  You can pause the display of the session data, or you can even “erase” the data from the current view (just like in Profiler).  You can even filter, aggregate or group the data in the view from the GUI.

The use of the GUI is up to you.  I still prefer to use the script route.  With that route, here is a sample of what you may need to write in order to display the data from the session data file that has been captured.

Source code

USE master;
GO
 
SELECT
event_data.VALUE('(event/@name)[1]', 'varchar(50)') AS event_name,
    event_data.VALUE('(event/@timestamp)[1]', 'varchar(50)') AS [TIMESTAMP],
	event_data.VALUE('(event/action[@name="database_name"]/value)[1]', 'varchar(max)') AS DBName
	,event_data.VALUE('(event/data[@name="statement"]/value)[1]', 'varchar(max)') AS SQLText
	,event_data.VALUE('(event/data[@name="object_name"]/value)[1]', 'varchar(max)') AS ObjName
	,event_data.VALUE('(event/action[@name="session_id"]/value)[1]', 'varchar(max)') AS SessionID
	,event_data.VALUE('(event/action[@name="nt_username"]/value)[1]', 'varchar(max)') AS ExecUser
	,event_data.VALUE('(event/action[@name="client_hostname"]/value)[1]', 'varchar(max)') AS Client_HostName,
	event_data.VALUE('(event/action[@name="client_app_name"]/value)[1]', 'varchar(max)') AS Client_AppName
FROM(
SELECT CONVERT(XML, t2.event_data) AS event_data
 FROM (
  SELECT target_data = CONVERT(XML, target_data)
   FROM sys.dm_xe_session_targets t
    INNER JOIN sys.dm_xe_sessions s 
        ON t.event_session_address = s.address
   WHERE t.target_name = 'event_file'
    AND s.name = 'TraceIncomingQueries') cte1
   CROSS APPLY cte1.target_data.nodes('//EventFileTarget/File') FileEvent(FileTarget)
   CROSS APPLY  sys.fn_xe_file_target_read_file(FileEvent.FileTarget.VALUE('@name', 'varchar(1000)'), NULL, NULL, NULL) t2)
    AS evts(event_data);

It really is that simple.  Now you have a means to run a Profiler-like trace on your server without the impact of Profiler.  XEvents provides the means to run a more evolved Profiler session on your data to capture things like queries from an application. XEvents does all of this without the severe penalty of Profiler and with many additional bonuses.  Try it out and enjoy!

The post How To: XEvents as Profiler appeared first on SQL Solutions Group.