Easy Permissions Audit

As the article title denotes, today I will be discussing a simple way to get quick permissions for various principals. If you are looking for a more comprehensive and human friendly report version, I recommend reading any of my many other articles on the topic such as the following article – here or here. In the second of those links there is a clue as to what tool we will be using in this easy version of the audit. That tool is called sp_helprotect.

The stored procedure sp_helprotect is a system stored procedure from Microsoft that can help divulge permissions for various principals in a simple table result set for you. Bearing in mind that I am keeping this to a simple audit, the examples will be simplistic in nature. Without further ado, here is the easy audit for your permissions.

sp_helprotect

This stored procedure was introduced in SQL Server 2008 and comes with a few parameters to help narrow the results down to a specific principal and even to any object to which that principal may have been granted permissions. Here are those parameters for quick review:

@name = This parameter is to filter your request down to a specific object or a statement that can be executed against that object (e.g. alter, create, drop)

@username = Is the name of the principal for which permissions are returned.

@grantorname = Is the name of the principal that granted permissions.

@permissionarea = This is the group of grant-able permissions. There are two types of groups: object and statement. The default setting here is to return both groups.

The easiest way to use sp_helprotect is as follows:

Source code

USE TestDB; --my test database...you need to change it
GO
 
EXECUTE sys.SP_HELPROTECT

Do you see how easy that is? This returns the following results for me.

Note from the results that I see results for roles and users for various different objects. This is due to how the procedure was executed – with no parameters. Using no parameters in this query, the default behavior is to return as much information as possible for all objects and principals within the database.

What if I only want the results for a principal named “Gargouille”? I can do that in the following way.

Source code

USE TestDB; --my test database...you need to change it
GO
 
EXECUTE sys.SP_HELPROTECT 
	@username = 'gargouille'

Now, I will receive the following results:

Recap

There are many ways to produce an audit. Today, I have shown how one can produce a permissions audit when in a hurry that will produce a simple result set for database permissions. I want to underscore that this was at the database level and not the server level. While this is an adequate means for a quick peek into some of the objects and granted permissions, I do recommend using one of the other methods I have introduced in the other articles for ongoing complex audits and results that are somewhat prettier and more human friendly to read.

The post Easy Permissions Audit appeared first on SQL Solutions Group.

The first article went into deeper detail about the first leg of this three-legged black box recorder—the default trace. In the second article, the focus was directed to the next leg of the black box recorder—or the system_health Extended Event Session. If by some measure you are in the dark about what Extended Events is, then I recommend you read my personal blog series on the subject. There are numerous articles stepping through the subject in easy-to-digest fashion. You can find many of the articles in the following table of contents—here.

The focus of this article will take on the topic of the third piece of the black box recorder, sp_server_diagnostics. I will show how this procedure is critical in gathering pertinent information that is crucial to gathering an overall picture of the system—from a default black box recorder perspective.

SQL Black Box Recorder

What is the sp_server_diagnostics process?

Beyond being a component of the black box for SQL Server, what exactly is this diagnostics process? The sp_server_diagnostics is much as the name implies—it is a “diagnostics” service that attempts to gather information about various events that may affect the overall health of the SQL Server instance.

The diagnostics process will trap various server related health (diagnostics) information related to the SQL Server instance in an effort to try and detect potential failures and errors. This diagnostics session/process traps information for five different categories by default. There is a sixth category of information for those special servers that happen to be running an Availability Group.

The Six

• system: overall health of the server including information about spinlocks, severe processing conditions, non-yielding tasks, page faults, and CPU usage.
• resource: health of resources, specifically including physical and virtual memory, buffer pools, pages, cache and other memory objects
• query_processing: captures query processing data about worker threads, tasks, wait types, CPU intensive sessions, and blocking tasks.
• io_subsystem: health related data to IO.
• events: errors and events of interest recorded by the server, including details about ring buffer exceptions, ring buffer events about memory broker, out of memory, scheduler monitor, buffer pool, spinlocks, security, and connectivity.
• AvailabilityGroup: basic information about the AvailabilityGroup if there is an AvailabilityGroup enabled on the server.

This is fantastic introductory information. As luck would have it, you can also read more about it from my blog – here.

How to use this process?

What good is any level of information if we don’t quite know how to use or consume that data? Do we consume it via trace? Do we execute the stored procedure? Do we capture the information from some other means? The answer is yes! Wait, yes to what? Yes to all of it! Let’s explore how to consume this data.

Option 1

The first step to learning how to consume this data is to start at probably the most basic of levels. As you have already noted, this is a stored procedure. It was probably the sp_ in the name that gave it away, right? Since it comes in the form of a stored procedure, it makes it pretty easy to figure out the first means to get the data from it—just execute it. Actually, let me clarify that just a bit. sp_server_diagnostics is a CLR stored procedure. Nonetheless, all that needs be done is execute it at this level to get to the data.

Source code

EXECUTE sp_server_diagnostics;

Look at that! Pretty slick! I execute the stored procedure and I can see a point in time result of the “health check” from this component of the black box recorder. Notice here that I only have five of the six categories that I had mentioned. This is due to the lack of a configured availability group (AlwaysOn) from this particular instance. Another point of interest is that the data column is not in a very friendly format for DBA consumption. This is flat text that just goes on and on. It is painful to consume in this manner.

Tidy that Code

Source code

CREATE TABLE #ServerDiagnosticsResult
	(
		create_time DATETIME
		, component_type sysname
		, component_name sysname
		, STATE INT
		, state_desc sysname
		, DATA XML
	);  
INSERT INTO #ServerDiagnosticsResult
		EXECUTE sp_server_diagnostics;  
 
SELECT *
	FROM #ServerDiagnosticsResult;
 
DROP TABLE #ServerDiagnosticsResult;
CREATE TABLE #ServerDiagnosticsResult
	(
		create_time DATETIME
		, component_type sysname
		, component_name sysname
		, STATE INT
		, state_desc sysname
		, DATA XML
	);  
INSERT INTO #ServerDiagnosticsResult
		EXECUTE sp_server_diagnostics;  
 
SELECT *
	FROM #ServerDiagnosticsResult;
 
DROP TABLE #ServerDiagnosticsResult;

Looking at the code, you can see quickly the difference. I have changed the data column to be XML-based to better match the data being output from the stored procedure. Let’s see that in action:

That is much better. Now I can simply click one of the data results and see a nicely formatted easy to read XML output of that component. Here is an example from the query_processing component:

Option 2

Cool! I have just shown one means of exploring this facet of the black box recorder. Now, let’s take a look at other means to get at this particular data, beyond just executing the stored procedure. Did you know that this procedure is very closely related to Extended Events? (I am sure that you picked up on that nugget due to reading the previous article in the series or possibly the other article I mentioned here). With such a close relationship to Extended Events, it makes sense that sp_server_diagnostics also has a couple of events and is consumed by multiple sessions.

SQL Server comes with various default Extended Event sessions, of those sp_server_diagnostics is consumed by the system_health session as well as the hidden session for SQLDiag that is used by Windows Clustering and Availability Groups to determine server health. You already heard about the system_health (from the previous article), but few have heard about the hidden session. I will take a look at the events as well as the sessions that consume the sp_server_diagnostics data.

Events

First, I will share the details for each of the events related to the sp_server_diagnostics process.

As I have shown here, you can see that these events are a part of the debug channel. In the next couple of images, I want to show some of the payload attached to these events.

These events show what I have shown previously with regards to output of the stored procedure. There is a consistency in the output of the diagnostics process. What I have yet to show in this data is one more piece of information that could be helpful down the road and that pertains specifically to the extended events data. That piece of information I need to share is the keyword associated with each of these events. When looking for related events, I need you to understand that the sp_server_diagnostics events are a part of the scheduling classification. There, I have said it. I just gave you a big clue to help you in your troubleshooting endeavors and the use of the diagnostics processes/events.

Which Sessions then?

Now that I have shown which events are related to diagnostics, I can finally show which sessions are dependent upon the diagnostics data. To show that kind of information, I have this next query to help:

Source code

/* session that is running */
SELECT xs.name AS SessionName
		, xse.event_name
	FROM sys.dm_xe_session_events xse
		INNER JOIN sys.dm_xe_sessions xs
			ON xse.event_session_address = xs.address
	WHERE xse.event_name IN ( 'sp_server_diagnostics_result_set',
								'sp_server_diagnostics_component_result' );
 
/* sessions on the server but may be stopped */
SELECT ses.name AS SessionName
		, sese.name AS event_name
	FROM sys.server_event_session_events sese
		INNER JOIN sys.server_event_sessions ses
			ON ses.event_session_id = sese.event_session_id
	WHERE sese.name IN ( 'sp_server_diagnostics_result_set',
							'sp_server_diagnostics_component_result' );

When I execute the statements in the preceding query, I will receive a single result—system_health. That said, do not believe everything I just showed you with that query. SQL Server does a decent job at hiding some of the details. When I run the next query, I will see a little bit more information.

Source code

SELECT *
	FROM sys.dm_xe_sessions;

When I run the preceding query, I will see the following two sessions (given that I have not change the default settings):

Take note of the names of those two sessions. I showed that the system_health session is reliant upon the diagnostics events. Now, I am showing that there is another event session called “sp_server_diagnostics session“. Wait, there is more! SQL Server continues to hide stuff from us.

Hidden Sessions

When you have AlwaysOn configured, there is an additional hidden session that is not visible within the sessions lists inside of SQL Server. To view this hidden session, you must follow the directions shown in this article on msdn. Once you have navigated to the folder containing the hidden XEL files, you can open them in Management Studio and view the data. It is while viewing the data that you will be able to see the relationship between sp_server_diagnostics and this newly discovered trace file.

The payload from this image demonstrates the relationship between sp_server_diagnostics and the hidden trace file. I challenge you to take some time to explore this hidden trace file on one of your servers where AlwaysOn is configured. I know you will be thankful for the investment at some point down the road.

Conclusion

I have just shown a ton of info about the third component of the black box recorder—sp_server_diagnostics. This procedure may be lesser known than the first two components, but it is highly integral into the overall monitoring and health of your SQL Server instance. In this article, I have not only shown how to use sp_server_diagnostics (albeit quickly), but I have also shown that the diagnostics are heavily relied upon by other components of the black box recorder. In addition, I have shown how this component is integrated into other hidden traces that are essential in the monitoring and troubleshooting of your instance.

The black box recorder is real. You need to invest time in getting to know the black box recorder components and how they can be of help to you in becoming a superhero DBA.

The post sp_server_diagnostics – Black Box Recorder appeared first on SQL Solutions Group.

• Default Trace
• system_health Extended Event Session
• sp_server_diagnostics procedure

The first article went into deeper detail about the first leg of this three-legged black box recorder – the default trace. In this article, the focus will be directed to the next leg of the black box recorder – or the system_health Extended Event Session. If by some measure you are in the dark about what Extended Events is, then I recommend you read my personal blog series on the subject. There are numerous articles stepping through the subject in easy-to-digest fashion. You can find many of the articles in the following table of contents – here.

The focus of this article will not be to introduce the topic of Extended Events. Rather, it will be to introduce the system_health session and dive into it a fair bit.

SQL Black Box Recorder

What is the system_health session?

Beyond being a component of the black box for SQL Server, what exactly is this event session? The system_health is much as the name implies – it is a “trace” that attempts to gather information about various events that may affect the overall health of the SQL Server instance.

The event session will trap various events related to deadlocks, waits, clr, memory, schedulers, and reported errors. To get a better grasp of this, let’s take a look at the event session makeup based on the available metadata in the dmvs and catalog views.

Source code

DECLARE @SessionName VARCHAR(256) = 'system_health';
 
/* note: this only works if the session is running and the source is "server" not internal or something else.
*/
 
SELECT xs.name AS SessionName
		, xs.session_source
		, xse.event_name
		, xo.description AS EventDesc
		, xsea.action_name
		, xoa.description AS ActionDesc
		, xse.event_predicate
		, sese.predicate, sese.predicate_xml
	FROM sys.dm_xe_sessions xs
		INNER JOIN sys.dm_xe_session_events xse
			ON xs.address = xse.event_session_address
		LEFT OUTER JOIN sys.dm_xe_session_event_actions xsea
			ON xse.event_session_address = xsea.event_session_address
				AND xse.event_name = xsea.event_name
		INNER JOIN sys.dm_xe_objects xo
			ON xse.event_name = xo.name
				AND xse.event_package_guid = xo.package_guid
				AND xo.object_type = 'event'
		LEFT OUTER JOIN sys.dm_xe_objects xoa
			ON xsea.action_name = xoa.name
				AND xsea.action_package_guid = xoa.package_guid
				AND xoa.object_type = 'action'
		INNER JOIN sys.server_event_sessions ses
			ON xs.name = ses.name
		LEFT OUTER JOIN sys.server_event_session_events sese
			ON ses.event_session_id = sese.event_session_id
			AND xse.event_name = sese.name
	WHERE xs.name = @SessionName
	ORDER BY xs.name,xse.event_name, xsea.action_name;

In addition to the types of events I mentioned, there are also a few more interesting things to discover from the results of this particular query. For instance, the actions that are being employed help to gather even more information such as the callstack, tsql_callstack, database id, and sql text of the query that was executing at the moment of the event that is trapped.

One very peculiar event that is trapped is the sp_server_diagnostics_component_result event. Remember from earlier in this article that I noted the third leg of the black box record happens to be a very similarly named session – “sp_server_diagnostics procedure”. When I cover this third leg, it will make a little more sense. Understand, for now, that your suspicion of it being a bit more integral to the black box (than merely 1/3 of the overall black box) is confirmed.

This is a good set of information that can be trapped. Furthermore, this is a good set of information which can give a fairly good diagnosis of several different potential problems within your SQL Server Instance! (Yes it may seem repetitive, but I wanted to underscore the importance of the data that can be trapped.) That said, compare the types of events to those of the default trace. Take note of how many events actually overlap between the two components of the black box recorder.

For ease of comparison, let’s simplify that previous query to get a set of events to read without the duplication due to the actions on each event.

Source code

DECLARE @SessionName VARCHAR(256) = 'system_health';
 
SELECT xs.name AS SessionName, xse.event_name
	FROM sys.dm_xe_sessions xs
		INNER JOIN sys.dm_xe_session_events xse
			ON xs.address = xse.event_session_address
	WHERE xs.name = @SessionName
	ORDER BY xse.event_name;

And since I am feeling particularly generous, I have lumped the events from the two traces together in the following query:

Source code

DECLARE @SessionName VARCHAR(256) = 'system_health';
 
SELECT xs.name AS SessionName
		, xse.event_name AS EventName
		, mv.map_value AS CategoryName
	FROM sys.dm_xe_sessions xs
		INNER JOIN sys.dm_xe_session_events xse
			ON xs.address = xse.event_session_address
		INNER JOIN sys.dm_xe_objects xo
			ON xse.event_package_guid = xo.package_guid
			AND xse.event_name = xo.name
			AND xo.object_type = 'event'
		INNER JOIN sys.dm_xe_object_columns oc
			ON oc.OBJECT_NAME = xo.name
			AND oc.object_package_guid = xo.package_guid
			AND oc.name = 'KEYWORD'
		INNER JOIN sys.dm_xe_map_values mv
			ON oc.type_name = mv.name
			AND oc.column_value = mv.map_key
			AND oc.object_package_guid = mv.object_package_guid
	WHERE xs.name = @SessionName
UNION ALL
/* events */
SELECT 'Default Trace' AS SessionName
		, te.name AS EventName
		, tca.name AS CategoryName
	FROM sys.traces t
		CROSS APPLY ( SELECT DISTINCT gei.eventid
							FROM sys.FN_TRACE_GETEVENTINFO(t.id) gei
					) ca
		INNER JOIN sys.trace_events te
			ON te.trace_event_id = ca.eventid
		INNER JOIN sys.trace_categories tca
			ON te.category_id = tca.category_id
	WHERE t.is_default = 1
	ORDER BY EventName;

That should help to compare the 17 events from the system_health session and the 34 from the default trace with a little more ease and speed. The events from the two sessions are rather complimentary to each as they help to construct the SQL Server Black Box recorder.

How to use this session

With the events trapped by this session, there are a few ways to use the trapped data that should stand out pretty easily. To me, the quick application of the data from this session can come from the review of deadlocks, reported errors, or even waits analysis. Using the deadlock as probably the most common problem people will want to troubleshoot, this information is suddenly much more valuable and easy to access (no more need for that trace flag). Since the system_health is now trapping the deadlock by default, one could quickly access the deadlock graph from the event data and proceed to troubleshooting. More information on getting that deadlock graph can be found here and here.

Learn to use the data made available through this member of the black box recorder. Becoming familiar with this tool (and additionally with Extended Events) will make you a rock-star in the office. The information needed to troubleshoot most problems is right at your fingertips through the use of the different pieces of the black box recorder. If it is not available there, then more than likely it can be made available through a bit of tinkering with Extended Events. Get to know the tools and be that rock-star DBA.

The post SQL Black Box Recorder part II appeared first on SQL Solutions Group.

An indexed view is a view where the result set from the query (the view definition) becomes materialized in lieu of the virtual table result set of a standard (non-indexed) view. Many times we see that an indexed view would be created to help improve performance. Far too often, an indexed view is created without consideration for the costs of the indexed view. In this article, I hope to cover some of the more important costs that are frequently overlooked when considering an indexed view as a performance panacea.

The Setup

To demonstrate the overlooked costs of indexed views, I have decided to use the AdventureWorks2014 database (if you don’t have this sample database, you can get your copy here). I will create a view in that database and then I will add a few indexes to that view. Prior to sharing the entire setup, and like all good DBAs, I need to first grab a baseline. For this baseline, I am doing nothing terribly complex. I will just grab table sizes and storage information for the entire database.

Here are my results for the size of the objects within the AdventureWorks2014 database:

These results show that the largest object in the database happens to be the Person.Person table at just about 30MB. Not terribly large, but the database is not terribly large as a whole. Let’s see what happens when I add a materialized view, based on the Person.Person table, to the mix. Here is the definition of that view along with the indexes that I will add to help the performance of some essential production related queries.

Source code

USE [AdventureWorks2014];
GO
 
CREATE VIEW [Person].[vPerson]
WITH SCHEMABINDING
AS
	SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
			, pp.AdditionalContactInfo
			, pp.Demographics
		FROM [Person].[StateProvince] sp
			INNER JOIN [Person].[CountryRegion] cr
				ON sp.[CountryRegionCode] = cr.[CountryRegionCode]
			INNER JOIN Person.Address pa
				ON sp.StateProvinceID = pa.StateProvinceID
			INNER JOIN Person.BusinessEntityAddress pbe
				ON pbe.AddressID = pa.AddressID
			INNER JOIN Person.Person pp
				ON pp.BusinessEntityID = pbe.BusinessEntityID
		WHERE pbe.AddressTypeID = 2;
 
GO
 
 
CREATE UNIQUE CLUSTERED INDEX CI_vPersonID ON Person.vPerson(BusinessEntityID);
 
CREATE INDEX IX_vPersonName ON Person.vPerson(FirstName,LastName);
CREATE INDEX IX_vPersonState ON Person.vPerson(FirstName,LastName,StateProvinceID,StateProvinceCode,StateProvinceName);

After executing all of that code to create this new view with its indexes, I have the following size results:

The creation of this view has chewed up a bunch of storage. It has jumped right up to the number two spot on the biggest objects list within this database. You can see that differences by comparing the highlighted rows to the previous image. The vPerson view is highlighted in red in this second image to help point it out quickly.

Surely this must be a contrived example and people don’t really do this in the real world, right? The answer to that is simply: NO! It DOES happen. I see situations like this all too often. Far too often, large text fields are added to an indexed view to make retrieval faster. I have mimicked that by adding in two XML columns from the Person.Person table. This is definitely overkill because a simple join back to the table based on the BusinessEntityID would get me those two columns. All I have effectively done is duplicated data being stored and I have achieved that at the low low cost of increased storage of 25% for this small database. If you are curious, the column count between the Person.Person table and this new view is 13 columns each.

I call the 25% increase storage cost a significant increase. An increase of 25% in storage for a single materialized view is not an uncommon occurrence. I have seen multi-terabyte databases with 25% of the total data storage being caused by a single view. If you are tight on storage, any unexpected data volume increase can cause a non-linear growth in data requirements to occur just because of the presence of indexed views. Take that into consideration when presented with an option to either performance tune the code or to take a short-cut and create an indexed view.

Band-aid Performance

I alluded to the use of indexed views as a cog to help improve performance. This is done by taking the query that sucks and turning it into a view. The thought is that it will run faster because an index is created specifically for that data set. Generally the performance gains from using an indexed view, to camouflage the bad query, are seen when the query is complex. Unfortunately, the query I provided for this article is not terribly complex. Due to the simplicity of the first example, let’s look first at the execution plan for another query that runs terribly slow but is also not terribly complex:

There are a few table spools and index scans. The most costly operator of the bunch appears to be a merge join with a many-to-many join operation. The indexed view in this case does give me a very significant improvement on this query (look at that estimated cost and then the estimated number of rows). What if I could tune the query a little and avoid the all of storage cost? Let’s see!

Consider the base query for this new view:

Source code

SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
			, pp.AdditionalContactInfo
			, pp.Demographics
		FROM [Person].[StateProvince] sp 
			INNER JOIN [Person].[CountryRegion] cr
				ON RTRIM(LTRIM(sp.[CountryRegionCode])) = RTRIM(LTRIM(cr.[CountryRegionCode]))
			INNER JOIN Person.Address pa
				ON RTRIM(LTRIM(sp.StateProvinceID)) = RTRIM(LTRIM(pa.StateProvinceID))
			INNER JOIN Person.BusinessEntityAddress pbe
				ON RTRIM(LTRIM(pbe.AddressID)) = RTRIM(LTRIM(pa.AddressID))
			INNER JOIN Person.Person pp
				ON RTRIM(LTRIM(pp.BusinessEntityID)) = RTRIM(LTRIM(pbe.BusinessEntityID))
		WHERE RTRIM(LTRIM(pbe.AddressTypeID)) = 2;

Looking at the query, you may be asking yourself why use so many trim functions on each of the joins? This is an extreme that can be easily fixed. None of the fields in the joins should need to be trimmed since they are all numeric in this case. Starting there and removing all of the function use in the joins should provide significant improvements in query speed and without the use of a view. Testing it out would yield similar performance results to the first view in this article. This small change would cause the query to complete in 1 second compared to about 90 seconds. That is a huge gain, and without the added cost of increased storage use.

Do I really see stuff like this in the field? Yes! Spending a little time to fix the root of the problem rather than looking for a quick fix and performance can be spectacular. There are times, though, that an indexed view may be an absolute requirement. That would be perfectly fine. There are times when it is warranted and there is no way around it. If an indexed view is an absolute must, then there are a couple more considerations to be taken into account.

Disappearing Act

I showed previously that the indexed view requires storage space and showed that the new view created quickly claimed its place as the second largest object within the database. Now, to stage the next segment, I will show once again that the indexes are present and consuming disk space. This can be shown via the following query:

Source code

SELECT o.name AS ObjName
		, i.name AS IdxName
		, SUM(ps.page_count) AS page_count
		, SUM(ps.record_count) AS record_count
		, SUM(ps.page_count) / 128.0 AS SizeMB
	FROM sys.indexes i
		INNER JOIN sys.objects o
			ON i.OBJECT_ID = o.OBJECT_ID
		INNER JOIN sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL,
													'detailed') ps
			ON ps.OBJECT_ID = o.OBJECT_ID
	WHERE o.name LIKE 'vperson%'
	GROUP BY o.name
		, i.name;

And the results of that query should look something like the following (quick reminder that I created a new indexed view for the second demo and these results will be based on that second view):

Now that we know the index is present and consuming space, time to make a change to the view and see what happens. I am proceeding under the premise that I have determined that including the two XML columns in the view are completely unnecessary and are the cause of too much space consumption. I can reap the benefits of the view at a fraction of the cost if I ALTER the view and remove those columns. So I will proceed by issuing the following ALTER statement:

Source code

ALTER VIEW [Person].[vPerson2]
WITH SCHEMABINDING
AS
	SELECT pp.BusinessEntityID
			, pp.FirstName
			, pp.LastName
			, sp.[StateProvinceID]
			, sp.[StateProvinceCode]
			, sp.[IsOnlyStateProvinceFlag]
			, sp.[Name] AS [StateProvinceName]
			, sp.[TerritoryID]
			, cr.[CountryRegionCode]
			, cr.[Name] AS [CountryRegionName]
			, pbe.AddressTypeID
		FROM [Person].[StateProvince] sp 
			INNER JOIN [Person].[CountryRegion] cr
				ON RTRIM(LTRIM(sp.[CountryRegionCode])) = RTRIM(LTRIM(cr.[CountryRegionCode]))
			INNER JOIN Person.Address pa
				ON RTRIM(LTRIM(sp.StateProvinceID)) = RTRIM(LTRIM(pa.StateProvinceID))
			INNER JOIN Person.BusinessEntityAddress pbe
				ON RTRIM(LTRIM(pbe.AddressID)) = RTRIM(LTRIM(pa.AddressID))
			INNER JOIN Person.Person pp
				ON RTRIM(LTRIM(pp.BusinessEntityID)) = RTRIM(LTRIM(pbe.BusinessEntityID))
		WHERE RTRIM(LTRIM(pbe.AddressTypeID)) = 2
;
 
GO

Take note of how long this ALTER command takes to complete – nearly instantaneous.  That rocks! Right? Let’s look at the indexes to make sure they are still in working order and don’t require a defrag. Based on that, I will re-run that query to check the indexes and sizes – for giggles.

Source code

SELECT o.name AS ObjName
		, i.name AS IdxName
		, SUM(ps.page_count) AS page_count
		, SUM(ps.record_count) AS record_count
		, SUM(ps.page_count) / 128.0 AS SizeMB
	FROM sys.indexes i
		INNER JOIN sys.objects o
			ON i.OBJECT_ID = o.OBJECT_ID
		INNER JOIN sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL,
													'detailed') ps
			ON ps.OBJECT_ID = o.OBJECT_ID
	WHERE o.name LIKE 'vperson%'
	GROUP BY o.name
		, i.name;

This time when the query completes, I get a nice little surprise: I no longer have any indexes on that view. Changing the view definition instantly drops any indexes that were built on that view. If this is done in a production system, imagine the implications and problems that could arise.

Conclusion

In this article, I have covered indexed views and three different considerations that are often overlooked when looking to implement a materialized view. The three considerations for indexed views, as discussed, are: storage, changes to the view, and the band-aid effect. Overlooking the cost of storage can have serious repercussions on the production environment. Implementing an indexed view without attempting to tune the code first could lead to overlooking the storage cost and could end up just being a waste of resources. The last tidbit is that any change to an indexed view will drop the indexes. That is an easily overlooked feature of indexed views. If you forget to recreate the indexes on the view after making changes, you could be looking at a production outage.

There are a couple more critical considerations for indexed views that are often overlooked as well. These additional considerations are blocking, deadlocking, and maintenance. These considerations will be discussed further in a future article.

The post Indexed Views – Performance Panacea or Plight appeared first on SQL Solutions Group.

One of the frequently required job functions of the database administrator is to track disk space consumption. Whether this requirement comes from management or from a learning opportunity after a production outage, the need exists.

As a hard working DBA, you want to make sure you hit all of the notes to make management sing your praises. Knowing just when the database may fill the drives and prevent a production outage just happens to be one of those sharp notes that could result in a raise and management singing hallelujah. The problem is, how do you do it from within SQL Server? You are just a DBA after all and the disk system is not your domain, right?

Trying to figure it out, you come across a pretty cool function within SQL Server. The name of the function is sys.dm_os_volume_stats. Bonus! This is an excellent discovery, right? Let’s see just how it might work. First a sample query:

Source code

SELECT DISTINCT  ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM sys.master_files mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

If I run that on my local system, I might end up with something that looks like the following:

Looking at the image you may be wondering to yourself right now why I have highlighted a couple of things. You may also be wondering why I used the word “might” in the previous paragraph as well. The reasoning will become more evident as we progress. For now, you have resolved to continue testing the script so execute it again and end up with something that may look like the following (for the same server):

Whoa! What just happened there? Why are there two listings for the C: drive? Why does each register a different value for the FreeSpace column? In addition without any additional usage on the C drive (as verified through other tools) the FreeSpace is changing between executions as well as within the same execution. This is problematic, so you continue testing:

And yet again!

This can’t be correct, can it? Just for giggles let’s modify it just a bit to see if there are any additional clues. Using the following changed script, hopefully a clue will help shed some light on this:

Source code

SELECT ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM sys.master_files mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

This script yields the following potential results:

Look at the different highlighted areas! There are three different values for FreeSpace for the C: drive in this particular execution. The case of the C: drive plays no role in whether the value is recorded differently or not. This seems to be more of a bug within the dynamic management function. From execution to execution, using this particular method, one could end up with duplicate entries but distinct values. The sort of the execution could be returned differently (though we could fix that).

All of these tests were run on my local machine and I really do only have one C: drive. I should never receive multiple entries back for any drive. If using this particular DMF to track space usage, it could be somewhat problematic if the duplicate drive data pops up. How do we get around it, you ask? Here is another example that I have that has not yet produced this duplication:

Source code

WITH presel AS (SELECT database_id, FILE_ID,LEFT(mf1.physical_name,3) AS Volume, ROW_NUMBER() OVER (PARTITION BY LEFT(mf1.physical_name,3) ORDER BY mf1.database_id) AS RowNum
					FROM sys.master_files mf1)
,roundtwo AS (SELECT DISTINCT pr.database_id, pr.FILE_ID
				FROM presel pr
				WHERE pr.RowNum = 1)
 
SELECT ovs.logical_volume_name AS VolumeName
		, ovs.volume_mount_point AS DiskDrive
		, ovs.available_bytes AS FreeSpace
		, ovs.total_bytes AS CurrentSize
	FROM roundtwo mf
		CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.FILE_ID) ovs;

Using this version of the script is not terribly more complex, but it will prove to be more reliable. You can see I used some CTEs to provide a little trickery and ensure that I limit my results. What if it is a mount point or a non-standard drive letter? I have not tested that. Let me know how that goes. As you can see, I am restricting the drive selection by using the row_number function against the drive letter.

For alternative reliable methods to find your disk space consumption, I would recommend something different because it is tried and tested. I would recommend using a wmi call to fetch the data. Samples are provided as follows:

Source code

/* for mount points, something like this */
EXECUTE sys.xp_cmdshell 'wmic volume get name, freespace, capacity, label'
/* the base wmi query that does not support mount points */
EXECUTE xp_cmdshell 'wmic logicaldisk get name,freespace,size,volumename,blocksize'

Easy peasy, right? Have at it and try tracking your disk space.

The post Disk Space and SQL Server appeared first on SQL Solutions Group.

We often pontificate about the what-ifs for a black box recorder within SQL Server, right? We wish and dream and go about creating our own little recorder to suit the requirements we deem necessary for successful monitoring of the server for the just-in-case scenario. Well, the truth of the matter is that we really don’t need to go to such great lengths to create a “black box recorder” because Microsoft has already done all of that for us.

Wait, what? Yes, that’s right! Truth be told this shouldn’t be much news for most Senior Level data professionals working with SQL Server for the past few years. But if you are new to the product, this might be good news for you. Then again, it might just be bad news depending on your take.

Very much like what you may find with the airline industry, the black box recorder in SQL Server is not just a single method (device) implemented to capture all of the desired data points. On a passenger jet, you may find that there are three or more such devices that contribute to the capture and recording of the in-flight data. In SQL Server, there are three major processes that help capture our in-flight data. Over the next few articles I will discuss each of these processes. These processes include:

• Default Trace
• system_health Extended Event Session
• sp_server_diagnostics procedure

This multi-tiered approach does provide a sort of fail-safe system. If one should be disabled, there still remain up to two more processes that may be running. That said, each of these can be disabled but it does take a bit of an overt effort. And since it does require somebody to put out effort to try and disable each of the black box components, you could potentially capture the culprit via audits from the other components or via an explicit audit that you create for these types of purposes.

Since this will be something to discuss over a few articles, I will break out each process into an individual article. For today, I will discuss the default trace.

Default Trace

The default trace by itself is something that can be turned off via configuration option. There may be good reason to disable the default trace. Before disabling the default trace, please consider the following that can be captured via the default trace. I will use a query to demonstrate the events and categories that are configured for capture in the default trace.

Source code

/* events */
SELECT te.name AS EventName
		,tca.name AS CategoryName
		, CASE tca.type WHEN '0' THEN 'Normal'
				WHEN '1' THEN 'Connection'
				WHEN '2' THEN 'ERROR' END AS CategoryType
		, t.PATH AS TracePath
		, oa.logical_operator,oa.comparison_operator, oa.VALUE AS FilteredValue
	FROM sys.traces t
		CROSS APPLY (SELECT DISTINCT gei.eventid FROM sys.FN_TRACE_GETEVENTINFO(t.id) gei) ca
		INNER JOIN sys.trace_events te
			ON te.trace_event_id = ca.eventid
		INNER JOIN sys.trace_categories tca
			ON te.category_id = tca.category_id
		OUTER APPLY (SELECT gfi.columnid,gfi.logical_operator,gfi.comparison_operator,gfi.VALUE FROM sys.FN_TRACE_GETFILTERINFO(t.id) gfi) oa
	WHERE t.is_default = 1

In this query, I have requested a few more data points than necessary to illustrate the point. That is to help illustrate an additional point that the default trace isn’t filtering out any data for these events. If the event fires (in this trace), it is recorded. Let’s divert right back to the events and categories for now. The results of that query will produce the following sample list of events for me on SQL Server 2014:

That is quite a range of events covered by this particular recording device. From changes to objects down to security related events and even errors and warnings. If somebody drops an object, the default trace can catch it. If one of the various DBCC statements is executed, it will be trapped in this trace.

There is one thing that is captured by this trace that is not overly obvious. In fact, it is rather annoying in how it is recorded in my opinion. Server configuration settings such as “cost threshold of parallelism” or the lesser utilized “user options” are not registered as an “Object:Alter” event but rather as an ErrorLog event. I wouldn’t call this type of “change” an error and don’t necessarily like seeing these registered with other legitimate errors. That said, it would be nice to see these logged differently (they are currently logged the same way with Extended Events). So, if somebody is dorking around with server configurations, it becomes a bit more convoluted to figure it out, but we can get there. Let’s see how that works:

Source code

-- server configs Audited via Def Trace
SELECT 	T.StartTime
		, T.SPID
		, T.LoginName
		, T.HostName
		, T.ApplicationName
		, T.DatabaseName
		--, ObjectName,sv.number AS ObjTypeVal, sv.name [ObjectType]
		--, T.TextData
		, ConfigOption = SUBSTRING(T.TextData,CHARINDEX('''',T.TextData)+1,CHARINDEX(' changed from ',T.TextData)-CHARINDEX('''',T.TextData)-2)
		, PrevValue = SUBSTRING(T.TextData,CHARINDEX('from ',T.TextData)+5,CHARINDEX('to ',T.TextData)-CHARINDEX('from ',T.TextData)-5)
		, NewValue = SUBSTRING(T.TextData,CHARINDEX('to ',T.TextData)+3,CHARINDEX('. Run',T.TextData)-CHARINDEX('to ',T.TextData)-3)
		, EventName = te.name
		, T.EventClass
	FROM sys.FN_TRACE_GETTABLE(CONVERT(VARCHAR(150), 
			( SELECT REVERSE(SUBSTRING(REVERSE(PATH),
					CHARINDEX('\',REVERSE(path)),256)) + 'LOG.trc'
				FROM    sys.traces
				WHERE   is_default = 1)), DEFAULT) AS T  
		INNER JOIN sys.trace_events AS te
			ON T.EventClass = te.trace_event_id
	WHERE T.EventClass = 22
		AND T.TextData LIKE '%config%'
	ORDER BY T.StartTime DESC;

Looking at my system, I get the following sample results:

You can see there are plenty of settings that I have been playing with captured by the default trace. If you happen to have a junior DBA or maybe a cowboy DBA that loves to shoot from the hip and make changes, this will help you find all the necessary details to bring to that DBA for discussion.

The default trace comprises one third of the black box recorder. I have shown a quick use for the default trace and have also shown some of the various events that are captured from this trace. I recommend getting to know your default trace just a little bit better. You never know when you may need to resort to the flight recorder data held in the default trace. Being familiar with the default trace before you need to use it will help improve your comfort level when under the stress of trying to figure out what happened just before the server went belly up!

The post SQL Black Box Recorder appeared first on SQL Solutions Group.

I am hopeful to see the room filled with people looking to get some fantastic training. I also want to note that this training is not a part of the Summit, though I hope it is viewed as an effort to enhance the learning made available during the week of Summit and to give the attendees more options.

An option is only really any good if some sort of detail can be attained about the option. While I announced previously that the event was going to happen with a brief introduction into the the topics to be covered, one may be left wondering what really will be covered during the day.

With this post, I hope to help make the SSG FreeCon option a better option  by including a little detail into what it is that I will be presenting during my session on Extended Events.

Extended Events

This should come as a huge surprise given the 60 Days of XEvents series that I am currently publishing. While I am publishing a lot of information about XE in this series, this FreeCon session will cover the methodology that I, as a consultant, would use to troubleshoot various issues when a client seeks my help.

It is well known that there is a wealth of information within Extended Events. One can tap plenty of information about performance issues, errors, or general interest type stuff by setting up an XEvent Session and trapping some (hopefully) useful information.

During this session at the FreeCon, I want to help show how I would use Extended Events as a consultant to help find the pertinent information that will bring to light the problems the client is having.

How can I use XE as a consultant to perform a health check on your server? That information will be covered throughout this session.

How can I determine where precisely the backup failed (and when) and be able to procure more information to determine if it was caused by some other event on the network? This is the type of information included in the methodology that I plan to share.

Think of this session as a condensed road-map to XE Success. I look forward to seeing you there.

The post Freakin’ out on XE at the FreeCon appeared first on SQL Solutions Group.

Well, for whatever reason, there seems to be a myth out there that when deadlocks come a-knocking, then just throw this little directive at it and all will be well. Today, we get to test that and show what will continue to happen.

First, lets look at what the ROWLOCK actually means:

Specifies that row locks are taken when page or table locks are ordinarily taken.

This seems like a fabulous idea if the deadlocks are occurring against a table involving a massive update. Let’s take a look at a small update scenario involving just a handful of records.

Source code

USE tempdb;
GO
IF OBJECT_ID('dbo.Test1') IS NOT NULL
	DROP TABLE dbo.Test1;
IF OBJECT_ID('dbo.Test2') IS NOT NULL
	DROP TABLE dbo.Test2;
CREATE TABLE dbo.Test1 ( col1 INT );
CREATE TABLE dbo.Test2 ( col2 INT );
INSERT INTO dbo.Test1
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
INSERT INTO dbo.Test2
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
GO
BEGIN TRANSACTION;
UPDATE dbo.Test1
	SET	col1 = col1 * 10
	WHERE col1 = 3;
 
/* query window 2*/
 
USE tempdb;
BEGIN TRANSACTION;
UPDATE dbo.Test2
	SET	col2 = col2 * 20
	WHERE col2 = 4;
UPDATE dbo.Test1
	SET	col1 = col1 * 20
	WHERE col1 = 3;
COMMIT TRANSACTION;
 
/* back to the first query window*/
UPDATE dbo.Test2
	SET	col2 = col2 * 10
	WHERE col2 = 4;
COMMIT TRANSACTION;

Looking at the code, we can see there are only five rows in each of the tables. In addition, an update will be performed to both col1 and col2 in each table for one specific row. So we are keeping this to a singleton type of update, and we are able to force a deadlock by using this setup. Not only do we see that a deadlock will occur consistently, we would see the following in the sys.dm_tran_locks DMV as well as in the deadlock graphs.

In my environment, I used session 51 and 54 consistently for the deadlock repro. In each of the tests, each spid did obtain page locks as well as RID locks (row locks) that were waiting on each other in order to perform an Update. This is what we saw when I ran the setup without the ROWLOCK directive. What if I used the ROWLOCK directive (are you guessing already that there would be no change because the deadlock occurs on the lock held for the update that is waiting on the other update to complete?)? Let’s take a look at that too!

Source code

/* what about using rowlock hint? let's try it */
USE tempdb;
GO
IF OBJECT_ID('dbo.Test1') IS NOT NULL
	DROP TABLE dbo.Test1;
IF OBJECT_ID('dbo.Test2') IS NOT NULL
	DROP TABLE dbo.Test2;
CREATE TABLE dbo.Test1 ( col1 INT );
CREATE TABLE dbo.Test2 ( col2 INT );
INSERT INTO dbo.Test1
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
INSERT INTO dbo.Test2
	VALUES	( 1 ),
			( 2 ),
			( 3 ),
			( 4 ),
			( 5 );
GO
BEGIN TRANSACTION;
UPDATE dbo.Test1 WITH (ROWLOCK)
	SET	col1 = col1 * 10
	WHERE col1 = 3;
 
/* query window 2*/
 
USE tempdb;
BEGIN TRANSACTION;
UPDATE dbo.Test2 WITH (ROWLOCK)
	SET	col2 = col2 * 20
	WHERE col2 = 4;
UPDATE dbo.Test1 WITH (ROWLOCK)
	SET	col1 = col1 * 20
	WHERE col1 = 3;
COMMIT TRANSACTION;
 
/* back to the first query window*/
UPDATE dbo.Test2 WITH (ROWLOCK)
	SET	col2 = col2 * 10
	WHERE col2 = 4;
COMMIT TRANSACTION;

The only change to this setup is that the ROWLOCK directive has been added to the update statements. Examining the sys.dm_tran_locks DMV reveals the same locks being held as was seen without the directive. This shouldn’t be too big of a surprise since the updates are against a single row.

In addition to the same locks being held, we continue to experience the same deadlock problem. Using an Extended Events session to trap deadlock information (similar to the article previously discussed), we can pull out some pretty good info. Let’s examine some of the deadlock data trapped by an XE session.

Source code

WITH	cte	AS ( SELECT t2.event_data.VALUE('(event/@name)[1]', 'varchar(50)') AS event_name
							, t2.event_data.VALUE('(event/@timestamp)[1]',
													'datetime2') AS StartTime
							, t2.event_data.VALUE('(event/data[@name="duration"]/value)[1]',
													'bigint') AS duration
							, t2.event_data.VALUE('(event/data[@name="database_name"]/value)[1]',
													'sysname') AS DBName
							, t2.event_data.VALUE('(event/action[@name="sql_text"]/value)[1]',
													'varchar(max)') AS sql_text
							, t2.event_data.VALUE('(event/action[@name="nt_username"]/value)[1]',
													'varchar(500)') AS nt_username
							, t2.event_data.VALUE('(event/data[@name="mode"]/value)[1]',
													'varchar(15)') + ' ('
							+ t2.event_data.VALUE('(event/data[@name="mode"]/text)[1]',
													'varchar(50)') + ')' AS mode
							, t2.event_data.VALUE('(event/data[@name="object_id"]/value)[1]',
													'integer') AS OBJECT_ID
							, t2.event_data.VALUE('(event/data[@name="resource_description"]/value)[1]',
													'varchar(max)') AS resource_description
							, t2.event_data.VALUE('(event/data[@name="resource_owner_type"]/text)[1]',
													'varchar(max)') AS resource_owner_type
							, t2.event_data.VALUE('(event/data[@name="resource_type"]/text)[1]',
													'varchar(max)') + ' ('
							+ t2.event_data.VALUE('(event/data[@name="resource_type"]/value)[1]',
													'varchar(max)') + ')' AS resource_type
							, t2.event_data.VALUE('(event/action[@name="server_principal_name"]/value)[1]',
													'varchar(max)') AS server_principal_name
							, t2.event_data.VALUE('(event/action[@name="session_id"]/value)[1]',
													'varchar(max)') AS session_id
							, t2.event_data.VALUE('(event/action[@name="client_pid"]/value)[1]',
													'integer') AS client_pid
							, t2.event_data.VALUE('(event/action[@name="client_app_name"]/value)[1]',
													'varchar(max)') AS client_app_name
							, t2.event_data
							,ROW_NUMBER() OVER (ORDER BY t2.event_data.VALUE('(event/@timestamp)[1]',
													'datetime2')
													,t2.event_data.VALUE('(event/@name)[1]', 'varchar(50)')) AS Event_Seq
						FROM sys.fn_xe_file_target_read_file('C:\Database\XE\Deadlock*.xel',
																NULL, NULL, NULL) t1
							CROSS APPLY ( SELECT CONVERT(XML, t1.event_data)
										) t2 ( event_data )
				)
	SELECT c1.event_name AS LockEvent, c2.event_name AS GraphEvent
			, c1.StartTime
			, DATEADD(MICROSECOND, c1.duration, CONVERT(DATETIME2, [c1].StartTime)) AS EndDate
			, c1.duration
			, c1.DBName
			, c1.sql_text
			, c1.nt_username
			, c1.server_principal_name
			, c1.mode
			, c1.OBJECT_ID
			, c1.resource_description
			, c1.resource_owner_type
			, c1.resource_type
			, c1.session_id
			, c1.client_pid
			, c1.client_app_name
			, c1.event_data
			, graph.query('.') AS EventDeadlockGraph
		FROM cte c1
			INNER JOIN cte c2
				ON c1.Event_Seq = c2.Event_Seq - 1
			CROSS APPLY c2.event_data.nodes('(event/data[@name="xml_report"]/value)[last()]/*')
				AS deadlock ( graph )
		WHERE c1.event_name = 'lock_deadlock'
			AND c2.event_name = 'xml_deadlock_report'
		ORDER BY c1.StartTime;

The results from this query will show us the deadlock graph, the event data, as well as several other pieces of data already parsed from the session data for you. And looking at the session data, one can see that the sql_text from each of the queries will demonstrate both the ROWLOCK directive and the directive-free versions of the query. In this query you can also see that I did a little black magic to match up the two event types from the event session (lock_deadlock and xml_deadlock_report). Then I was able to join the two together to produce one row per deadlock event and to see the sql_text with the deadlock graph on one row. Otherwise, the sql_text does not produce with the deadlock_report event. I leave the rest of the query to the reader to discover and question.

From the EventDeadlockGraph column, we could click the cell and take a close look at the XML generated for the deadlock event. Further, if I choose to save the xml as an XDL file and then reopen it in SSMS, I can see the deadlock graphical report as shown in the following.

We see that row locks are still in effect for the update coming from both sides. This further supports that the directive really is just a waste of time in trying to combat this type of deadlock. This is one of those cases where the best option would be to optimize the code and work things out without trying to take a shortcut.

Wrapping Up

Look to optimize the code instead of trying to take a shortcut. In addition, take a look at the deadlocks, the locks held, and the code to get a better understanding of what is truly happening.

The post An Experiment with Deadlocks appeared first on SQL Solutions Group.

While, this post is not a part of the party, the defaults to be discussed are fairly important. I have seen severe consequences due to these defaults being ignored and not changed. So today, I implore you to make some fundamental changes to your production servers with regards to various defaults.

A Trio of msdb Defaults

There aren’t really that many defaults within the msdb database that must be changed, are there? I mean, seriously, beyond the defaults that are available to every database, what could possibly be unique to this database that could have a severe consequence?

I am so glad you asked!

The defaults in the msdb database are more about what is missing than what is actually there. By default, this database is missing quite a few things that could be deemed critical to your environment.

Let’s start with an easy one – Indexes

There are a few out there that may disagree, but the proof really is in the effect on performance for backup jobs and such. I have three indexes I like to put on every instance. I have seen the implementation of these indexes aid in improved job times as well as aid in reduced time to “clean” up the database.

Source code

/* Number 1 */
/* msdb maintenance activity related indexes */
 
CREATE INDEX IX_maintplan_taskdetailid ON [msdb].[dbo].[sysmaintplan_logdetail]
([task_detail_id])
WITH (FILLFACTOR = 90);
 
CREATE INDEX IX_backupset_type ON msdb.dbo.backupset
([type],[backup_finish_date])
INCLUDE ([media_set_id], [database_name])
WITH (FILLFACTOR = 90);
 
CREATE INDEX IX_jobactivity_jobid ON [msdb].[dbo].[sysjobactivity]
([job_id])
INCLUDE ([session_id])
WITH (FILLFACTOR = 90);

Easy enough. These indexes are very straight forward and pretty small in the grand scheme of things. But if the index can help improve performance by a factor of 10, then I am in favor of them (and I have seen that performance gain).

Now that we have some supporting indexes to help a bit with performance, we should take a look at the next item. This one can help with job performance as well as help with keeping the msdb database nice and trim.

Data Pruning

I have walked into client instances that had backup history dating all the way back to 2005 and included two-three full backups a day per database with quarter-hourly log backups. Oh and this was for an instance containing well north of 200 databases. Can you say sluggish backups and sluggish msdb overall?

The fix is very easy! Not only do I recommend pruning the backup history, but also the job history, mail history and maintenance plan history (eew – if you use those things). Think about it – do you really need to know that Job XYZ ran successfully in 2006 and only took 15 seconds? This is 2015 and that kind of data is probably not pertinent at this point.

The pruning of this data is not enabled by default! You have to configure this for each of the servers under your purview. Luckily, this is easy to do!

Source code

/* NUMBER 2 */
/****** Object:  Job [DBA - Clean msdb Historical data]    Script Date: 03/06/2013 17:40:02 ******/
BEGIN TRANSACTION;
DECLARE @ReturnCode INT;
SELECT @ReturnCode = 0;
/****** Object:  JobCategory [DBA Maintenance]]    Script Date: 03/06/2013 17:40:03 ******/
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.syscategories
					WHERE name = N'DBA Maintenance'
						AND category_class = 1 )
	BEGIN
		EXEC @ReturnCode = msdb.dbo.SP_ADD_CATEGORY @CLASS = N'JOB',
			@type = N'LOCAL', @name = N'DBA Maintenance';
		IF ( @@ERROR <> 0
				OR @ReturnCode <> 0
			)
			GOTO QuitWithRollback;
 
	END;
 
DECLARE @jobId BINARY(16);
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOB @job_name = N'DBA - Clean msdb Historical data',
	@enabled = 1, @notify_level_eventlog = 0, @notify_level_email = 0,
	@notify_level_netsend = 0, @notify_level_page = 0, @delete_level = 0,
	@description = N'No description available.',
	@category_name = N'DBA Maintenance', @owner_login_name = N'sa',
	@job_id = @jobId OUTPUT;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
/****** Object:  Step [Purge Backup History]    Script Date: 03/06/2013 17:40:03 ******/
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSTEP @job_id = @jobId,
	@step_name = N'Purge Backup History', @step_id = 1,
	@cmdexec_success_code = 0, @on_success_action = 3, @on_success_step_id = 0,
	@on_fail_action = 2, @on_fail_step_id = 0, @retry_attempts = 0,
	@retry_interval = 0, @os_run_priority = 0, @subsystem = N'TSQL',
	@command = N'/* Cleanup old backup history */
DECLARE @BackupHistoryOlderThan DATETIME
Set @BackupHistoryOlderThan = DATEADD(d, -90, GETDATE())
EXECUTE msdb.dbo.sp_delete_backuphistory @BackupHistoryOlderThan
 
', @database_name = N'msdb', @flags = 4;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
/****** Object:  Step [Purge MaintPlan Log Data]    Script Date: 03/06/2013 17:40:04 ******/
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSTEP @job_id = @jobId,
	@step_name = N'Purge MaintPlan Log Data', @step_id = 2,
	@cmdexec_success_code = 0, @on_success_action = 3, @on_success_step_id = 0,
	@on_fail_action = 2, @on_fail_step_id = 0, @retry_attempts = 0,
	@retry_interval = 0, @os_run_priority = 0, @subsystem = N'TSQL',
	@command = N'Declare @OldDate DateTime
Set @OldDate = GETDATE() - 14;
 
Execute msdb.dbo.sp_maintplan_delete_log @oldest_time = @OldDate',
	@database_name = N'msdb', @flags = 4;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
/****** Object:  Step [Purge Mail History]    Script Date: 03/06/2013 17:40:04 ******/
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSTEP @job_id = @jobId,
	@step_name = N'Purge Mail History', @step_id = 3, @cmdexec_success_code = 0,
	@on_success_action = 3, @on_success_step_id = 0, @on_fail_action = 2,
	@on_fail_step_id = 0, @retry_attempts = 0, @retry_interval = 0,
	@os_run_priority = 0, @subsystem = N'TSQL',
	@command = N'/* Cleanup old mail items */
DECLARE @MailItemHistoryOlderThan DATETIME
Set @MailItemHistoryOlderThan = DATEADD(d, -14, GETDATE())
EXECUTE msdb.dbo.sysmail_delete_mailitems_sp @sent_before = @MailItemHistoryOlderThan
 
', @database_name = N'master', @flags = 0;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
/****** Object:  Step [Purge Job History]    Script Date: 03/06/2013 17:40:04 ******/
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSTEP @job_id = @jobId,
	@step_name = N'Purge Job History', @step_id = 4, @cmdexec_success_code = 0,
	@on_success_action = 1, @on_success_step_id = 0, @on_fail_action = 2,
	@on_fail_step_id = 0, @retry_attempts = 0, @retry_interval = 0,
	@os_run_priority = 0, @subsystem = N'TSQL',
	@command = N'/*Cleanup of Old Job History*/
Declare @JobHistoryOlderThan DATETIME
Set @JobHistoryOlderThan = DATEADD(d, -14, GETDATE())
EXECUTE msdb.dbo.sp_purge_jobhistory @oldest_date = @JobHistoryOlderThan',
	@database_name = N'master', @flags = 0;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
EXEC @ReturnCode = msdb.dbo.SP_UPDATE_JOB @job_id = @jobId, @start_step_id = 1;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSCHEDULE @job_id = @jobId,
	@name = N'Daily', @enabled = 1, @freq_type = 4, @freq_interval = 1,
	@freq_subday_type = 1, @freq_subday_interval = 0,
	@freq_relative_interval = 0, @freq_recurrence_factor = 0,
	@active_start_date = 20120831, @active_end_date = 99991231,
	@active_start_time = 0, @active_end_time = 235959;
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
EXEC @ReturnCode = msdb.dbo.SP_ADD_JOBSERVER @job_id = @jobId,
	@server_name = N'(local)';
IF ( @@ERROR <> 0
		OR @ReturnCode <> 0
	)
	GOTO QuitWithRollback;
COMMIT TRANSACTION;
GOTO EndSave;
QuitWithRollback:
IF ( @@TRANCOUNT > 0 )
	ROLLBACK TRANSACTION;
EndSave:
 
GO

If you use this code sample, be sure to adjust the number of days shown in the retention to match your specific needs.

Now we have addressed a couple of defaults in msdb that can impact your performance. We are tidying up the database and in a much happier state these days. There is one more default, though, that is really critical to your data’s well being. This one is set within the msdb database but it really is for all of your databases!

Configuring Alerts!

I’m not talking about just any alerts. There are some very specific alerts that really should be configured. These are the alerts that can help you intervene to minimize corruption.

If you haven’t faced a problem with corruption – you will. It is only a matter of time. Corruption happens. When it happens, the earlier one can intervene, usually the better the outcome. Every minute counts, so why not try to reduce that time as much as possible?

This one is not terribly difficult to implement. I happen to have a query ready to go for that as well. All that needs to be done is a minor adjustment to the alert email address:

Source code

/* Number 3 */
USE [msdb];
GO
 
IF EXISTS ( SELECT name
				FROM msdb.dbo.sysalerts
				WHERE name = N'DBA Alert: Message Id: 823' )
	EXEC msdb.dbo.SP_DELETE_ALERT @name = N'DBA Alert: Message Id: 823';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysalerts
					WHERE name = N'DBA Alert: Message Id: 823' )
	EXEC msdb.dbo.SP_ADD_ALERT @name = N'DBA Alert: Message Id: 823',
		@message_id = 823, @severity = 0, @enabled = 1,
		@delay_between_responses = 600, @include_event_description_in = 1,
		@notification_message = N'I/O error detected. This could be a sign of impending I/O failure. Forward this ticket to the primary DBA for this account for followup.',
		@category_name = N'[Uncategorized]',
		@job_id = N'00000000-0000-0000-0000-000000000000';
GO
 
IF EXISTS ( SELECT name
				FROM msdb.dbo.sysalerts
				WHERE name = N'DBA Alert: Message Id: 824' )
	EXEC msdb.dbo.SP_DELETE_ALERT @name = N'DBA Alert: Message Id: 824';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysalerts
					WHERE name = N'DBA Alert: Message Id: 824' )
	EXEC msdb.dbo.SP_ADD_ALERT @name = N'DBA Alert: Message Id: 824',
		@message_id = 824, @severity = 0, @enabled = 1,
		@delay_between_responses = 600, @include_event_description_in = 1,
		@notification_message = N'I/O error detected. This could be a sign of impending I/O failure. Forward this ticket to the primary DBA for this account for followup.',
		@category_name = N'[Uncategorized]',
		@job_id = N'00000000-0000-0000-0000-000000000000';
GO
 
IF EXISTS ( SELECT name
				FROM msdb.dbo.sysalerts
				WHERE name = N'DBA Alert: Message Id: 825' )
	EXEC msdb.dbo.SP_DELETE_ALERT @name = N'DBA Alert: Message Id: 825';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysalerts
					WHERE name = N'DBA Alert: Message Id: 825' )
	EXEC msdb.dbo.SP_ADD_ALERT @name = N'DBA Alert: Message Id: 825',
		@message_id = 825, @severity = 0, @enabled = 1,
		@delay_between_responses = 600, @include_event_description_in = 1,
		@notification_message = N'I/O error detected. This could be a sign of impending I/O failure. Forward this ticket to the primary DBA for this account for followup.',
		@category_name = N'[Uncategorized]',
		@job_id = N'00000000-0000-0000-0000-000000000000';
GO
 
IF EXISTS ( SELECT name
				FROM msdb.dbo.sysalerts
				WHERE name = N'DBA Alert: Message Id: 829' )
	EXEC msdb.dbo.SP_DELETE_ALERT @name = N'DBA Alert: Message Id: 829';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysalerts
					WHERE name = N'DBA Alert: Message Id: 829' )
	EXEC msdb.dbo.SP_ADD_ALERT @name = N'DBA Alert: Message Id: 829',
		@message_id = 829, @severity = 0, @enabled = 1,
		@delay_between_responses = 600, @include_event_description_in = 1,
		@notification_message = N'I/O error detected. This could be a sign of impending I/O failure. Forward this ticket to the primary DBA for this account for followup.',
		@category_name = N'[Uncategorized]',
		@job_id = N'00000000-0000-0000-0000-000000000000';
GO
 
IF EXISTS ( SELECT name
				FROM msdb.dbo.sysalerts
				WHERE name = N'DBA Alert: Message Id: 832' )
	EXEC msdb.dbo.SP_DELETE_ALERT @name = N'DBA Alert: Message Id: 832';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysalerts
					WHERE name = N'DBA Alert: Message Id: 832' )
	EXEC msdb.dbo.SP_ADD_ALERT @name = N'DBA Alert: Message Id: 832',
		@message_id = 832, @severity = 0, @enabled = 1,
		@delay_between_responses = 600, @include_event_description_in = 1,
		@notification_message = N'Page memory error detected. This could be a sign of impending memory failure. Forward this ticket to the primary DBA for this account for followup.',
		@category_name = N'[Uncategorized]',
		@job_id = N'00000000-0000-0000-0000-000000000000';
GO
 
IF NOT EXISTS ( SELECT name
					FROM msdb.dbo.sysoperators
					WHERE name = N'DBA Support' )
	EXEC msdb.dbo.SP_ADD_OPERATOR @name = N'DBA Support', @enabled = 1,
		@weekday_pager_start_time = 90000, @weekday_pager_end_time = 180000,
		@saturday_pager_start_time = 90000, @saturday_pager_end_time = 180000,
		@sunday_pager_start_time = 90000, @sunday_pager_end_time = 180000,
		@pager_days = 0, @email_address = N'su*****@*BA.com',
		@category_name = N'[Uncategorized]';
GO
EXEC msdb.dbo.SP_ADD_NOTIFICATION @alert_name = N'DBA Alert: Message Id: 823',
	@operator_name = N'DBA Support', @notification_method = 1;
EXEC msdb.dbo.SP_ADD_NOTIFICATION @alert_name = N'DBA Alert: Message Id: 824',
	@operator_name = N'DBA Support', @notification_method = 1;
EXEC msdb.dbo.SP_ADD_NOTIFICATION @alert_name = N'DBA Alert: Message Id: 825',
	@operator_name = N'DBA Support', @notification_method = 1;
EXEC msdb.dbo.SP_ADD_NOTIFICATION @alert_name = N'DBA Alert: Message Id: 829',
	@operator_name = N'DBA Support', @notification_method = 1;
EXEC msdb.dbo.SP_ADD_NOTIFICATION @alert_name = N'DBA Alert: Message Id: 832',
	@operator_name = N'DBA Support', @notification_method = 1;
GO

Wrap

Wow! Now there are three quick defaults that must be changed on every server. These defaults will help improve performance as well as help you stay on top of things when they start to go south (corruption). With timely notifications, and better performance, your servers will be happier, healthier, and longer lasting.

The post Defaults In msdb Database appeared first on SQL Solutions Group.

In that article, I showed one common use-case for SQL Server Profiler and how to achieve the same result via Extended Events. What I didn’t show in that article was how to correlate all of your favorite trace/profiler options into the shiny new XE equivalents. Is there even a way to do that?

As luck would have it, there is a means to correlate trace/profiler events to Extended Event events (yeah that sounds weird).  With the release of SQL Server 2012, Microsoft introduced a couple of catalog views to help with this correlation effort. Those views have a pretty straightforward naming convention. For example, one of the views is named sys.trace_xe_event_map while the other is sys.trace_xe_action_map. For this article, I will be focusing on the former.

Making the Upgrade

When looking to finally make the upgrade away from trace/profiler, a big key is to figure out if your favorite trace events are even an option at the next level. I have talked about finding events in XE in previous articles by trying to search for a specific topic that might apply to your current condition.

But if you are already familiar with specific trace events, you may just want/need to know what the new name is in XE. This is where those catalog views come into play. And in support of that, here is a query that can help in that upgrade effort:

Source code

SELECT te.trace_event_id,tc.name AS CategoryName
	, CASE tc.type
		WHEN 0 THEN 'Normal'
		WHEN 1 THEN 'Connection'
		WHEN 2 THEN 'ERROR'
		END AS TraceEventType
	,te.name AS TraceEventName
	, xem.package_name, xem.xe_event_name
FROM sys.trace_xe_event_map xem
	RIGHT OUTER JOIN sys.trace_events te
		ON te.trace_event_id = xem.trace_event_id
	INNER JOIN sys.trace_categories tc
		ON te.category_id = tc.category_id
ORDER BY tc.category_id,te.trace_event_id ASC;

Looking through the results, one would notice that not every trace event maps to an Extended Event event. The events that don’t map, in general, deal with audits (which is actually driven by Extended Events).

This is a really good start to getting you on your way to that much needed upgrade away from profiler/trace. What if there are several traces that are already in use or scripted for various reasons in the environment?

Decision Time

If you happen to have traces that are already deployed in the environment or in script form, it can be a bit of a tedious pain to convert those to Extended Events. Do you manually recreate the traces as XE sessions? Do you abandon the upgrade due to the effort and annoyance it will create? Or do you find some automated means of performing the conversion?

Of those three options, only two are valid options. Those options involve performing the conversion of the traces to XE sessions. There are pros and cons for each. You may opt to take your time and learn more about Extended Events by performing the manual upgrade, or you may choose to save time by using an automated routine.

Should you decide to try the automated routine, there is one already out and available to help you on your way. Jonathan Kehayias wrote the script and you can download it here.

The post Upgrading From SQL Server Profiler appeared first on SQL Solutions Group.