This feature is not suitable for all applications. Microsoft says (at https://technet.microsoft.com/en-us/library/gg162709%28v=WS.10%29.aspx):

Because of the overhead associated with moving TCP/IP processing to the network adapter, TCP Chimney Offload offers the most benefit to applications that have long-lived connections and transfer large amounts of data. Servers that perform database replication, function as file servers, or perform backup functions are examples of computers that may benefit when you enable TCP Chimney Offload.

With the different operating systems versions, this feature is by default in different states:

With all of these changes to the OS, which setting should we use for SQL Server? In general, for all of these operating systems, I recommend that TCP Chimney Offload be disabled – because you can see odd connectivity problems in any other state. Notice in the above quote that Microsoft says that this feature is best used for applications with long-lived connections that transfer large amounts of data – hopefully your OLTP database is performing lots of short-lived connections and they are not transferring large amounts of data (if they are, contact us… we can help you with that!). Some of the error messages that you can encounter are:

[Microsoft][ODBC SQL Server Driver][DBNETLIB] General Network error. Check your network documentation

ERROR [08S01] [Microsoft][SQL Native Client]Communication link failure

System.Data.SqlClient.SqlException: A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 0 – An existing connection was forcibly closed by the remote host.)

These errors are not exclusive to having problems with the TCP Chimney Offload. Additionally, they may only occur during times of high network load on the server.

Do you keep upgrading hardware to improve

SQL Server performance issues without seeing any benefits?

You need a Health Check from SQL Solutions Group.

Learn More

To determine the Current TCP Chimney Offload Setting and to Disable it

In typical Microsoft format, this also varies between different operating system versions.

For Windows Server 2003, you need to check the registry. From a DOS command prompt, run:

Source code

reg query HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\PARAMETERS /v EnableTCPChimney

If disabled, this will have a value of 0x0; if enabled, it will have a value of 0x1.

To disable, from an elevated DOS command prompt, run:

Source code

netsh INT ip SET chimney DISABLED

Source code

netsh INT tcp show GLOBAL

Source code

netsh INT tcp SET GLOBAL chimney=disabled

The post How TCP Chimney Offloading Affects SQL Server appeared first on SQL Solutions Group.

The part that is really astonishing is that we have known how to stop SQL Injection for more than a decade. Completely. Dead. In. Its. Tracks. Which means that the problems happening today are because of careless coding techniques, and as such, they can be prevented.

The “Funky query”

A few years ago, I was working for a company that developed and sold software. One day, my supervisor sends me an email titled “Funky query”. Hmm, that sounds interesting… I’ll open this up right away, instead of ignoring it until I finish what I’m currently working on (like I would usually do).

Wayne… one of our customer support engineers sent us this from the application error log of a customer. They were running a report. Any ideas on how this funky query came to be?

Query from error log (genericized… because I can’t use the real query):

SELECT * FROM sys.objects WHERE name = ‘O’Malley’

Details from the error log:

Incorrect syntax near 'Malley'.
Unclosed quotation mark after the character string ''.

Me: Umm, boss… Does this report allow users to enter search criteria?

Boss: But of course!

Me: Well, I really hate to tell you this, but we have a SQL Injection problem.

And after a bit of back and forth where the developers were insisting that no way was there a SQL Injection problem, I sat down with the dev team lead and the boss and proved it to them. We created a dummy table in the database, went to the report criteria form, and I dropped the table.

Wayne: +1000

Development Team: -1000

Defining SQL Injection

Before we go into how SQL injection occurs, let’s first examine what it is:  a code injection technique that allows SQL code that the user enters in the front-end to be run against the back-end database.

Please note a few things about this definition:

1. SQL Server was not mentioned. Any and all database platforms are susceptible to SQL injection. SQL Server. Oracle. PostgreSQL. DB2. It doesn’t matter – all database platforms are susceptible.
2. A generic front-end is mentioned. This is anything that connects to a back-end database, be it a web form or your favorite application. Anything that connects to a database can allow it.
3. Improperly written code within the database (such as a stored procedure) can allow it. It’s all about how the query is created and run against the database.
4. There are other forms of injection attacks that can occur. Other forms include Cross-site scripting, Web Parameter Tampering, LDAP Injection and XPath Injection (as I write this, WordPress 4.3.1 is being released to fix two Cross-site scripting issues).

SQL injection can happen in three different ways:

1. Classic – this is when a user enters SQL code, and can immediately view the results on screen.
2. Blind – this is essentially a classic attack, but the user does not see the results. Things may still happen, and the user may be able to send results to files that can be downloaded, but the results are not shown on screen.
3. Second Order – the user-supplied SQL code is actually stored in the database, and at a later point in time it is run.

Creating SQL Injection

SQL injection happens when a SQL statement is created from the data supplied by the user, and it is run by the application or other process. Obviously the intention of the application is to only run the queries that it supplies. Consider a login form with user name and password data entry fields. Assume that the user enters “Mike” as the user, and “1234” as the password. In this case, the application essentially performs:

Source code

DECLARE @UserName VARCHAR(50) = 'Mike';
DECLARE @Password VARCHAR(50) = '1234'
DECLARE @SQLCMD VARCHAR(MAX);
SET @SQLCMD = 'SELECT UserName, Password
FROM dbo.Users
WHERE UserName = ''' + @UserName + ''' AND Password = ''' + @Password + '''';
 
EXECUTE (@SQLCMD);

This will create the SQL statement:

Source code

SELECT        UserName, Password
FROM          dbo.Users
WHERE         UserName = 'Mike'
AND           Password = '1234'

And when we run this, we get the expected results. But now, a hacker becomes the user. Being wise to SQL injection, the hacker enters for the password the string

Source code

1234' OR 1=1;--

This results in the following SQL Statement being generated:

Source code

SELECT        UserName, Password
FROM          dbo.Users
WHERE         UserName = 'Mike'
AND           Password = '1234' OR 1=1; --

Since every row matches the “OR 1=1” predicate, every username and password will be returned.

Hacker: +1

Developer: -1

Notice that I showed this attack with T-SQL code – this means that you can have SQL injection vulnerabilities within your T-SQL code (such as stored procedures), as well as any SQL commands that the application might construct and send to the database. The application might be properly handling for SQL injection, but if it is calling SQL code that is improperly coded, you can still be vulnerable. Everything has to be done properly – all it takes is one point where injection is possible to become the latest victim.

What can a hacker do with a vulnerability?

At this point, the hacker knows that the application is susceptible to SQL injection, and can now issue any SQL statement desired. Depending on the access level that the application has to connect to the database, the hacker can:

1. Get a list of all of the databases in the instance.
2. Get a list of all of the tables in the current database, or any other database.
3. Query the data in any of the other tables that can be seen.
4. Modify (insert / update / delete) data in any of the tables that can be seen.
5. Create / modify / drop any object in any database on the instance.
6. Create elevated permissions at the SQL Server instance level. This will allow the hacker to do anything they desire on this instance. Drop tables? Yes. Query any and all data? Yes.
7. Create elevated permissions at the Operating System level.
8. Install programs on the server.
9. Access any server on the network, and perform all of the above on those servers.

Scary? Absolutely. However, as I mentioned before, we know how to prevent this. And it turns out that it’s not hard. Instead of making SQL Statements that include data (either as directly supplied from the user, or from the database), pass that data to the SQL statement through parameters.

If we take the previous example, and modify it to use parameters, we would have:

Source code

DECLARE @UserName VARCHAR(50) = 'Mike';
DECLARE @Password VARCHAR(50) = '1234'' OR 1=1;--'
SELECT  UserName, Password
FROM    dbo.Users
WHERE   UserName = @UserName 
AND     Password = @Password;

What are the chances that you have a password of “1234′ OR 1=1; –” in your system? Assuming that you don’t, this query will return zero rows. SQL injection has been prevented.

So now you want an example of the difference between classic and blind SQL injection? Well, I’m not going to show you an example… because this will normally come down to how the application handles the data being returned. In the “OR 1=1” example, if the application only shows the first record, then the hacker won’t see the results of the SQL injection (unless they notice that it’s showing the wrong data). Or the application may have been coded to expect only one row, and if more are returned then it won’t display any.

Second Order SQL Injection

But how about a second order injection? Let me explain that with an example. Imagine a form for entering new users. The development team has learned about injection on the forms, and they are now using parameters. When the hacker enters for the password:

1234′ OR 1=1;–

The generated SQL statement is:

Source code

DECLARE @UserName VARCHAR(100) = 'Sam';
DECLARE @Password VARCHAR(100) = '1234'' OR 1=1;--';
INSERT INTO dbo.Users (UserName, Password)
VALUES (@UserName, @Password);

When the data is queried:

Source code

SELECT  UserName, Password
FROM    dbo.Users
WHERE   UserName = 'Sam';

We get the following results:

Source code

UserName Password
-------- ----------------
Sam      1234' OR 1=1;--'

The data has been stored in the database, and we are now primed for a second order injection attack. Some other process comes along (for example, a month-end report), and it creates dynamic SQL by building the SQL statement from the data. And the second order SQL injection has just been released…

What have we learned so far?

1. SQL Injection can happen to any database system.
2. SQL Injection can be within SQL code (stored procedures), or it can be in the statements passed to the database from the application.
3. Parameters will prevent the SQL code from being executed immediately, thereby preventing Classic and Blind SQL Injection attacks.
4. The SQL Injection attempt can still be stored in the database, priming yourself for a Second Order SQL Injection attack.

Preventing SQL Injection

Whenever you are creating a SQL statement, it must never create the statement with data (either from the database, or from what the user may pass). Instead, all SQL statements must use parameters. Even the month-end report.

Within SQL Server, have your queries reference the parameters (ColumnName = @VariableName). If you must use dynamic SQL to build your statement, use sp_executesql – it allows for the building of SQL statements with parameters, and then defining and passing those parameters in as the procedure is called.

In your application, there are several component choices. Some directly execute SQL, and others can use parameters. You want to always use the components that use parameters, and you must use those parameters.

Learning more

The Open Web Application Security Project is a community-driven, non-profit organization where experts analyze web application security issues and then compile a list of what the biggest issues are. For the last several years, the category “Injection” has been the biggest issue, and this includes SQL Injection.

This isn’t all that they do. They also have educational resources to not only instruct you on how these issues manifest themselves, but what to do about them. I strongly encourage everyone to take this resource back to your company, and to start identifying and addressing security issues in your code.

Another issue is that developers don’t really know about SQL Injection. In the workshops that I have given where this topic is presented, very few attendees have seen SQL Injection in a test environment. Most can barely give an adequate description of what it is. Those that have seen it are usually those who have been a victim of it. So I also recommend getting your company to arrange for SQL Injection training for the developers and DBAs. It would be good to send your QA team to this also.

The post Combating SQL Injection appeared first on SQL Solutions Group.

I was recently reading this msdn article on Ghost Records, and it mentioned that you could get the number of ghost records on a page with DBCC DBTABLE… and it also mentioned that you need to be sure that you enable Trace Flag 3604 in order to see the results. So, two things immediately jumped out at me. First, I wanted to look at this to see where the ghost records were located. Secondly, I’ve just written a few articles (here, here, here and here) where I’ve been able to use the “WITH TABLERESULTS” option on the DBCC command to avoid using this trace flag and to provide automation for the process, and I wanted to see if that would work here also.

The good news is that “WITH TABLERESULTS” does indeed work with DBCC DBTABLE. The bad news is that I could not find the ghost record count in the results.

When I was looking for this information, I noted that the results meta-data are identical to the way DBCC PAGE has its output, so this means that the automation processes already developed will work for them. And as I was looking through the results, looking for a ghost record counter, I noticed two interesting fields:

Hmm, this is showing me the disk Sector Size of each database file. After checking things on a few different systems, it looks like the m_ActualSectorSize is what the sector size is for the disk that the database file is currently on, and the m_FormattedSectorSize appears to be the sector size for when the database was created – and it is copied from the model database, so it appears to be what the disk was like when Microsoft created the model database.

Here’s where I answer my question from the first paragraph. After digging through the Object and ParentObject columns, I developed this script to get the current allocation using size (Sector Size) for each drive:

Source code

USE master;
GO
IF OBJECT_ID('tempdb.dbo.#DBTABLE') IS NOT NULL DROP TABLE #DBTABLE
CREATE TABLE #DBTABLE (
    ParentObject VARCHAR(255),
    OBJECT       VARCHAR(255),
    Field        VARCHAR(255),
    VALUE        VARCHAR(255));
INSERT INTO #DBTABLE
EXECUTE ('DBCC DBTABLE WITH TABLERESULTS');
 
WITH cte1 AS
(
-- get the objects for the dbt_dbid. Distinct to return only one per database
SELECT  DISTINCT OBJECT
FROM    #DBTABLE
WHERE   Field = 'dbt_dbid'
), cte2 AS
(
-- get the objects related to the dbt_dbid for the m_Startup% field
-- SQL 2005/2008/2008R2 - looking for m_StartupState
-- SQL 2012+ - Looking for m_StartupPhase
-- So use m_Startup%
SELECT  DISTINCT t1.OBJECT
FROM    #DBTABLE t1
JOIN    cte1 ON cte1.OBJECT = t1.ParentObject
WHERE   t1.Field LIKE 'm_Startup%'
), cte3 AS
(
-- get the filepath and sector size for each file
SELECT  fcb_filepath = MAX(CASE WHEN Field = 'fcb_filepath' THEN VALUE ELSE NULL END),
        m_ActualSectorSize = MAX(CASE WHEN Field = 'm_ActualSectorSize' THEN VALUE ELSE NULL END)
FROM    #DBTABLE t1
JOIN    cte2 ON cte2.OBJECT = t1.ParentObject
WHERE   t1.Field IN ('fcb_filepath', 'm_ActualSectorSize')
GROUP BY cte2.OBJECT, t1.OBJECT
)
-- and now get the distinct list of drives and their sector sizes
SELECT  DISTINCT Drive,
        m_ActualSectorSize,
        is_64kb = CASE WHEN m_ActualSectorSize % 65535 = 0 THEN 1 ELSE 0 END
FROM    cte3
CROSS APPLY (SELECT Drive = UPPER(LEFT(fcb_filepath, 2))) ca
ORDER BY Drive;

And here we have yet another way for how a process can be automated by using “WITH TABLERESULTS” on a DBCC command. I think that this one is a particularly good one to show the possibilities – to get this information you have to hit multiple parts of the DBCC results, and repeat it for each file in each database. Doing this by using the 3604 trace flag, finding the appropriate piece and then proceeding on to the piece would be very time consuming to do manually.

Finally, a quick note here: there are better ways of getting the disk sector size – since you can get it with WMI calls, you can get it with PowerShell (or even dos), and there are also command line utilities that will also get you this information. This is just a way to do it from within SQL Server. Note also that this only gets the drives that contain database files on this SQL Server instance – if you are looking for other drives, then this won’t work for you.

Take a look at the other fields that are available in DBTABLE – you just might find another item that you’d like to be able to retrieve.

The post Automating DBCC DBTABLE – Obtaining the Disk Sector Size appeared first on SQL Solutions Group.

This post will also do this, but in this case, we’ll be using it to bust a common myth—data in a clustered index is physically stored on disk in the order of the clustered index.

To bust this myth, we’ll create a database, put a table with a clustered index into this database, and then we’ll add some rows in random order. Next, we will show that the rows are stored on the pages in logical order, and then we’ll take a deeper look at the page internals to see that the rows are not stored in physical order.

To start off with, let’s create a database and a table, and add a few rows to this table:

Source code

IF DB_ID('CIPageTest') IS NULL
CREATE DATABASE CIPageTest;
GO
-- use the database
USE CIPageTest;
GO
-- if the PageTest table exists, then drop it to start all over
IF OBJECT_ID('dbo.PageTest','U') IS NOT NULL DROP TABLE dbo.PageTest;
GO
-- create the dbo.PageTest table
CREATE TABLE dbo.PageTest (
RowID INTEGER PRIMARY KEY CLUSTERED,
Col1 VARCHAR(1000)
);
INSERT INTO dbo.PageTest (RowID, Col1) VALUES (1, REPLICATE('Row01', 100));
INSERT INTO dbo.PageTest (RowID, Col1) VALUES (3, REPLICATE('Row03', 100));
INSERT INTO dbo.PageTest (RowID, Col1) VALUES (5, REPLICATE('Row05', 100));
INSERT INTO dbo.PageTest (RowID, Col1) VALUES (2, REPLICATE('Row02', 100));
INSERT INTO dbo.PageTest (RowID, Col1) VALUES (4, REPLICATE('Row04', 100));

What we have is a table with 5 rows. The table’s clustered key is the RowID integer column. The rows are inserted so that the odd rows are inserted first, followed by the even rows.

Determine row’s physical location

At this point, let’s look at where the system reports these rows to be at. To do this, we’ll utilize two undocumented system commands. The first is %%physloc%%, which returns the physical location in a hexadecimal format. The second is fn_PhysLocFormatter, which converts this into a format of FileID:PageID:SlotID. This is just simply added to the select clause, so the query is:

Source code

SELECT  RowID, 
        Col1, 
        sys.fn_PhysLocFormatter(%%physloc%%) AS [FILE:Page:Slot]
FROM    dbo.PageTest;

This query produces these results:

From these results, you can see that the rows are all on the same page, and that the slot in the slot array on that page are incrementing for the appropriate RowID value. Remember also that the slot array is zero-based, where the first slot is slot #0. This is the logical order.

How inserts change the slot the row is in

To see this changing around as the rows are inserted, just put the select statement (which I just introduced) after each of the prior insert commands and run the script to create the table and add the rows again. You will initially see RowID=1 put into Slot #0, RowID =3 into Slot #1 and RowID=5 into slot #2. When you then add RowID=2, this needs to be between RowID #s 1 and 3, so #2 is now in slot array #1, #3 moves to slot array #2, and #5 is moved to slot array #3. When you add RowID=4, it gets inserted into slot #3 and RowID#5 is again pushed down, to slot #4. The logical order follows what we are expecting:

Check the physical storage

In order to show that these are not physically stored in this order, we will need to crack this page and look internally at where the data is actually stored. We will accomplish this with this code:

Source code

DECLARE @Page   INTEGER,
       @SQLCMD VARCHAR(MAX);
 
DECLARE @DBCCIND TABLE (
   PageFID         INTEGER,   
   PagePID        INTEGER,
   IAMFID         INTEGER,
   IAMPID         INTEGER,
   ObjectID       INTEGER,
   IndexID        INTEGER,
   PartitionNumber INTEGER,
   PartitionID    BIGINT,
   iam_chain_type  VARCHAR(100),
   PageType       INTEGER,
   IndexLevel     INTEGER,
   NextPageFID    INTEGER,
   NextPagePID    INTEGER,
   PrevPageFID    INTEGER,
   PrevPagePID     INTEGER
);
 
INSERT INTO @DBCCIND 
  EXECUTE ('DBCC IND (CIPageTest, ''dbo.PageTest'', -1)');
 
SELECT @Page = PagePID
FROM   @DBCCIND
WHERE   PageType = 1;
 
SET @SQLCMD = 'DBCC PAGE (CIPageTest, 1, ' + 
              CONVERT(VARCHAR(15), @Page) + 
              ', 3) WITH TABLERESULTS';
 
DECLARE @DBCCPAGE TABLE (
   RowID           INTEGER IDENTITY,
   ParentObject    VARCHAR(255),
   OBJECT          VARCHAR(255),
   Field           VARCHAR(255),
   VALUE           VARCHAR(255));
 
INSERT INTO @DBCCPAGE EXECUTE (@SQLCMD);
 
SELECT DISTINCT Page = @Page, ParentObject
FROM   @DBCCPAGE
WHERE   ParentObject LIKE 'Slot%Offset%';

Which produces these results:

The row’s offset is where the row starts at on the page

As we look at these results, pay close attention to the Offset. This is where the row physically starts on the page. You can see how for RowID #2, that this offset is higher than the offset for RowID #3… and even RowID #5. When the row was added, the data was added to the end of the other existing rows on the page, and the offset where this data starts was entered into the slot array, after having the remaining slot array entries pushed down in order to maintain the correct logical order. We can see this happening once again when RowID #4 is inserted.

And, finally, let’s use DBCC PAGE to crack open this page and look at the raw data. For this, we want to use dump style 2:

Source code

DECLARE @Page INTEGER;
SET @Page = xyz; --<< set to the appropriate page from above
DBCC PAGE (CIPageTest, 1, @Page, 2) WITH TABLERESULTS;

If you expand out the VALUE column, you can see it go from Row01 to Row03 to Row05, then to Row02 and Row04.

So there you go… the data is physically stored in the next available space on the page; however, the page’s slot array is in the clustered key order. Furthermore, we now know that we can look at the offset and determine the order on the page. And we also have yet another way to use DBCC PAGE in an automated manner to show this.

The post Automating DBCC PAGE – Part 3: Is a Clustered Index physically sorted on disk? appeared first on SQL Solutions Group.

You know, it sure would be nice if the manual intervention could be removed and to completely automate the task that you are looking for. This blog post is going to show you how this can be done, and it will give a few examples of doing so.

In my last blog post, I introduced an optional setting to several DBCC commands: WITH TABLERESULTS. This returns the output in tabular format, in a manner that allows the output to be consumed. As it turns out, this optional setting also works with DBCC PAGE. You can see this by examining the database information page (page 9) of the master database:

Source code

DBCC PAGE ('master', 1, 9, 3) WITH TABLERESULTS;

This statement, which can be run without the trace flag, returns the data on the page in 4 columns: ParentObject, Object, Field and VALUE. As it turns out, this particular page of every database has a lot of interesting information on it. Let’s automate grabbing a specific piece of information out of this page:

Source code

IF OBJECT_ID('tempdb.dbo.#DBCCPAGE') IS NOT NULL DROP TABLE #DBCCPAGE;
CREATE TABLE #DBCCPAGE (
       ParentObject VARCHAR(255),
       [OBJECT]     VARCHAR(255),
       Field       VARCHAR(255),
       [VALUE]     VARCHAR(255));
INSERT INTO #DBCCPAGE
EXECUTE ('DBCC PAGE (''master'', 1, 9, 3) WITH TABLERESULTS;');
 
SELECT LastGoodDBCCDate = CONVERT(DATETIME, VALUE)
FROM   #DBCCPAGE
WHERE   Field = 'dbi_dbccLastKnownGood';

This wonderful snippet of code returns the last time a successful DBCC CHECKDB was run against the master database. Whoa… how sweet is that? When you discover a server where the CHECKDB job has been failing to the point that the job only has failed entries in the job history, you can now find out just how long it’s been since it was last run successfully.

The “trick” to making this work is to encapsulate the DBCC command as a string, and to call it with the EXECUTE () function. This is used as part of an INSERT INTO / EXECUTE statement, so that the results from DBCC PAGE are inserted into a table (in this case a temporary table is used, although a table variable or permanent table can also be used). There are three simple steps to this process:

1. Create a table (permanent / temporary) or table variable to hold the output.
2. Insert into this table the results of the DBCC PAGE statement by using INSERT INTO / EXECUTE.
3. Select the data that you are looking for from the table.

By utilizing a cursor, you can easily spin through all of the databases on your instance to get when they last had a successful DBCC CHECKDB run against them. The following utilizes sp_MSforeachdb (which itself uses a cursor to spin through all of the databases) to do just this:

Source code

IF OBJECT_ID('tempdb.dbo.#DBCCPAGE') IS NOT NULL DROP TABLE #DBCCPAGE;
IF OBJECT_ID('tempdb.dbo.#CheckDBDates') IS NOT NULL DROP TABLE #CheckDBDates;
CREATE TABLE #DBCCPAGE (
       ParentObject   VARCHAR(255),
       [OBJECT]       VARCHAR(255),
       Field           VARCHAR(255),
       [VALUE]         VARCHAR(255));
CREATE TABLE #CheckDBDates (
       database_name   sysname,
       LastCheckDB     DATETIME);
 
EXECUTE sp_MSforeachdb '
TRUNCATE TABLE #DBCCPAGE;
INSERT INTO #DBCCPAGE
   EXECUTE (''DBCC PAGE (''''?'''', 1, 9, 3) WITH TABLERESULTS;'');
INSERT INTO #CheckDBDates
SELECT ''?'', CONVERT(DATETIME, VALUE)
FROM   #DBCCPAGE
WHERE   Field = ''dbi_dbccLastKnownGood'';';
 
SELECT database_name,
       LastCheckDB
FROM   #CheckDBDates;

And here we have every database on your instance, and the date that DBCC CHECKDB was last run successfully against it – well, at least for the databases that sp_MSforeachdb didn’t skip (see https://www.mssqltips.com/sqlservertip/2201/making-a-more-reliable-and-flexible-spmsforeachdb/ for more information about this).

In summary, by utilizing the “WITH TABLERESULTS” option of DBCC, we can automate processes that use DBCC PAGE, instead of needing manual intervention to work your way through this.

In the next few installments, we’ll look at other uses of using “WITH TABLERESULTS” to automate DBCC output.

The post Automating DBCC Page appeared first on SQL Solutions Group.

October… the month of Halloween. Spooky events. SQL Server has all kinds of Halloween-ish themed activities like Split Brains, Zombies, Ghosts, Phantoms, and of course Halloween itself. In this post, we’re going to mill about with phantom reads a bit.

Phantoms in SQL Server are actually called “Phantom Reads”. This ectoplasmic phenomenon manifests itself when an identical query being run multiple times, in a single connection, returns a different result set for each time it is run. A Phantom Read is one of the transaction isolation level concurrency events. The read uncommitted, read committed and repeatable read transaction isolation levels may exhibit Phantom Reads; the serializable and snapshot transaction isolation levels are not susceptible to this phenomenon.

Let’s set up a demo to see this in action. First, let’s create our sample data:

Sample Data

USE master; 
GO 
 
IF DB_ID('IsolationLevelTest') IS NOT NULL 
BEGIN 
    ALTER DATABASE IsolationLevelTest SET SINGLE_USER WITH ROLLBACK IMMEDIATE; 
    DROP DATABASE IsolationLevelTest; 
END; 
CREATE DATABASE IsolationLevelTest; 
GO 
 
USE IsolationLevelTest; 
GO 
CREATE PROCEDURE dbo.db_reset AS 
IF OBJECT_ID('dbo.IsolationTests','U') IS NOT NULL 
    DROP TABLE dbo.IsolationTests; 
CREATE TABLE dbo.IsolationTests (
    Id   INTEGER IDENTITY,
    ColA CHAR(1)
); 
INSERT INTO dbo.IsolationTests(ColA) 
SELECT 'A' UNION ALL 
SELECT 'A' UNION ALL 
SELECT 'A' UNION ALL 
SELECT 'A' UNION ALL 
SELECT 'A' UNION ALL 
SELECT 'A' UNION ALL 
SELECT 'A'; 
 
IF EXISTS (SELECT 1 FROM sys.databases WHERE name = N'IsolationLevelTest' AND snapshot_isolation_state = 1)
    ALTER DATABASE IsolationLevelTest SET ALLOW_SNAPSHOT_ISOLATION OFF;
GO 
EXECUTE dbo.db_reset; 
GO

Now, let’s test phantom reads in the repeatable read transaction isolation level. Open the following 2 queries in separate query windows. Run Code Script 1, and within 10 seconds run Code Script 2.

Code Script 1

/* Execute this script from the first query window */
 
USE IsolationLevelTest;
GO
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
BEGIN TRANSACTION;
SELECT * FROM dbo.IsolationTests;
WAITFOR DELAY '00:00:10';
SELECT * FROM dbo.IsolationTests;
ROLLBACK;
SELECT * FROM dbo.IsolationTests;

Code Script 2

-- Run this in query window 2 while Code Script 1 is running
USE IsolationLevelTest;
GO
UPDATE dbo.IsolationTests SET ColA = 'W';
SELECT * FROM dbo.IsolationTests;

The repeatable read transaction isolation level guarantees that there will not be any updates or deletes, so the second query is blocked until the transaction in Code Script 1 has finished. You can see that the second result set is identical to the first one. Once the transaction completes, the update in Code Script 2 can run, and the third result set from Code Script 1 shows that the result set is now changed. However, the repeatable read transaction isolation level still allows inserts, which means that you can still have a phantom read. Let’s see this by changing Code Script 2 to:

Code Script 2 (Allowing Inserts)

-- Run this in query window 2 while Code Script 1 is running
USE IsolationLevelTest;
GO
INSERT INTO dbo.IsolationTests (ColA)
VALUES ('W');
SELECT * FROM dbo.IsolationTests;

Run both code scripts again (Code Script 1 first, and Code Script 2 within 10 seconds). This time, you will see that Code Script 2 completes immediately without being blocked, and when Code Script 1 finishes, it has spawned additional data in its result set. A phantom read in action.

If you want to hide from phantom reads completely, then you’ll need to use either the serializable or snapshot transaction isolation levels. Both of these have the same concurrency effects: No dirty reads, non-repeatable reads, or phantom reads. The difference is in how they are implemented: the serializable transaction isolation level will block all other transactions affecting this data, while the snapshot isolation level utilizes row versions to create connection-specific versions of the table for the transaction – all of these row versions will cause increased activity in the tempdb database. Let’s take a look at how the snapshot isolation level will eradicate the phantom reads. First off, we need to modify the database to accept this isolation level.

Snapshot isolation level setup

ALTER DATABASE IsolationLevelTest SET ALLOW_SNAPSHOT_ISOLATION ON;
GO

To see this in action, we will once again have two queries. Run the code from Code Script 3 in one query window, and within 10 seconds run the code from Code Script 4 in another query window.

Code Script 3

-- Run this code in query window 1
USE IsolationLevelTest;
GO
SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
BEGIN TRANSACTION;
SELECT * FROM dbo.IsolationTests;
WAITFOR DELAY '00:00:10';
SELECT * FROM dbo.IsolationTests;
ROLLBACK;

Code Script 4

-- Run this in query window 2 while the 1st query is running
USE IsolationLevelTest;
GO
INSERT INTO dbo.IsolationTests(ColA)
VALUES ('X');
SELECT * FROM dbo.IsolationTests;

Notice that Code Snippet 2 was executed immediately, however the row that it inserted was not returned in the result set from Code Snippet 1.

And now you know how to dodge phantoms.

The post Dodging Phantom Reads appeared first on SQL Solutions Group.

Just reformat the drives that contain your database data and log files.

If you are running on a version of Windows prior to Windows Server 2008, or if the drives were migrated to a new system without reformatting them, then you really could improve IO performance simply by reformatting the drives. Older versions of Windows will create disk partitions that default to having the boundaries based upon the ancient Cylinder/Head/Sector addressing scheme that was common in disk controllers of that era. Once these boundaries are set, the only way to change them is to reformat the drive, so migrating this drive to a Windows Server 2008 or newer server alone does not fix the issue.

How reformatting the drive affects IO performance

File Allocation Size

On the hard drive, the minimum amount of data that can be read or written is called a sector, which historically was a mere 512 bytes (newer drives may have sector sizes up to 4KB). A group of sectors is a cluster (no not that type of cluster). A drive with a 64KB cluster size (at 512 bytes per cluster) will have 128 sectors, which is also known as the file allocation size. The first 63 sectors of a drive are reserved, or hidden, and include the master boot record (MBR). Simply multiplying the 63 sectors by the sector size of 512 bytes shows that this hidden area occupies 32,256 bytes, or 31.5KB. The drive dutifully starts storing data immediately after this.

SQL Server IO operations

Now, consider how SQL Server performs disk IO operations – an extent at a time (an extent is 8 pages, each of which are 8KB, for a total of 64KB). To store this information on disk, with the default settings, this 64KB will start immediately after the 31.5KB and continue for 64KB… and it will span two clusters. When the drive goes to read this data, it will have to read the first cluster, and then the second cluster, in order to get all of the data needed for the extent being read from disk. When the next extent is read, it will re-read the second cluster, and read the third cluster. The wasted IO operation should be obvious.

How to have SQL Server work better with disk

What we need to do is to offset the beginning of the data being stored on disk to a location more conducive to how the program is operating. This offset is known as the “Partition Alignment Offset”. To be in tune with SQL Server, this value should be an increment of 64KB. However, you also need to consider the entire storage subsystem – the disks, controllers and memory. Starting with Windows Server 2008, this offset is at 1024KB – a nice increment of 64KB that also works very nicely with most RAID disks/controllers. Prior to Windows Server 2008, partition alignment offset was not explicitly performed, so this will need to be performed.

Determining a drive’s partition alignment offset

To determine what the partition alignment offset is for a windows basic disk, there is a very simple wmic command to run:

Source code

wmic partition GET BlockSize, StartingOffset, Name, INDEX

The Index value is the same as the disk number reported in the Disk Management tool, and also from the wmic command wmic Volume. However, we tend to think in terms of drive letters, not drive indexes, so the following PowerShell script will return Partition Alignment Offset for all drives on the system that it is run on and perform the mapping from index to drive letter:

Source code

#
# -------------------------------------------------------------------------------
# Partition Information
# -------------------------------------------------------------------------------
#                                MODIFICATION LOG
# -------------------------------------------------------------------------------
# 2012-08-09 WGS Initial Creation.
# 2012-08-11 WGS Modified TO WORK WITH PowerShell v1.
# 2013-02-05 WGS Modified TO GET Drive Letter.
# 2014-01-27 WGS Modified TO RETURN volume blocksize.
# -------------------------------------------------------------------------------
#
<#
.PARAMETER OpenInNotepad
VALUE passed AS $True or $False. Defaults TO $False.
#>
 
param
(
[BOOLEAN] $OpenInNotepad = $false
)
 
cls
 
FUNCTION Get-DriveLetter($PartPath) {
    #Get the logical DISK mapping
    $LogicalDisks = Get-WMIObject Win32_LogicalDiskToPartition | `
        Where-OBJECT {$_.Antecedent -eq $PartPath}
    $LogicalDrive = Get-WMIObject Win32_LogicalDisk | `
        Where-OBJECT {$_.__PATH -eq $LogicalDisks.Dependent}
    $LogicalDrive.DeviceID
}
 
FUNCTION Get-VolumeBlockSize($PartPath) {
    $Drive = Get-DriveLetter($PartPath)
    IF ($Drive -gt "") {
        #Get the BlockSize OF the volume
        $Volume = Get-WMIObject Win32_Volume | `
            Where-OBJECT {$_.DriveLetter -eq  $Drive}
        $Volume.BlockSize
    }
}
 
FUNCTION Get-PartitionAlignment {
    Get-WMIObject Win32_DiskPartition | `
        Sort-OBJECT DiskIndex, INDEX | `
        Select-OBJECT -Property `
            @{Expression = {$_.DiskIndex};Label="Disk"},`
            @{Expression = {$_.INDEX};Label="Partition"},`
            @{Expression = {Get-DriveLetter($_.__PATH)};Label="Drive"},`
            @{Expression = {$_.BootPartition};Label="BootPartition"},`
            @{Expression = {"{0:N3}" -f ($_.SIZE/1Gb)};Label="Size_GB"},`
            @{Expression = {"{0:N0}" -f ($_.BlockSize)};Label="Partition_BlockSize"},`
            @{Expression = {Get-VolumeBlockSize($_.__PATH)};Label="Volume_BlockSize"},
            @{Expression = {"{0:N0}" -f ($_.StartingOffset/1Kb)};Label="Offset_KB"},`
            @{Expression = {"{0:N0}" -f ($_.StartingOffset/$_.BlockSize)}; Label="OffsetSectors"},`
            @{Expression = {IF (($_.StartingOffset % 64KB) -EQ 0) {" Yes"} ELSE {"  No"}};Label="64KB"}
}
 
 
# Hash TABLE TO SET the alignment OF the properties in the format-TABLE
$b = `
@{Expression = {$_.DISK};Label="Disk"},`
@{Expression = {$_.Partition};Label="Partition"},`
@{Expression = {$_.Drive};Label="Drive"},`
@{Expression = {$_.BootPartition};Label="BootPartition"},`
@{Expression = {"{0:N3}" -f ($_.Size_GB)};Label="Size_GB";align="right"},`
@{Expression = {"{0:N0}" -f ($_.Partition_BlockSize)};Label="PartitionBlockSize";align="right"},`
@{Expression = {"{0:N0}" -f ($_.Volume_BlockSize)};Label="VolumeBlockSize";align="right"},`
@{Expression = {"{0:N0}" -f ($_.Offset_KB)};Label="Offset_KB";align="right"},`
@{Expression = {"{0:N0}" -f ($_.OffsetSectors)};Label="OffsetSectors";align="right"},`
@{Expression = {$_.{64KB}};Label="64KB"}
 
 
$a = Get-PartitionAlignment
 
 
# Display formatted DATA ON the screen
$a | Sort-OBJECT Drive, DISK, Partition | Format-TABLE $b -AutoSize
 
IF ($OpenInNotepad -eq $true) {
    # Export TO a pipe-delimited FILE
    $a | Sort-OBJECT Drive, DISK, Partition | Export-CSV $ENV:temp\PartInfo.txt -Delimiter "|" -NoTypeInformation
 
    # OPEN the FILE in NotePad
    Notepad $ENV:temp\PartInfo.txt
}

If you want to open the results in Notepad, pass in the parameter $True.

You may note that I said that using these wmic commands is for a Windows basic disk. If you are using a dynamic disk, you need to use alternative methods. With Windows Server 2008, you would use the tool diskdiag.exe (in earlier versions of Windows, this was called dmdiag). For either, you need to use the –V flag: dmdiag.exe –V or diskdiag.exe –V.

Changing the Partition Alignment Offset

To change the Partition Alignment Offset on Windows Server 2008 or higher, simply format the drive. On earlier systems, you will need to use the diskpart.exe tool to change the partition alignment offset, and then format the drive. In either case, this is a destructive operation – you will lose all data on the drive.

For more information on disk partition alignment, refer to this TechNet article: https://technet.microsoft.com/en-us/library/dd758814(v=sql.100).aspx.

These steps should yield a noticeable IO performance improvement. Test it for yourself and let me know what you experience.

The post How to improve SQL Server’s IO performance by up to 40% appeared first on SQL Solutions Group.