One of the more intriguing features introduced in SQL Server 2022 is Contained Availability Groups (AGs). At first glance, they look very similar to traditional AGs — but there are some key differences that can trip you up if you’re not prepared.

In this post, we’ll walk through:

• What a normal AG looks like (and its limitations)

• How Contained AGs change the game

• The pitfalls we’ve seen in real-world use

• Practical solutions if you run into these issues

A Quick Refresher: Always On Availability Groups

Availability Groups, introduced back in SQL Server 2012, are Microsoft’s built-in solution for high availability and disaster recovery. They let you replicate databases across multiple instances, managed by a Windows Server Failover Cluster (WSFC).

Key benefits:

• Seamless failover between primary and secondary nodes

• Read-only replicas to offload workloads

• The ability to perform backups and DBCC checks on secondaries, easing the load on your primary

But AGs have always had one big limitation: only user databases are replicated. System databases like msdb and master stay local to each node.

Why does that matter? Think about Agent jobs:

• You have to manually install jobs on every node (and keep them in sync).

• Jobs need to be AG-aware — otherwise they might try to run against a read-only secondary.

• Job history lives only on the current primary, which makes tracking a little messy.

For years, DBAs have worked around these issues, but it’s always been a pain point.

*Enter Contained Availability Groups*

Contained AGs look and act like traditional AGs, but with one important twist: they include their own contained system databases.

For example, if you create a Contained AG called MyAg, you’ll see:

• MyAg_master

• MyAg_msdb

These are replicated to all nodes in the AG, just like user databases.

Why this matters

• Agent jobs now live inside the contained msdb. No more duplicating jobs on every node.

• Job history is replicated. After failover, you don’t lose tracking.

• System tables travel with the AG. Everything stays consistent across nodes.

That’s a big win for manageability and one of the headline features of SQL Server 2022.

Gotchas Seen in the Wild

As with most “new shiny” features, Contained AGs aren’t a silver bullet. There’s a few things to watch out for:

1. No more job offloading to secondaries. Jobs inside the contained msdb only run on the primary. If you used secondaries to handle CHECKDB or backups, you’ll need to rethink.

2. Replication is not supported. If transactional replication is part of your design, contained AGs may not be an option.

3. Seeding is clunky. The GUI doesn’t support manual seeding yet. You’ll need to fall back to T-SQL.

4. No distributed AG support. If you rely on DAGs for hybrid or geo-distributed HA, contained AGs won’t help.

5. Confusion with system databases. When you connect to the listener, the msdb and master you see belong to the contained AG — not the local system copies.

Workarounds & Solutions

For Agent Jobs

If you truly need jobs to run on a secondary node, you still can — but you’ll have to install them in the local msdb outside the contained AG. That means:

• Jobs must be AG-aware again

• You’ll have to maintain them separately

It’s not elegant, but it gets the job done.

For System Databases

This is case-by-case. If your code doesn’t touch master or msdb, you’re fine. But if you do, you’ll need to carefully evaluate whether those objects belong in the contained copies or the traditional system DBs.

Remember: you can query both contained and local msdbs — just make sure your scripts account for both.

Final Word

Contained Availability Groups solve one of the long-standing headaches of AGs — managing Agent jobs across nodes. But they come with trade-offs that every DBA needs to be aware of.

Test carefully. Know where your jobs live. And make sure your disaster recovery plan accounts for these differences.

If you want to see them in action, check out our webinar on SQL Server 2022 features, where we demo Contained AGs in detail.

The post Contained Availability Groups in SQL Server 2022: What You Need to Know appeared first on SQL Solutions Group.

Thank you to our SQL Server Expert and Senior DBA, Rich Benner for providing the valuable Technical info behind this blog. To learn more about Wait Statistics from Rich, check out his recent webinar and two new YouTube videos- an explainer and a demo, on our channel.

What Are Wait Stats?

Anytime something inside Microsoft SQL Server is waiting for a resource, it’s assigned a wait stat. For example: 

• A query waiting for CPU or memory resources

• A query waiting to access a data object like a data page or table

• An internal SQL Server process waiting to complete a task

Some waits are completely normal. For example, in Availability Groups, certain threads simply wait to synchronize data with secondary nodes. That’s expected behavior and not necessarily a sign of trouble.

Key Things to Know About Wait Stats

1. They’re cumulative – Wait times are aggregated since the last time the data was cleared, usually at the last server restart (though other events can reset them).

2. They measure time in seconds or milliseconds – depending on the stat you’re viewing.

3. They’re invaluable for troubleshooting – By looking at wait stats, you can quickly zero in on what’s slowing down your system.

How to Find Wait Stats

The primary DMV for retrieving wait stats is:

SELECT * FROM sys.dm_os_wait_stats;

On a SQL Server 2022 instance, you might see over 1,300 different wait types. Even seasoned DBAs only know a fraction of them by heart. That’s where resources like the SQLskills Wait Types Library come in handy. This community-maintained list covers around 95% of known waits, detailing whether they’re benign or problematic and suggesting possible resolutions.

Useful Tools for Analyzing Wait Stats

• sp_WhoIsActive – An enhanced version of sp_who2 that shows what queries are running, their process IDs, and any current waitss.

• sp_BlitzFirst – A free tool that captures your current wait stats along with other diagnostic information.

• Custom Queries – Many DBAs have their own scripts for retrieving & formatting wait stats. A quick search for “Wait Stats query” will yield plenty of examples.

The Top 5 Most Common Wait Types 

1. LCK_* – Lock Waits

Occurs when one query locks a resource (e.g., a data page) and another query has to wait for it to be released. Example: An UPDATE query locks a row, preventing a SELECT query from reading it until the update completes.

2.

CXPACKET / CXCONSUMER

– Parallelism Waits

Happens when a query is split across multiple CPU cores. If one thread finishes before the others, it waits for the slowest one to catch up. Normal in moderation, but excessive CX waits can indicate an inefficient parallelism setup.

3.

ASYNC_NETWORK_IO

– Application Processing Delays

Indicates SQL Server has finished processing results but is waiting for the application to consume them. Often caused by applications retrieving rows one at a time instead of in batches.

4.

LATCH_* and PAGE_LATCH_* – Structure & Page Locks

• LATCH_*: Locks on internal structures (not data pages)

• PAGE_LATCH_*: Locks on data pages themselves. These can signal contention in memory structures or data access patterns.

5.

WRITELOG

– Transaction Log Bottlenecks

This occurs when SQL Server is waiting to write to the transaction log. Common causes include slow disk performance, excessive write activity, or log file contention.

Why Wait Stats Matter

Wait stats provide a high-level overview of where SQL Server is spending its time waiting.

They help you answer questions like:

• Is my performance issue due to locking, disk speed, or parallelism?

• Is the problem in SQL Server or in the application consuming data?

• Are my hardware resources (CPU, memory, storage) configured optimally?

By combining wait stats with tools like sp_WhoIsActive and sp_BlitzFirst, you can pinpoint bottlenecks faster and focus your tuning efforts where they’ll make the biggest impact. Not all waits are bad — many are just part of normal SQL Server operations. The key is knowing which waits are worth investigating and which you can safely ignore. With a solid understanding of wait stats, you’ll be better equipped to diagnose issues, improve query performance, and keep your SQL Server running smoothly.

Need help interpreting your wait stats?

At SQL Solutions Group, we work with clients every day to identify and fix performance issues. Contact us for a health check and start optimizing your environment today.

The post Understanding Wait Stats in SQL Server: A Practical Guide appeared first on SQL Solutions Group.

The post Safeguarding Your Data: Navigating SQL Backup, Compliance, Maintenance, and Support Challenges appeared first on SQL Solutions Group.

Anyway, in this blog post I want to cover a technology for copying databases across SQL Managed Instances that saved my posterior recently. Well, it didn’t really save it, but it sure made my life easier. We have a client who uses Azure SQL Managed Instance for both their production and test environments. I think this is awesome.

As a quick background, Azure SQL Managed Instance is a full SQL Server instance served up as a PaaS service. Unlike Azure SQL Database where you are only working at the database level, Managed Instance gives you near 100% instance compatibility level capabilities with the latest version of SQL Server Enterprise Edition SQL Server database engine. SQL Managed Instance provides the lift and shift capabilities of on-premises databases with minimal database changes.

So, back to our customer. They essentially lifted and shifted their on-premises databases to Azure SQL Managed Instance and have been using it successfully for nearly two years. Again, this is awesome.

Last week they came to us and asked about reporting with Managed Instance. They were looking at data marts and data warehouses, but we needed more information. It turns out they have some people that just want the ability to query the databases, and potentially hook up Excel to these databases for data analysis.

The caveat is that the people I was talking to didn’t want to give the other group direct access to the production environment. Toootally get that. Yeah, like 100% get it. So, what are the options?

After some discussion we decided to give the group wanting to run reports access to the test Managed Instance environment. We’ll just refresh the test environment data with the production environment data on a schedule. For them, refreshing data nightly is OK for now.

So, the question then becomes, how do you get data, or copy a database, from one SQL Managed Instance to another SQL Managed Instance? There are two ways, and I’ll discuss both here.

PowerShell cmdlets

Microsoft released some Powershell cmdlets that make copying data across SQL Managed Instances simple.

There are essentially 3 cmdlets:

• Copy-AzSqlInstanceDatabase – Starts an online copy operation.
• Get-AzSqlInstanceDatabaseCopyOperation – Returns copy operation details.
• Complete-AzSqlInstanceDatabaseCopy – Completes the copy operation.

The Copy-AzSqlInstancesDatabase cmdlet starts an online copy operation of a database across managed instances using Always On availability group technology. Copy-AzSqlInstanceDatabase (Az.Sql) | Microsoft Learn

The Get-AzSqlInstanceDatabaseCopyOperation cmdlet returns database copy operation details. You can call this cmdlet in a loop so you can track the status of the copy. Get-AzSqlInstanceDatabaseCopyOperation (Az.Sql) | Microsoft Learn

The Complete-AzSqlInstanceDAtabaseCopy cmdlet completes the copy operation of the database and makes it available for read/write workloads. Complete-AzSqlInstanceDatabaseCopy (Az.Sql) | Microsoft Learn

You don’t have to call the Get-AzSqlInstanceDatabaseCopyOperation cmdlet as you could just call the Copy and Complete, but it is HIGHLY recommended you call it so you know when to call the complete.

The way this works is you start by calling the Copy-AzSqlInstancesDatabase cmdlet. This starts an online copy operation of one or more databases. Once the copy option starts, data is being seeded from the source to the target. During the process, you can call the Get-AzSqlInstanceDatabaseCopyOperation cmdlet to check the copy operation status, either in-progress or completed.

Once the copy operation has completed, the status will be copy ready for completion. Within the next 24 hours the user can complete the database copy operation by calling the Get-AzSqlInstanceDatabaseCopyOperation cmdlet. Note that until the Get-AzSqlInstanceDatabaseCopyOperation cmdlet is called, all the changes from the source are being replicated to the target database.

When Get-AzSqlInstanceDatabaseCopyOperation is called, the link between the source and target databases is broken, the target database becomes online and ready.

A very simple example would be something like this:

Source code

$dbname = "db1"
$sourceMI = "sqlmi1"
$targetMI = "sqlmi2"
$sourceRG = "rg1"
$targetMI = "rg2"
 
Copy-AzSqlInstanceDatabase `
    -DatabaseName $dbName `
    -InstanceName $sourceMi `
    -ResourceGroupName $sourceRg `
    -TargetInstanceName $targetMi `
    -TargetResourceGroupName $targetRg 
 
While(true$)
{
 
    Start-Sleep -Seconds 10
    $operation = Get-AzSqlInstanceDatabaseCopyOperation `
    -DatabaseName $dbname `
    -InstanceName $sourceMI `
    -ResourceGroupName $sourceRG `
    -TargetInstanceName $targetMI `
    -TargetResourceGroupName $targetRG
 
    try
    {
        $copysucceeded = $true
        Write-Host "Status of copy for " $dbname " is " $operation.State
        if (!($operation.State -eq "Succeeded")) {$copysucceeded = $false;}
    }
    catch { $copysucceeded = $false; Write-Host "Didn't work" }
 
    if ($copysucceeded)
    {
        Write-Host "Copy completed"
        break
    }
    else { Write-Host "Still copying..." }
 
}
 
Complete-AzSqlInstanceDatabaseCopy `
    -DatabaseName $dbName `
    -InstanceName $sourceMI `
    -ResourceGroupName $sourceRG `
    -TargetInstanceName $targetMI `
    -TargetResourceGroupName $targetRG

There is a GREAT example here: Using and automating Azure SQL Managed Instance database copy and database move operations with Azure PowerShell (microsoft.com)

The example copies multiple databases, but if you are needing to copy only one, you can tweak it.

Oh, by the way, you can also do the copy via the Azure portal.

Backup / Restore

The other option is backup and restore, meaning, back up to Azure blob storage then restore from there. Since Managed Instance provides access to the SQL Agent, a job is created on both managed instances; one to back up, and one to restore.

This option gets a bit tricky because in the scenario for our client, and I would think in most environments, you want to restore the database in the target managed instance with the same name, and have the backup file the same as well, just to make it easy.

The backup job is simple: back up the database to Azure, ensuring the backup file name is always the same name. The restore job is just as simple: delete the database and then restore the database with the same name.

However, the issue in this scenario is that when you back up to the same file name, it will error because the file already exists. When we tested we thought, “no problem, we’ll just use PowerShell (Remove-AzureStorageBlob) to remove the file as the first step in the job, then the second step in the job will be to back up the database!”

Unfortunately, Managed Instance does not support PowerShell job steps. Sigh. So, we have an on-premises job (or it can even be a Windows Task Scheduler job) that runs the delete. At this point it is just a timing thing. Since the client only wants to back up and restore a single database, we run the backup job at 1 a.m., for example. Then 30 minutes later we run the restore, then another 30 minutes later, we run the delete blob job.

For this solution you’ll need to create a storage account in Azure, then create a CREDENTIAL in SQL Server on both Managed Instances:

Source code

CREATE CREDENTIAL [https://mystorageaccount.blob.core.windows.net/mycontainer]
WITH IDENTITY N'SHARED ACCESS SIGNATURE',
SECRET = N'<storageaccountkey>'

Once that is created, simply back up the database.

Source code

BACKUP DATABASE [MyDB] TO URL = N'https://mystorageaccount.blob.core.windows.net/mycontainer/MyDB.bak'
	WITH BLOCKSIZE = 65535, COPY_ONLY, NOFORMAT, NOINIT, 
	NAME = N'MyDB-Full Database Backup', NOSKIP, NOREWIND, NOUNLOAD, STATS = 10

Or, if you are using Ola Hallengren’s solution (which we are a major fan of):

Source code

EXECUTE [dbo].[DatabaseBackup]
@Databases = 'SentryOne',
@URL = 'https://mystorageaccount.blob.core.windows.net/mycontainer',
@Credential = 'https://mystorageaccount.blob.core.windows.net/mycontainer',
@BackupType = 'FULL',
@Verify = 'N',
@CheckSum = 'Y',
@LogToTable = 'Y',
@Compress = 'Y',
@CopyOnly = 'Y'

Then restore onto the other Managed Instance:

Source code

RESTORE DATABASE [MyDB] FROM URL = N'https://mystorageaccount.blob.core.windows.net/mycontainer/MyDB.bak'

Yeah, a bit cumbersome, but it works and runs, and solves the client’s problem.

Which?

As in all things SQL, it depends. For our client, the simple backup and restore was the solution. They were OK with data being up to 24 hours old for their reporting in this scenario.

If using the PowerShell cmdlets, the target database is unavailable for connections until you issue the Complete cmdlet. But it lets you copy multiple databases, not just one.

It really just comes down to what you are attempting to do. Honestly, for the customer, we could have easily used the PowerShell cmdlets to copy the database each night (deleting the database on the target Managed Instance before starting the copy) and that would eliminate the need to delete the file from blob storage.

So if you looking for ways of copying databases across SQL Managed Instances, you have several options.

The post Copying Databases Across SQL Managed Instances appeared first on SQL Solutions Group.

One potential issue here is the lack of time to delve really deeply into each programming language at one’s disposal. The thing about SQL Server is that it is a language that performs very differently than, say, C#. Those languages are row based, which means it’s common to iterate through a data set and process each row individually. This works quite well in those languages, but quite terribly in SQL Server. Being a set-based language, SQL Server prefers to deal with the whole data set at once.

Because of these nuances, it’s easy for developers to make mistakes when creating SQL server code. Let’s go through some common errors that we see.

T-SQL

Let’s start with basic T-SQL code that we commonly see, which are either not great for performance or have other considerations that are often overlooked.

Select *

You’re probably not surprised that we see this a lot. This is often used because the person writing the code isn’t sure exactly what they’ll need to return from the object they’re querying. Using Select * is not a great idea for a several reasons. Here are things to consider:

• Do you really need all of the data returned? Supporting indexes for queries like this are likely to be wider than necessary in order to cover all of the columns in the table. Wouldn’t it be nice if your indexes were smaller? They’d take less disk space but more importantly would require less I/O to retrieve and occupy less space in memory.
• Do you need to return that nvarchar(max) field that you’re got in your table? Not sure if you’ve got one?
• What if the underlying structure changes and somebody adds in a (max) field? Do they have to check with you first or will your query suddenly tank and cause you and your users a headache? Wouldn’t happen if we were selecting what is strictly necessary for our query.

One of the valid uses of SELECT * is when using something like WHERE EXISTS, where you don’t actually need to return any of the data in the query, something like this:

You could even replace that with a SELECT 1 to be more explicit what you’re doing. This is very much personal preference though.

Select Into

Using SELECT..INTO seems like a really easy way of creating tables (whether temporary or permanent), as it takes out a lot of the leg work in defining the structure. An obvious issue here is that your object is created with no constraints (should it be a unique primary key?) and no indexes (do you really want this object to be a heap table?). It also makes troubleshooting more difficult, as if you have something like a truncation issue then you can’t easily see the definition of your object without creating it and going to look for it (do you really want to have to run this code before you can complete the code review?).

Overall, it’s much more controlled if you spend the small amount of time required to declare your object explicitly. You can then make sure it’s exactly what you need and it’s much easier to maintain for those that follow.

Implicit Conversions

We often come across implicit conversions when analysing a server’s most poorly performing queries. This happens when you are comparing two pieces of data that are not the same data type. This could be comparing a column to a variable, or it could equally be joining two tables together. If these fields are not the exact same data type, then SQL Server is going to have to perform an implicit convert and convert one to match the other.

This can cause a couple of separate issues:

1. If the server has to convert a column in a table to a different data type then it is not going to be able to use any statistics on the table. These statistics are used when creating the execution plan and will inform how the query is actually run. Without these stats SQL Server can easily make a bad estimate and cause the execution of this query to run very poorly.
2. The other reason that this will be slow is that SQL Server will have to scan one of the tables in its entirety and convert this field before it can even consider joining the tables together. The larger the underlying tables get, the worse performance becomes.

The best way to avoid this issue is to ensure that your data types match. Whether that be making sure your variables match the field or if you are joining two tables on a certain field then making sure that the data types match. Also, remember, varchar(10) is not the same as nvarchar(10); they need to be the exact same data type.

Obfuscation

One of the most common issues we see is code obfuscation. What we mean by this is abstracting some code out of what we’re doing and storing it elsewhere. Doing this makes sense to a certain extent. Why write the same code more than once? If we’re going to use it more than once, why not create an object that we can use everywhere to do this? While this sounds like an excellent idea, let’s discuss why it might not be as great as it first appears

Nested Objects

Views

We see so many times that people will write a view to perform some logic and then use it everywhere. Simple right? Well, this grows over time. It may be that this is fine for now, but for a certain subset of queries you want to tweak that a little, so a view is created over the top of this view. This can very easily snowball and then you have views relying on views … relying on views … relying on views (you get the idea).

Primary issues:

1. Changing logic can be fraught with danger. What if we have to tweak one of these nested views? Do we have the ability to test every single place where this view is used? Do we even know everywhere it’s used?
2. Troubleshooting can be an absolute nightmare. You’ve got some incorrect data in the final query, but which of the sub views is it coming from? It makes a spider’s web of dependencies that are very time consuming and difficult to track. How much time can you invest in resolving this?
3. When creating execution plans, SQL Server will only spend so much time investigating these sub views before simply giving up and just running the query however it guesses with no available data. Any ideas how well that’s going to go and how our query is going to perform? Not well.

While views make a lot of sense when used properly, being aware of the nested views issue is something that could save you some major headaches in the future.

Scalar Functions

We see so many functions being used in the wild. Some work well, some not so much. By far the largest problem we come across is the (miss)use of scalar functions. Using a function is extremely common in other languages and they work very well. But there are a couple of very large “gotchas” in SQL Server:

1. As soon as you use a scalar function in SQL Server, it forces the transaction to run serially. This means you’re removing any ability for the query to run in parallel, and it therefore will likely dramatically impact the performance of your query. There are some advances here in SQL Server 2022 with Scalar UDF Inlining, but this is a relatively new feature and is not available for everybody.
2. Functions are also effectively treated as separate transactions, once called. You won’t see them in execution plans and this makes them extremely difficult to locate, investigate and troubleshoot. It’s super easy to miss them all together.
3. Scalar functions are also executed once per row of the data set. Remember how SQL Server is a set-based language and likes dealing with a large data set all at once? Well, scalar functions will pull that particular rug out from under you and turn it into a row-based calculation which will have those very same performance issues.
4. You also have the recurring issue of knowing exactly what the function is doing. What parameters are being passed? What’s the actual code that the function is executing? These questions can often be tricky to answer.

In our experience, scalar functions are fine when used correctly (sparingly and not in set-based operations. Think setting a local variable using a function, but it’s extremely easy to use them incorrectly and shoot yourself in the foot using them.

IN (Subqueries)

Something that is very easy to do is to run a query.

Using the Stack Overflow database, let’s say I wanted to see all posts for all users with more than 10,000 reputation points. I could write something like this;

Seems pretty simple, right? Well, the issue here is what goes on behind the scenes. That IN statement gets parsed out to something like this;

In the example query here, there would be 25,334 clauses in this statement. That’s a lot of data to parse through and that just won’t scale well within SQL Server. Joins would be the correct way of doing this. Whether a direct join is possible or whether the subquery is better to be separated and the result put into a temp table, that will all depend on the logic of your particular query.

Triggers

Aah, triggers, the silent killers. Sometimes necessary, always costly. The amount of times that we’ve seen poorly performing servers and there are triggers firing all over the place is definitely a non-zero number. They have the same issue as scalar functions, in which they are treated as a separate transaction and won’t be found in execution plans and the like.

Sometimes triggers are necessary, but it’s so easy to introduce them unnecessarily and then suffer the performance impact that follows. Need to do something upon or after insert? There are other ways of resolving those requirements without resorting to Triggers.

Synonyms

We’re glad that a lot of people don’t even know that synonyms are a thing in SQL Server, but when we do find them they can cause major confusion. While we totally understand that the business will often push for the simplest (cheapest) solution to resolve the issue of object renaming, it’ll definitely cause confusion when troubleshooting and ultimately become a maintenance issue.

Linked Servers used Incorrectly

Linked servers can be highly effective when used correctly. The issues that occur are with understanding how linked servers execute queries locally and remotely. If we just include a linked server using four part naming convention ([RemoteServerName].[Database].[Schema].[Object]), then our local server retrieves all data from the remote server and processes data locally. This may be fine if we’re doing a simple query, where we’re selecting all data in a single table. But once we get to adding in filtering and/or manipulating the data in any way (joining for example), then this can get inefficient very quickly.

There has been some change in recent versions of SQL Server on how it deals with these types of connections. Ultimately the better way of processing this data is to either push your results from the remote server and then deal with these results, or to use OPENROWSET. This avoids the issues above and represents best practice for querying linked servers.

Many Database Problem

For companies who are vendors or provide a service to multiple clients, it is not unusual to see a database created for each customer. The thinking here is that it silos each customer’s data into a separate database to ensure there is no risk of data being accessed by the wrong user. This makes sense, right? Sure, but there are definitely some issues that can arise from this:

1. Standardisation is a big one. Can you guarantee that your databases are identical to each other? Are they even meant to be? If not, you will have issues performing operations that have to update all databases. Do you need to push a maintenance patch or hotfix? What if some databases have incompatible schemas? You’re going to run into a big headache here really quick.
2. Context switching is absolutely an issue you should be considering in this scenario. Because the engine has to switch between databases constantly, it can put a hidden workload onto the server and cause some big issues if not monitored very closely.
3. Maintenance is also another thing to consider. Many databases will require many backups, will contain many indexes, and statistics that require maintenance (you’re doing this, right?). Everything becomes a list that needs sorting through before any potential issues can be resolved.

The two main options we have here are to consolidate your customers into fewer databases (we’re aware of how much work this can be to implement) or to simply split your server into smaller servers and host fewer customers on each. In today’s world of virtualisation, this is a lot simpler than it used to be. You just need to monitor them to ensure you don’t have all of your most busy customers on the same server.

Many Tenants on a Host

This one is tricky without having access to the virtualisation layer. We have absolutely seen issues with over provisioning hardware between multiple tenants. It seems obvious that you should consider doing this. Each server is not going to be using 100% of the resources allocated to it, so why not allow them all to have a bit more so you can utilise your hardware?

Let’s say we have 20 CPU cores and 1TB of ram on a server and we have 10 tenants on this host. If we were hard provisioning, we would allocate two CPU cores and 100GB of ram to each of these tenants. That way, everybody always has everything they want. But then we notice that the resources are not often used — we sit around with spare CPU going idle. Why not put a few more servers on this box so we can save money? Makes sense, right? The directors would love if we can provide a cost saving.

The issue comes when you have your busy periods, whether that be the holidays, or whether that’s Monday mornings when your customers come online. What then? Well, the tenants are going to fight for those resources. Your servers are going to think they each have two cores and can use them. But because we’ve now got 20 tenants on this host, they can not necessarily have that many resources at any time.

Seeing this inside SQL Server is extremely difficult and even experienced troubleshooters would find it hard to find that this was the problem.

A similar issue is that of “noisy neighbours”. What if one particular tenant on the host uses most of the resources? There’s not much the other tenants can do about it. They’re going to struggle to do what they need to and there’s nothing they can do about it.

Keeping Connections Open

When connecting to SQL Server, you’ll likely do multiple things, so it makes sense to keep the connection open until you finish the entire workload, right? That can come with some very expensive lessons to learn if this becomes commonplace in your environment.

There are a total of 32,767 user connections available in SQL Server. What happens when these are all used up? You simply cannot make new connections to the server. I will leave the consequences of that to your own imagination.

It’s very much best practice to only keep a connection open for as long as you absolutely require it. The resources required to open a new connection are negligible, true. But the long term consequences of keeping them open can be drastic. It’s also worth noting that you may not have an issue with this right now, but as databases and user counts grow, do you want to risk this or get it right from the beginning?

Debug Mode

It’s rare that a company will add the option to be able to debug SQL code. You may have the error message that something’s throwing, but where specifically is that error coming from? Most of the time somebody will need to extract the T-SQL code piece by piece and run it until you get to the problem code. This can be extremely time consuming and is luckily something that is absolutely avoidable.

By adding a @debug variable and the related code, it will save you so much time. The idea of @debug is that when you enable the flag, it will log relevant information into a logging table so you can analyse it separately.

Adding the variable and related logging isn’t free. It’ll take a little time to ensure it works, but by getting to the standard where code has a debug option, you will make the long-term viability and maintenance so much easier.

The post SQL Server Antipatterns: Common Mistakes with SQL Code appeared first on SQL Solutions Group.

We’re two weeks into 2024 and I’m sitting here staring at my carousel M&M dispenser (it used to be full) of dark chocolate peanut M&Ms. Some people collect hats, stamps, or Lego sets. I collect M&M dispensers. I probably have 50 of them (and the dark chocolate peanut M&M is the ONLY way to go, btw), and I’m wondering what 2024 will bring.

And yes, that is a mic propped up on a mini Starbucks cup because my laptop mic sounds like crap. So I hooked that thing up, but it lacks a stand, so I’ll be ordering one of those.

Anyway, back to 2024.

I don’t want to turn this blog into a “look back at 2023” or “what could have been” post. I want to look forward. I mean, it’s 2024 and, barely two weeks in, a lot has happened. Microsoft has already released CU 11 for SQL Server 2022. Pete Carroll and Bill Belichick (two GREAT NFL coaches) were let go, and Nick Saban retired (a phenomenal college coach), all in the span of 24 hours.

Plus, for the past few days my laptop has been popping up messages that my Norton antivirus software has expired (and others), and I am not even running it. Looks like I’ll be paving my laptop this weekend. Ugh, welcome to 2024.

But honestly, I’m looking forward to 2024. From many perspectives, I think we are going to see great advancements, including in technology, that on one hand will make our lives easier, but also make our decisions a bit harder. My dad died last July, a week before his 90th birthday, and for the few months following, my bothers and I helped Mom clean and organize a lot of my dad’s things.

He had a lot of tools. A lot. It took us days just to go through his tools. I brought some back home with me. My dad had a really nice snow blower, because where he lived it snowed quite a bit, and he used it to clear the pathways of not only his house, but his neighbors’. He enjoyed bundling up and firing up his trusty mega snowblower 9000 (no, it’s not really called that), making mounds and piles of snow by clearing the sidewalks and walkways of his neighbors.

Where I live, it doesn’t snow that much, so I’m torn. I’d love for it to snow enough to justify firing up the snow blower, but I also DON’T want it to snow that much. When it does snow (like it did the other night), I’m looking at my inherited snow blower and snow shovels and wondering what the best tool for the job would be. My neighbors would probably get a kick out of seeing me fire that thing up for only a 1/8 of an inch of snow, which, if I just waited a couple of hours, would be gone anyway. *Sigh*

So, here I sit, two weeks into 2024, pushing the button on my carousel M&M dispenser waiting for it to drop out those precious dark chocolate peanut M&M’s, thinking about the upcoming projects and the technology available to me.

Do I use SSIS or Azure Data Factory? Would a particular client benefit from a distributed availability group? Why isn’t Query Store enabled at this other client? What about AI (have you seen the movies M3GAN and Moonfall?. Um, no.) I’m kidding a bit, because I just read a great blog post by Daniel Janik on how to use AI to generate test data in SQL Server.

Anyway, make 2024 a great one. Help others, be kind, play with awesome technology, and eat a bunch of dark chocolate peanut M&Ms (preferably out of a carousel dispenser).

The post The Right Tool for the Right Job appeared first on SQL Solutions Group.

When out in the log files, there arose such a clatter,
I sprang from my desk to see what was the matter.
Away to the console, I flew like a flash,
Entered some commands, my heart in the trash.

The server lights flickered, a troubling sign,
In the midst of the chaos, the disk space declined.
When, what to my wondering ear should I hear,
A reassuring voice that allayed my fear.

“Now deadlock! Now timeout! Now, replication delay!
 On, fragmentation! On, failed backup today!
 To the top of the stack! To the cause of the stall!
 Now dash away, dash away, dash away all!”

And then, in a twinkling, I heard on the drive,
The whirring and clicking, as if barely alive.
As I scratched my head, and was turning around,
Down the Ethernet cable, St. Randy came with a bound.

He was dressed all in code, from his head to his foot,
And his clothes were all tarnished with bytes and soot.
A bundle of patches he had flung on his back,
And he looked like a troubleshooter, just opening his pack.

He was cheerful and spry, a jolly old DBA,
And I laughed when I saw him, in spite of the fray.
A wink of his eye and a nod of his head,
Soon gave me to know I had nothing to dread.

He spoke not a word, but went straight to his work,
And fixed all the queries, then turned with a jerk.
And laying a finger aside of his nose,
With a nod, up the Ethernet cable he rose.

He sprang to his sleigh, to his team gave a shout,
And away they all flew, like a backup job, no doubt.
But I heard him exclaim, as he drove out of sight,
“Happy SQL to all, and to all a good night!“ 

The post A Visit from St. Randy: ‘Twas the night in the server room, silent and cold … appeared first on SQL Solutions Group.

The good news is that this data isn’t often needed at all or is only needed very infrequently. Therefore, we can use a few approaches to make your database simply perform better and utilize fewer resources. Wouldn’t it be nice if your queries ran faster and processes like your backups had to do less work because your database is just smaller?

We can get there! In this post, we’ll look at a few different things you can implement in order to improve your database’s health.

Unused and Redundant Data

The first and most obvious way to slim down your database is to find unused data and simply delete it from the database if it’s not needed any more. Do you have a table where you have a large amount of data that’s old and just not used? Is there a reason that we’re keeping it around?

There are some obvious blockers to this. Perhaps you deal with data that you have to keep for a certain amount of time due to regulatory reasons (financial, medical, etc.). If that’s your scenario, you’ll want to look at some of the other options on this list.

Do remember, though, that the requirement to store data does not necessarily mean that it needs to be attached to a live server and accessible at all times. In some cases, backing up your database and storing the backup is completely acceptable to meet the requirement. Considering how cheap blob storage is nowadays, this could be a very cost-effective way of being compliant with data retention rules.

Archive Tables

Is the case above true but sometimes you do need to query the historical data? You could consider an archive table. These are usually the same table name followed by _Archive or _History and used for storing data that still needs to be “queryable” but not used for most queries.

This will require code changes if you’re accessing the data via an application. If you require the older data then you’ll need a UNION ALL to join the two tables together. You’ll also want to watch out for any schema changes to either database as you’ll want to keep them the same for performance reasons.

The advantage of this approach is that your most commonly executed queries that deal with the latest data simply have less work to do. You have less data to read from disk, less data to store in memory, and less data to join to other data.

Another trick you could apply is to use different filegroups for the archive tables. When you do that, you can put the archive data on a drive that isn’t your nice, expensive, fast disks. It’ll be slower to retrieve, but with this data, does that matter?

Partitioning

If you don’t want to delete this data or move it to another table, then you could consider partitioning your tables. This process can be tricky to get right so definitely research this option, but when it does work well it can be an excellent solution.

Partitioning is the process of separating your data into segments between given values with the idea being that you only query the range that contains the data you need. Dates are often used for this. For example, people might partition the last years’ worth of data by month (so 12 partitions), and then have anything older than that sit in a large partition dedicated to historical data. This allows for queries that query this month’s data to just deal with a small fraction of the data in the table, which is great for performance.

Important to note: There is usually maintenance required on this, however. You probably don’t want to make 100< partitions if you were going with monthly data — in other words, having a partition for each month from the start of time, for this table, and allowing for the next 10 or 20 years.

The better approach is to have a monthly SQL Agent job that deletes your oldest partition (and moves the data into the ‘historical’ partition) and then makes a new one for the next month. This also applies if you’re partitioning based upon a surrogate key (e.g. splitting it into ranges of a certain number of IDs).

The advantage of partitioning is that if your queries always search on a field that uses the partitioning key then you don’t need to make any code changes in the app to take advantage of this feature!

Filtered indexes

Being able to use filtered indexes is a godsend when it comes to dealing with certain types of data. One example we see a lot is the use of soft deletes where a table contains a column called IsActive or IsDeleted (or similar) and a bit field used for this data. Often when queries are written against these tables, they’ll always search for IsActive = 1 or IsDeleted = 0 so they’re only looking at live data. This makes them an ideal target for filtered indexes.

Filtered indexes act exactly like a normal non clustered index but with the addition of a filter. It will look something like this:

Source code

CREATE INDEX IX_IndexName
ON TableName (TableIDField, IsActive)
WHERE IsActive = 1

It’s important to realize that you want the filtered column in the indexed fields for best performance and to ensure the optimizer chooses to use this index.

Reduced Index Width

Another issue that we see more than we’d like is the use of very wide indexes. Every field in your index costs extra storage space and increases your memory requirements. It’s very easy to find a list of indexes that have a large amount of columns. Just use a query like this:

Source code

SELECT 
     t.name TableName,
     i.name IndexName,
     i.index_id IndexID,
     COUNT(col.name) ColumnCount
FROM sys.indexes i 
INNER JOIN sys.index_columns ic ON  i.OBJECT_ID = ic.OBJECT_ID and i.index_id = ic.index_id 
INNER JOIN sys.columns col ON ic.OBJECT_ID = col.OBJECT_ID and ic.column_id = col.column_id 
INNER JOIN sys.tables t ON i.OBJECT_ID = t.OBJECT_ID 
WHERE i.is_primary_key = 0 
	AND i.is_unique_constraint = 0 
	AND t.is_ms_shipped = 0 
GROUP BY t.name, i.name, i.index_id
ORDER BY COUNT(col.name) DESC

That will return a list of all of your indexes and the number of columns they include, sorted descending by column count. You’ll then have to look at these indexes, look at the queries that hit these tables (query store can be excellent for that) and decide if you can reduce the number of columns. The biggest wins will be the largest data types so look out for things like (max) fields and ntext.

Fragmentation

If you don’t have index maintenance scheduled, you could well have some very fragmented tables. This is caused by data deletions and updates, which can happen a lot on heap tables too. It’s easy to check out your fragmentation using a query like the one below:

Source code

SELECT 
	T.name TableName,
	I.name IndexName,
	ps.avg_fragmentation_in_percent,
	ps.page_count
FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS ps
INNER JOIN sys.tables t ON ps.OBJECT_ID = t.OBJECT_ID 
INNER JOIN sys.schemas s ON t.schema_id = s.schema_id
INNER JOIN sys.indexes i ON i.OBJECT_ID = ps.OBJECT_ID AND ps.index_id = i.index_id
WHERE I.name IS not null
	AND ps.avg_fragmentation_in_percent > 0
	AND ps.page_count > 1000
ORDER BY ps.avg_fragmentation_in_percent DESC

This query returns all fragmented indexes, as long as they’re over 1000 data pages (8MB) in size. If you have some large and fragmented indexes, then you’ll want to do some reorganizing or rebuilding. You’ll then want to set up regular maintenance. To do this, we recommend using Ola Hallengren’s index maintenance solution.

Compression

There are some excellent options for compressing your data — depending on what your data is exactly, because some data is more compressible than others.

Row and page compression can be an effective way to reduce the size of your tables. There are some considerations here, including SQL Server editions (e.g. all versions of SQL Server 2022 support data compression, but if you’re on 2016 then you need to be on Service Pack 1 or later).

Columnstore compression is a completely different way to store and compress data. As the name would suggest, this doesn’t compress data row by row. Rather, it compresses columns worth of data. This is excellent if you have a lot of duplicated data within columns of your table. This is particularly useful in data warehouse solutions. The downside with columnstore is that in an OLTP workload environment, any inserts into or updates to this table will have to do more work due to the compression. This very much depends on the data and your workloads.

Conclusion

This is not an exhaustive list of all the options, but you should now have a great start and some ideas of what you can do in your own environment. Smaller data is faster data, after all, so let’s try and get your database running like greased lightning. If you’re reading before it happens, you can also join me for a webinar on this topic scheduled for Wed., Jan. 24, 2024. We’ll cover many of the same topics — just more in depth and with demos and additional useful insights.

If you want a hand with any of this, then please get in touch and we’d be more than willing to help.

The post SQL Server on a Diet: Improve the health of your database appeared first on SQL Solutions Group.

And I don’t mean the health of your data, your queries, your hardware, or anything like that. I’m talking about the health of the most critical machine, your body.

Steve Jones recently published an article on SQL Server Central encouraging readers to “don’t (always) be a hero.” Steve relates an experience early in his career when he was supporting an installation of SQL Server 4.2 on OS/2 1.2 on New Year’s Eve. Just as he was prepping to leave 30 minutes into the new year, the install locked up.

In short, Steve didn’t leave until early on Jan. 2nd. “I was a hero that January, as were my co-workers, with all of us logging around 100 hours each week that month,” Steve reports.

You’ve probably been there once or twice … or two or three dozen times. Something hits the fan and the idea of “9 to 5” and any personal plans go out the window for the foreseeable future. To be sure, anyone in this space who is expecting 9 to 5 probably needs to see a therapist, but working 60 hours a week or more should be rare, not normal.

Steve’s point is:

We need to be heroes at times, but those ought to be emergencies, and they ought to be rare. Most IT professionals (and others) I know will work longer or harder when needed, but the need can’t be constant or even regular. It should be an unusual situation. If it’s not, then something has to change, at least for me.

Most jobs involve some stress, and that’s almost unavoidable. The problem is when the stress is constant. Chronic stress is recognized as a risk factor for mental diseases and disease states, including cardiovascular disease.

Heart rate variability, or HRV, is a useful measure for stress. High variability is specifically associated low stress, leading to reduced mortality and improved well-being and quality of life. If that sentence wasn’t worded how you expected, think of low variability as always being in a “fight or flight” mode. Your heart is constantly racing, so the heartrate remains high … and variability is low.

So now the good news. Wearable devices, including Apple Watches, can help you track HRV. Equipped with this knowledge, you can work on (or at least try to work on) relieving your stress. Meditation, short walks, breathing exercises, visualization … these and other techniques can all help.

Quality sleep is another key part of maintaining your health. More good news: the Wall Street Journal recently reported on a study that showed getting less than eight hours of sleep is not as bad as previously thought:

Sleep “regularity”—going to bed and waking up at consistent times with few mid-slumber interruptions—matters more than how long you sleep. Sleeping six hours every night on a consistent schedule was associated with a lower risk of early death than sleeping eight hours with very irregular habits.

Sleep duration was still important: People who got long, consistent sleep had the lowest mortality risk, says Angus Burns, a research fellow at Harvard Medical School who co-wrote the study. But shorter, regular sleep was generally associated with lower mortality than longer, inconsistent sleep.

Diet is another key consideration. Instead of getting into that, just consider these: Cut back on sugar-laden soda, and instead of chips or other high calorie snacks, try veggies (baby carrots, anyone?) or fruit. It’s an easy switch that can make a big difference.

If your health is suffering, you’re not as much of an asset to your employer, so it’s in their and your best interests to ensure as much “balance” as possible. In Steve’s words, don’t be a hero. If you’re struggling and don’t see any light at the end of the tunnel, speak up.

The post Health Considerations in the Data Management Space appeared first on SQL Solutions Group.

Several of our clients have looked at a range of different solutions, such as copying “old” data to a different report or read-only server and hooking up a Linked Server through which to query the older data when needed. Another option is creating a “history” table in the same database and moving the older data to that table.

In both cases, this doesn’t solve the disk space issue, and it is still a costly storage solution. With the first scenario, I’m sure we have all been a little frustrated at the performance of Linked Servers at one time or another. In all cases, we had a requirements-gathering session with each client to determine several items, including:

• Does the archive data need to be queried, and if so, how often?
• How much data needs to be archived?
• Does referential integrity need to be kept between current and archive data?
• Is cloud storage an option?

There are other questions that we could ask, but these are the most critical because it establishes a baseline and determines the technology we can work with. In some cases, cloud storage is not an option, so we needed to stick with an on-premises solution. Where cloud storage was an option, that opened things up and we were able to look at and utilize some great technologies.

Now, where cloud storage wasn’t an option, there were still some things that we could look at that are strong, viable options, thus the topic of this blog. With SQL Server 2016, Microsoft added the ability to use PolyBase to query other SQL Server instances. This is a game changer, as PolyBase is very powerful. We at SSG set out to determine which is faster and more efficient, PolyBase or Linked Servers, when querying one SQL Server instance from another SQL Server instance.

Honestly, what we found was very surprising. Extremely surprising. Granted, all our tests were performed using SQL Server 2019 and SQL Server 2022, but even then, what we found was quite shocking.

I could walk through everything here, which would make for an extremely long blog, so I did a webinar on this exact topic with a lot of demos, discussion, and insights, and you can view it here. Let us know what you think as we always like to hear from you.

The post PolyBase vs Linked Servers  appeared first on SQL Solutions Group.