True Full and Differential Backups on Secondary Replicas

For years, database professionals faced a frustrating limitation. With Availability Groups, you could offload some workloads to secondary replicas—but not your real backup strategy.Yes, you could run copy-only backups on a secondary. But true full backups? Differential backups that maintain the LSN chain? Those had to run on the primary replica, until now. SQL Server 2025 introduces the ability to run true full backups and true differential backups on secondary replicas.

That means:

• You can move your entire backup workload off the primary.

• Your backup plan behaves exactly the same as if it were running on the primary.

• You potentially reduce your RPO by running backups more frequently.

This isn’t just a convenience feature—it’s a performance and resilience upgrade.

ZSTD Backup Compression

Another enhancement is the introduction of ZSTD backup compression, an algorithm originally developed at Facebook. Until now, SQL Server used MSXpress as the default compression method. ZSTD brings:

• Slightly better compression ratios

• Faster compression in many cases

• Less CPU overhead during decompression

• Potentially faster restores

In Randy’s testing—and in benchmarking done by Aaron Bertrand on an 8TB Stack Overflow database—backup sizes weren’t dramatically smaller. A 50GB database might shrink from 15GB to 14.5GB. That doesn’t sound revolutionary, but at scale it really adds up!

If you’re backing up multi-terabyte databases with long retention periods, even small savings compound quickly. Smaller backups also mean improved storage efficiency and potential restore performance gains. It’s not flashy—but it’s practical, which matters. 

Immutable Backups in Azure Blob Storage

Backing up directly to Azure Blob Storage isn’t new. But SQL Server 2025 now supports backups to immutable Azure Blob storage accounts.This enhancement strengthens ransomware protection. Immutable storage ensures that once a backup is written, it cannot be modified. The only permitted action is deletion under defined rules. While not truly air-gapped, this approach creates a powerful safeguard against backup tampering. In a world where ransomware targets backups first, this feature significantly improves recovery confidence.

Why These Enhancements Matter

SQL Server 2025 doesn’t reinvent backup strategy—it strengthens it.

• Offload backup workloads to secondary replicas

• Improve compression efficiency at scale

• Harden backup security against ransomware

Individually, these updates are incremental. Together, they represent a meaningful evolution in database resilience and performance. For organizations running mission-critical systems, that’s an especially big deal.

The post Backup Just Got Smarter: What’s New in Microsoft SQL Server 2025 appeared first on SQL Solutions Group.

SQL Server 2025 continues Microsoft’s steady focus on performance, reliability, and intelligent automation. Rather than introducing one dramatic overhaul, this release delivers a collection of meaningful, incremental improvements that—when combined—can result in noticeable gains for many workloads.

During a recent discussion, Rich and Kyle walked through several of the most impactful enhancements, focusing on Intelligent Query Processing, concurrency improvements, columnstore indexes, and long-awaited TempDB upgrades.

Smarter Query Performance with Intelligent Query Processing (IQP)

Intelligent Query Processing isn’t new, but SQL Server 2025 refines it further. IQP is an umbrella term covering a set of features designed to make the query optimizer more adaptive and accurate over time.

One key area of improvement is cardinality estimation—the process SQL Server uses to estimate how many rows will flow through each step of a query plan. Anyone who has tuned performance knows how damaging a mismatch between estimated and actual rows can be. SQL Server 2025 introduces incremental enhancements to these estimations, which, in most cases, translate to better execution plans and improved default performance without any code changes.

As with any optimizer change, edge cases still exist. Large jumps in behavior—such as the one between SQL Server 2014 and 2016—sometimes caused regressions in specific workloads. While the changes in 2025 are far less dramatic, testing remains essential when upgrading.

Another IQP enhancement is parameter-sensitive optimization, which addresses long-standing issues related to parameter sniffing. When query parameters vary significantly between executions, SQL Server can now better recognize those differences and adjust execution strategies accordingly, reducing performance instability.

Degree of parallelism (DOP) feedback also continues to evolve. Rather than making a one-time decision based solely on cost thresholds and configuration values, SQL Server now evaluates how well a query performs when running in parallel and adjusts its behavior over time. This allows the engine to fine-tune parallelism instead of blindly running at maximum settings. While powerful, it does require familiarity with execution plans to fully understand how SQL Server is adapting behind the scenes.

Adaptive Joins That Adjust on the Fly

Adaptive joins build on SQL Server’s ability to recover from imperfect statistics. Traditionally, SQL Server commits to a join strategy—such as nested loops or hash joins—based on cardinality estimates. If those estimates are wrong, the query can suffer dramatically.

With adaptive joins, SQL Server can change its mind mid-execution. If a join starts as a loop join but processes far more rows than expected, the engine can switch to a more appropriate join type, such as a hash join, while the query is still running.

SQL Server 2025 improves adaptive joins further, especially in complex scenarios involving multiple joins. This makes the engine more resilient in real-world environments where statistics are rarely perfect.

Improved Concurrency Through Smarter Locking

Concurrency is another area receiving meaningful attention. SQL Server has long relied on various lock types—row, key, page, range, and table locks—with relatively rigid thresholds for escalation.

Historically, modifying a few thousand rows could trigger escalation to a full table lock, leading many teams to design batch operations around that behavior. SQL Server 2025 is more nuanced. It can now hold thousands of locks without immediately escalating to a table lock when doing so would be unnecessary or harmful.

These optimized locking improvements, especially when combined with accelerated database recovery, have already shown dramatic reductions in blocking in real-world workloads. The engine is simply making better decisions about how much data truly needs to be locked.

Faster and More Efficient Columnstore Indexes

For organizations using columnstore indexes—particularly in data warehouse and analytics workloads—SQL Server 2025 brings performance gains across the board.

Compression is faster, index rebuilds complete more quickly, and the engine handles delta stores more efficiently. Delta stores, which temporarily hold newly inserted uncompressed data before it’s merged into compressed columnstore segments, are now managed more intelligently. This improves both data ingestion and query performance.

Reading from columnstore indexes has also been optimized, meaning analytics queries can run faster with no changes required at the application level.

TempDB Enhancements That DBAs Will Appreciate

TempDB has long been one of SQL Server’s most critical—and fragile—components. SQL Server 2025 introduces several enhancements that directly address common pain points.

One of the most significant changes is the ability to govern TempDB usage using Resource Governor. DBAs can now set quotas that prevent individual queries or workloads from consuming excessive TempDB space. When a query exceeds its limit, SQL Server cancels it before TempDB fills up, preventing server-wide outages.

This is especially valuable in environments where large reports or unfiltered queries can consume hundreds of gigabytes and bring systems to a halt—often in the middle of the night.

SQL Server 2025 also brings accelerated database recovery (ADR) to TempDB. Large rollbacks that previously took minutes can now complete almost instantly, reducing downtime and improving system responsiveness during heavy workloads.

To support monitoring and alerting, new DMV columns expose detailed TempDB usage metrics, including current usage, peak usage, and the number of TempDB limit violations. These additions make it far easier for DBAs and monitoring tools to detect issues before they escalate into outages.

A Practical Step Forward

Rather than reinventing SQL Server, the 2025 release refines the engine’s intelligence. Query plans adapt more effectively, locking decisions are smarter, analytics workloads run faster, and TempDB becomes far more manageable.

For teams considering an upgrade, SQL Server 2025 offers tangible performance and reliability improvements—especially for environments that struggle with concurrency, large analytical workloads, or TempDB pressure. As always, thorough testing is key, but for many workloads, the benefits will be felt immediately.

The post SQL Server 2025 Performance Enhancements: What Developers and DBAs Need to Know appeared first on SQL Solutions Group.

One of the more intriguing features introduced in SQL Server 2022 is Contained Availability Groups (AGs). At first glance, they look very similar to traditional AGs — but there are some key differences that can trip you up if you’re not prepared.

In this post, we’ll walk through:

• What a normal AG looks like (and its limitations)

• How Contained AGs change the game

• The pitfalls we’ve seen in real-world use

• Practical solutions if you run into these issues

A Quick Refresher: Always On Availability Groups

Availability Groups, introduced back in SQL Server 2012, are Microsoft’s built-in solution for high availability and disaster recovery. They let you replicate databases across multiple instances, managed by a Windows Server Failover Cluster (WSFC).

Key benefits:

• Seamless failover between primary and secondary nodes

• Read-only replicas to offload workloads

• The ability to perform backups and DBCC checks on secondaries, easing the load on your primary

But AGs have always had one big limitation: only user databases are replicated. System databases like msdb and master stay local to each node.

Why does that matter? Think about Agent jobs:

• You have to manually install jobs on every node (and keep them in sync).

• Jobs need to be AG-aware — otherwise they might try to run against a read-only secondary.

• Job history lives only on the current primary, which makes tracking a little messy.

For years, DBAs have worked around these issues, but it’s always been a pain point.

*Enter Contained Availability Groups*

Contained AGs look and act like traditional AGs, but with one important twist: they include their own contained system databases.

For example, if you create a Contained AG called MyAg, you’ll see:

• MyAg_master

• MyAg_msdb

These are replicated to all nodes in the AG, just like user databases.

Why this matters

• Agent jobs now live inside the contained msdb. No more duplicating jobs on every node.

• Job history is replicated. After failover, you don’t lose tracking.

• System tables travel with the AG. Everything stays consistent across nodes.

That’s a big win for manageability and one of the headline features of SQL Server 2022.

Gotchas Seen in the Wild

As with most “new shiny” features, Contained AGs aren’t a silver bullet. There’s a few things to watch out for:

1. No more job offloading to secondaries. Jobs inside the contained msdb only run on the primary. If you used secondaries to handle CHECKDB or backups, you’ll need to rethink.

2. Replication is not supported. If transactional replication is part of your design, contained AGs may not be an option.

3. Seeding is clunky. The GUI doesn’t support manual seeding yet. You’ll need to fall back to T-SQL.

4. No distributed AG support. If you rely on DAGs for hybrid or geo-distributed HA, contained AGs won’t help.

5. Confusion with system databases. When you connect to the listener, the msdb and master you see belong to the contained AG — not the local system copies.

Workarounds & Solutions

For Agent Jobs

If you truly need jobs to run on a secondary node, you still can — but you’ll have to install them in the local msdb outside the contained AG. That means:

• Jobs must be AG-aware again

• You’ll have to maintain them separately

It’s not elegant, but it gets the job done.

For System Databases

This is case-by-case. If your code doesn’t touch master or msdb, you’re fine. But if you do, you’ll need to carefully evaluate whether those objects belong in the contained copies or the traditional system DBs.

Remember: you can query both contained and local msdbs — just make sure your scripts account for both.

Final Word

Contained Availability Groups solve one of the long-standing headaches of AGs — managing Agent jobs across nodes. But they come with trade-offs that every DBA needs to be aware of.

Test carefully. Know where your jobs live. And make sure your disaster recovery plan accounts for these differences.

If you want to see them in action, check out our webinar on SQL Server 2022 features, where we demo Contained AGs in detail.

The post Contained Availability Groups in SQL Server 2022: What You Need to Know appeared first on SQL Solutions Group.

Thank you to our SQL Server Expert and Senior DBA, Rich Benner for providing the valuable Technical info behind this blog. To learn more about Wait Statistics from Rich, check out his recent webinar and two new YouTube videos- an explainer and a demo, on our channel.

What Are Wait Stats?

Anytime something inside Microsoft SQL Server is waiting for a resource, it’s assigned a wait stat. For example: 

• A query waiting for CPU or memory resources

• A query waiting to access a data object like a data page or table

• An internal SQL Server process waiting to complete a task

Some waits are completely normal. For example, in Availability Groups, certain threads simply wait to synchronize data with secondary nodes. That’s expected behavior and not necessarily a sign of trouble.

Key Things to Know About Wait Stats

1. They’re cumulative – Wait times are aggregated since the last time the data was cleared, usually at the last server restart (though other events can reset them).

2. They measure time in seconds or milliseconds – depending on the stat you’re viewing.

3. They’re invaluable for troubleshooting – By looking at wait stats, you can quickly zero in on what’s slowing down your system.

How to Find Wait Stats

The primary DMV for retrieving wait stats is:

SELECT * FROM sys.dm_os_wait_stats;

On a SQL Server 2022 instance, you might see over 1,300 different wait types. Even seasoned DBAs only know a fraction of them by heart. That’s where resources like the SQLskills Wait Types Library come in handy. This community-maintained list covers around 95% of known waits, detailing whether they’re benign or problematic and suggesting possible resolutions.

Useful Tools for Analyzing Wait Stats

• sp_WhoIsActive – An enhanced version of sp_who2 that shows what queries are running, their process IDs, and any current waitss.

• sp_BlitzFirst – A free tool that captures your current wait stats along with other diagnostic information.

• Custom Queries – Many DBAs have their own scripts for retrieving & formatting wait stats. A quick search for “Wait Stats query” will yield plenty of examples.

The Top 5 Most Common Wait Types 

1. LCK_* – Lock Waits

Occurs when one query locks a resource (e.g., a data page) and another query has to wait for it to be released. Example: An UPDATE query locks a row, preventing a SELECT query from reading it until the update completes.

2.

CXPACKET / CXCONSUMER

– Parallelism Waits

Happens when a query is split across multiple CPU cores. If one thread finishes before the others, it waits for the slowest one to catch up. Normal in moderation, but excessive CX waits can indicate an inefficient parallelism setup.

3.

ASYNC_NETWORK_IO

– Application Processing Delays

Indicates SQL Server has finished processing results but is waiting for the application to consume them. Often caused by applications retrieving rows one at a time instead of in batches.

4.

LATCH_* and PAGE_LATCH_* – Structure & Page Locks

• LATCH_*: Locks on internal structures (not data pages)

• PAGE_LATCH_*: Locks on data pages themselves. These can signal contention in memory structures or data access patterns.

5.

WRITELOG

– Transaction Log Bottlenecks

This occurs when SQL Server is waiting to write to the transaction log. Common causes include slow disk performance, excessive write activity, or log file contention.

Why Wait Stats Matter

Wait stats provide a high-level overview of where SQL Server is spending its time waiting.

They help you answer questions like:

• Is my performance issue due to locking, disk speed, or parallelism?

• Is the problem in SQL Server or in the application consuming data?

• Are my hardware resources (CPU, memory, storage) configured optimally?

By combining wait stats with tools like sp_WhoIsActive and sp_BlitzFirst, you can pinpoint bottlenecks faster and focus your tuning efforts where they’ll make the biggest impact. Not all waits are bad — many are just part of normal SQL Server operations. The key is knowing which waits are worth investigating and which you can safely ignore. With a solid understanding of wait stats, you’ll be better equipped to diagnose issues, improve query performance, and keep your SQL Server running smoothly.

Need help interpreting your wait stats?

At SQL Solutions Group, we work with clients every day to identify and fix performance issues. Contact us for a health check and start optimizing your environment today.

The post Understanding Wait Stats in SQL Server: A Practical Guide appeared first on SQL Solutions Group.

As SQL Server consultants, we’re always working with customer requirements and tackling diverse database needs, from the mundane to the critical. Clients turn to us for various reasons—maybe they don’t have the personnel, maybe the challenge is beyond their skill—but regardless of the circumstance, without question we have more SQL Server knowledge than they do.

That probably doesn’t read very well, and it’s not meant as a knock on our clients. But, if it weren’t true, they wouldn’t hire us, right?

Back to working with customer requirements. I recently blogged about times when too much knowledge could impede problem solving. This sparked further internal discussions about potentially uncomfortable situations where the customer provide solutions, especially when the problem is ill-defined. (This is a plate of spaghetti…let’s see if it will stick to the wall). This invariably leads to lots of wheel spinning. Here’s a comical look at how this might go.

In contrast, when the client provides accurate business requirements, we can get right down to brass tacks. 

Acting like a Consultant

This is where soft skills are so important. You clearly don’t want to alienate or offend a client. And, after all, you’re working with their data and their systems. On the other hand, if they could have solved it, they wouldn’t have called you. The challenge of the client presenting consultants—a team of experts—with a solution to an urgent problem is that the identified problem may very well not be the root cause of the issue and even if it is, the proposed solutions may not be the best way to solve the problem. 

So how do we, at SSG, handle “solutions” that are brought to us when a customer has an urgent crisis or emergency? First, we hope that our customers realize that SQL Server is what we do day in and day out. That, collectively, there is more than 95 years of SQL Server experience at SSG. We’ve seen it all and done it all, which is why our clients trust us to fix their problems.  

Secondly, we take the time to talk to our customers and listen to their experiences so we can get to the highly useful accurate business requirements. What problem do they perceive, and what are the symptoms? What is the pain that motivated them to seek out a team with more knowledge and experience with SQL? Did they develop a solution? 

When a customer comes to us with a requirement for solving a problem, we have to step back and evaluate if the customer even wants us to make sure they have identified the real problem. Sometimes, customers really do just want us to quickly throw together their solution and worry about the next problem later. Honestly, we do that sometimes. We let customers know that there may be a bigger issue, that there may be other things that arise later. Or, that ultimately down the road they will run into more problems. If that is the path they choose, that’s OK as well. 

Getting to the Solution

Well I tell them there’s no problems…only solutions.

— John Lennon

Ultimately, if you want to get your problem solved quickly, correctly, and efficiently, you don’t need to spend a ton of time creating a list of requirements. Bring us the problem, and let us work with you as we make sure the real problem has been identified and then develop a robust solution. In this sense, it doesn’t really matter if the customer is always right. It only matters if we can come to grips with the problem and find the appropriate solution in short order.

We’ll come back to this issue again in the near future to address proper collection of requirements. 

The post Working with Customer Requirements: Is the Customer Always Right? appeared first on SQL Solutions Group.

The post Safeguarding Your Data: Navigating SQL Backup, Compliance, Maintenance, and Support Challenges appeared first on SQL Solutions Group.

A Refreshed Look for Better Navigation

SSMS 21 introduces a cleaner, more intuitive design, making it easier to navigate large databases and complex queries. Microsoft has streamlined menus, improved font rendering, and optimized color schemes to reduce eye strain during long working sessions.

Key UI improvements include:

✔ Refined toolbar and menu layout – More logical grouping of frequently used functions for faster access.

✔ Improved tab management – A more organized way to handle multiple query windows efficiently.

✔ Customizable themes – Dark mode and high-contrast options for a more personalized experience.

Enhanced Query Editing Experience

The Query editor in SSMS 21 now feels smoother and more responsive, thanks to performance optimization under the hood. Users will notice faster syntax highlighting, better IntelliSense suggestions, and an upgraded results pane for easier data analysis.

Notable Enhancements:

✔ Auto-complete improvements – More accurate and context-aware suggestions for writing queries faster.

✔ Resizable panels – Adjust the layout effortlessly to fit your workflow.

✔ Improved error highlighting – Making debugging SQL scripts quicker and more intuitive. 

A Step Toward a More Modern SQL Experience

Microsoft is clearly prioritizing usability and productivity with SSMS 21. While the core functionality remains unchanged, these visual and functional updates make daily SQL tasks smoother and more enjoyable.

If you rely on SSMS for database management, this upgrade is definitely worth exploring. What do you think about the new look? Are there any features you’re hoping to see in future updates? Let us know in the comments!

The post SSMS 21 Gets a Fresh New Look: What’s Changed? appeared first on SQL Solutions Group.

Anyway, in this blog post I want to cover a technology for copying databases across SQL Managed Instances that saved my posterior recently. Well, it didn’t really save it, but it sure made my life easier. We have a client who uses Azure SQL Managed Instance for both their production and test environments. I think this is awesome.

As a quick background, Azure SQL Managed Instance is a full SQL Server instance served up as a PaaS service. Unlike Azure SQL Database where you are only working at the database level, Managed Instance gives you near 100% instance compatibility level capabilities with the latest version of SQL Server Enterprise Edition SQL Server database engine. SQL Managed Instance provides the lift and shift capabilities of on-premises databases with minimal database changes.

So, back to our customer. They essentially lifted and shifted their on-premises databases to Azure SQL Managed Instance and have been using it successfully for nearly two years. Again, this is awesome.

Last week they came to us and asked about reporting with Managed Instance. They were looking at data marts and data warehouses, but we needed more information. It turns out they have some people that just want the ability to query the databases, and potentially hook up Excel to these databases for data analysis.

The caveat is that the people I was talking to didn’t want to give the other group direct access to the production environment. Toootally get that. Yeah, like 100% get it. So, what are the options?

After some discussion we decided to give the group wanting to run reports access to the test Managed Instance environment. We’ll just refresh the test environment data with the production environment data on a schedule. For them, refreshing data nightly is OK for now.

So, the question then becomes, how do you get data, or copy a database, from one SQL Managed Instance to another SQL Managed Instance? There are two ways, and I’ll discuss both here.

PowerShell cmdlets

Microsoft released some Powershell cmdlets that make copying data across SQL Managed Instances simple.

There are essentially 3 cmdlets:

• Copy-AzSqlInstanceDatabase – Starts an online copy operation.
• Get-AzSqlInstanceDatabaseCopyOperation – Returns copy operation details.
• Complete-AzSqlInstanceDatabaseCopy – Completes the copy operation.

The Copy-AzSqlInstancesDatabase cmdlet starts an online copy operation of a database across managed instances using Always On availability group technology. Copy-AzSqlInstanceDatabase (Az.Sql) | Microsoft Learn

The Get-AzSqlInstanceDatabaseCopyOperation cmdlet returns database copy operation details. You can call this cmdlet in a loop so you can track the status of the copy. Get-AzSqlInstanceDatabaseCopyOperation (Az.Sql) | Microsoft Learn

The Complete-AzSqlInstanceDAtabaseCopy cmdlet completes the copy operation of the database and makes it available for read/write workloads. Complete-AzSqlInstanceDatabaseCopy (Az.Sql) | Microsoft Learn

You don’t have to call the Get-AzSqlInstanceDatabaseCopyOperation cmdlet as you could just call the Copy and Complete, but it is HIGHLY recommended you call it so you know when to call the complete.

The way this works is you start by calling the Copy-AzSqlInstancesDatabase cmdlet. This starts an online copy operation of one or more databases. Once the copy option starts, data is being seeded from the source to the target. During the process, you can call the Get-AzSqlInstanceDatabaseCopyOperation cmdlet to check the copy operation status, either in-progress or completed.

Once the copy operation has completed, the status will be copy ready for completion. Within the next 24 hours the user can complete the database copy operation by calling the Get-AzSqlInstanceDatabaseCopyOperation cmdlet. Note that until the Get-AzSqlInstanceDatabaseCopyOperation cmdlet is called, all the changes from the source are being replicated to the target database.

When Get-AzSqlInstanceDatabaseCopyOperation is called, the link between the source and target databases is broken, the target database becomes online and ready.

A very simple example would be something like this:

Source code

$dbname = "db1"
$sourceMI = "sqlmi1"
$targetMI = "sqlmi2"
$sourceRG = "rg1"
$targetMI = "rg2"
 
Copy-AzSqlInstanceDatabase `
    -DatabaseName $dbName `
    -InstanceName $sourceMi `
    -ResourceGroupName $sourceRg `
    -TargetInstanceName $targetMi `
    -TargetResourceGroupName $targetRg 
 
While(true$)
{
 
    Start-Sleep -Seconds 10
    $operation = Get-AzSqlInstanceDatabaseCopyOperation `
    -DatabaseName $dbname `
    -InstanceName $sourceMI `
    -ResourceGroupName $sourceRG `
    -TargetInstanceName $targetMI `
    -TargetResourceGroupName $targetRG
 
    try
    {
        $copysucceeded = $true
        Write-Host "Status of copy for " $dbname " is " $operation.State
        if (!($operation.State -eq "Succeeded")) {$copysucceeded = $false;}
    }
    catch { $copysucceeded = $false; Write-Host "Didn't work" }
 
    if ($copysucceeded)
    {
        Write-Host "Copy completed"
        break
    }
    else { Write-Host "Still copying..." }
 
}
 
Complete-AzSqlInstanceDatabaseCopy `
    -DatabaseName $dbName `
    -InstanceName $sourceMI `
    -ResourceGroupName $sourceRG `
    -TargetInstanceName $targetMI `
    -TargetResourceGroupName $targetRG

There is a GREAT example here: Using and automating Azure SQL Managed Instance database copy and database move operations with Azure PowerShell (microsoft.com)

The example copies multiple databases, but if you are needing to copy only one, you can tweak it.

Oh, by the way, you can also do the copy via the Azure portal.

Backup / Restore

The other option is backup and restore, meaning, back up to Azure blob storage then restore from there. Since Managed Instance provides access to the SQL Agent, a job is created on both managed instances; one to back up, and one to restore.

This option gets a bit tricky because in the scenario for our client, and I would think in most environments, you want to restore the database in the target managed instance with the same name, and have the backup file the same as well, just to make it easy.

The backup job is simple: back up the database to Azure, ensuring the backup file name is always the same name. The restore job is just as simple: delete the database and then restore the database with the same name.

However, the issue in this scenario is that when you back up to the same file name, it will error because the file already exists. When we tested we thought, “no problem, we’ll just use PowerShell (Remove-AzureStorageBlob) to remove the file as the first step in the job, then the second step in the job will be to back up the database!”

Unfortunately, Managed Instance does not support PowerShell job steps. Sigh. So, we have an on-premises job (or it can even be a Windows Task Scheduler job) that runs the delete. At this point it is just a timing thing. Since the client only wants to back up and restore a single database, we run the backup job at 1 a.m., for example. Then 30 minutes later we run the restore, then another 30 minutes later, we run the delete blob job.

For this solution you’ll need to create a storage account in Azure, then create a CREDENTIAL in SQL Server on both Managed Instances:

Source code

CREATE CREDENTIAL [https://mystorageaccount.blob.core.windows.net/mycontainer]
WITH IDENTITY N'SHARED ACCESS SIGNATURE',
SECRET = N'<storageaccountkey>'

Once that is created, simply back up the database.

Source code

BACKUP DATABASE [MyDB] TO URL = N'https://mystorageaccount.blob.core.windows.net/mycontainer/MyDB.bak'
	WITH BLOCKSIZE = 65535, COPY_ONLY, NOFORMAT, NOINIT, 
	NAME = N'MyDB-Full Database Backup', NOSKIP, NOREWIND, NOUNLOAD, STATS = 10

Or, if you are using Ola Hallengren’s solution (which we are a major fan of):

Source code

EXECUTE [dbo].[DatabaseBackup]
@Databases = 'SentryOne',
@URL = 'https://mystorageaccount.blob.core.windows.net/mycontainer',
@Credential = 'https://mystorageaccount.blob.core.windows.net/mycontainer',
@BackupType = 'FULL',
@Verify = 'N',
@CheckSum = 'Y',
@LogToTable = 'Y',
@Compress = 'Y',
@CopyOnly = 'Y'

Then restore onto the other Managed Instance:

Source code

RESTORE DATABASE [MyDB] FROM URL = N'https://mystorageaccount.blob.core.windows.net/mycontainer/MyDB.bak'

Yeah, a bit cumbersome, but it works and runs, and solves the client’s problem.

Which?

As in all things SQL, it depends. For our client, the simple backup and restore was the solution. They were OK with data being up to 24 hours old for their reporting in this scenario.

If using the PowerShell cmdlets, the target database is unavailable for connections until you issue the Complete cmdlet. But it lets you copy multiple databases, not just one.

It really just comes down to what you are attempting to do. Honestly, for the customer, we could have easily used the PowerShell cmdlets to copy the database each night (deleting the database on the target Managed Instance before starting the copy) and that would eliminate the need to delete the file from blob storage.

So if you looking for ways of copying databases across SQL Managed Instances, you have several options.

The post Copying Databases Across SQL Managed Instances appeared first on SQL Solutions Group.

Introduce your business and what you do there.
I’m the CEO of a cloud-based practice management software.

What challenge were you trying to address with SQL Solutions Group?
We needed help improving the speed of our software.

What was the scope of the project?
SQL Solutions Group is helping us optimize our software’s speed. First, they conducted a health check for our SQL server. After that, they had a full presentation detailing areas for improvement. They’ve then made a plan where easier fixes are prioritized over more complicated ones. SQL Solutions Group has installed a tool on our server that tells us a query’s speed. Their team advises us on what improvements we can make to increase our system’s performance.

How did you come to work with SSG?
We found them through a Google search for SQL server consulting providers.

What is the status of this engagement?
We’ve started working together in January 2021, and it’s ongoing.

What evidence can you share that demonstrates the impact of the engagement?
We’ve gotten more than our money’s worth with SQL Solutions Group.

How did SSG perform from a project management standpoint?
SQL Solutions Group is always available whenever we need them. We communicate through Zoom calls.

What did you find most impressive about this engagement?
SQL Solutions Group’s high level of expertise with SQL is impressive. SQL is a complicated product to work with, and their more than 25 years of experience in SQL is what we need for our business.

Are there any areas where SSG could improve?
I can’t think of anything. We’ve been pleased with their services.

Do you have any advice for potential customers?
The larger your application and database, the more necessary SQL Solutions Group’s services are. Work with them if you want to scale your business and grow your software company.

The post Chirospring: “We’ve gotten more than our money’s worth with SQL Solutions Group.” appeared first on SQL Solutions Group.

One potential issue here is the lack of time to delve really deeply into each programming language at one’s disposal. The thing about SQL Server is that it is a language that performs very differently than, say, C#. Those languages are row based, which means it’s common to iterate through a data set and process each row individually. This works quite well in those languages, but quite terribly in SQL Server. Being a set-based language, SQL Server prefers to deal with the whole data set at once.

Because of these nuances, it’s easy for developers to make mistakes when creating SQL server code. Let’s go through some common errors that we see.

T-SQL

Let’s start with basic T-SQL code that we commonly see, which are either not great for performance or have other considerations that are often overlooked.

Select *

You’re probably not surprised that we see this a lot. This is often used because the person writing the code isn’t sure exactly what they’ll need to return from the object they’re querying. Using Select * is not a great idea for a several reasons. Here are things to consider:

• Do you really need all of the data returned? Supporting indexes for queries like this are likely to be wider than necessary in order to cover all of the columns in the table. Wouldn’t it be nice if your indexes were smaller? They’d take less disk space but more importantly would require less I/O to retrieve and occupy less space in memory.
• Do you need to return that nvarchar(max) field that you’re got in your table? Not sure if you’ve got one?
• What if the underlying structure changes and somebody adds in a (max) field? Do they have to check with you first or will your query suddenly tank and cause you and your users a headache? Wouldn’t happen if we were selecting what is strictly necessary for our query.

One of the valid uses of SELECT * is when using something like WHERE EXISTS, where you don’t actually need to return any of the data in the query, something like this:

You could even replace that with a SELECT 1 to be more explicit what you’re doing. This is very much personal preference though.

Select Into

Using SELECT..INTO seems like a really easy way of creating tables (whether temporary or permanent), as it takes out a lot of the leg work in defining the structure. An obvious issue here is that your object is created with no constraints (should it be a unique primary key?) and no indexes (do you really want this object to be a heap table?). It also makes troubleshooting more difficult, as if you have something like a truncation issue then you can’t easily see the definition of your object without creating it and going to look for it (do you really want to have to run this code before you can complete the code review?).

Overall, it’s much more controlled if you spend the small amount of time required to declare your object explicitly. You can then make sure it’s exactly what you need and it’s much easier to maintain for those that follow.

Implicit Conversions

We often come across implicit conversions when analysing a server’s most poorly performing queries. This happens when you are comparing two pieces of data that are not the same data type. This could be comparing a column to a variable, or it could equally be joining two tables together. If these fields are not the exact same data type, then SQL Server is going to have to perform an implicit convert and convert one to match the other.

This can cause a couple of separate issues:

1. If the server has to convert a column in a table to a different data type then it is not going to be able to use any statistics on the table. These statistics are used when creating the execution plan and will inform how the query is actually run. Without these stats SQL Server can easily make a bad estimate and cause the execution of this query to run very poorly.
2. The other reason that this will be slow is that SQL Server will have to scan one of the tables in its entirety and convert this field before it can even consider joining the tables together. The larger the underlying tables get, the worse performance becomes.

The best way to avoid this issue is to ensure that your data types match. Whether that be making sure your variables match the field or if you are joining two tables on a certain field then making sure that the data types match. Also, remember, varchar(10) is not the same as nvarchar(10); they need to be the exact same data type.

Obfuscation

One of the most common issues we see is code obfuscation. What we mean by this is abstracting some code out of what we’re doing and storing it elsewhere. Doing this makes sense to a certain extent. Why write the same code more than once? If we’re going to use it more than once, why not create an object that we can use everywhere to do this? While this sounds like an excellent idea, let’s discuss why it might not be as great as it first appears

Nested Objects

Views

We see so many times that people will write a view to perform some logic and then use it everywhere. Simple right? Well, this grows over time. It may be that this is fine for now, but for a certain subset of queries you want to tweak that a little, so a view is created over the top of this view. This can very easily snowball and then you have views relying on views … relying on views … relying on views (you get the idea).

Primary issues:

1. Changing logic can be fraught with danger. What if we have to tweak one of these nested views? Do we have the ability to test every single place where this view is used? Do we even know everywhere it’s used?
2. Troubleshooting can be an absolute nightmare. You’ve got some incorrect data in the final query, but which of the sub views is it coming from? It makes a spider’s web of dependencies that are very time consuming and difficult to track. How much time can you invest in resolving this?
3. When creating execution plans, SQL Server will only spend so much time investigating these sub views before simply giving up and just running the query however it guesses with no available data. Any ideas how well that’s going to go and how our query is going to perform? Not well.

While views make a lot of sense when used properly, being aware of the nested views issue is something that could save you some major headaches in the future.

Scalar Functions

We see so many functions being used in the wild. Some work well, some not so much. By far the largest problem we come across is the (miss)use of scalar functions. Using a function is extremely common in other languages and they work very well. But there are a couple of very large “gotchas” in SQL Server:

1. As soon as you use a scalar function in SQL Server, it forces the transaction to run serially. This means you’re removing any ability for the query to run in parallel, and it therefore will likely dramatically impact the performance of your query. There are some advances here in SQL Server 2022 with Scalar UDF Inlining, but this is a relatively new feature and is not available for everybody.
2. Functions are also effectively treated as separate transactions, once called. You won’t see them in execution plans and this makes them extremely difficult to locate, investigate and troubleshoot. It’s super easy to miss them all together.
3. Scalar functions are also executed once per row of the data set. Remember how SQL Server is a set-based language and likes dealing with a large data set all at once? Well, scalar functions will pull that particular rug out from under you and turn it into a row-based calculation which will have those very same performance issues.
4. You also have the recurring issue of knowing exactly what the function is doing. What parameters are being passed? What’s the actual code that the function is executing? These questions can often be tricky to answer.

In our experience, scalar functions are fine when used correctly (sparingly and not in set-based operations. Think setting a local variable using a function, but it’s extremely easy to use them incorrectly and shoot yourself in the foot using them.

IN (Subqueries)

Something that is very easy to do is to run a query.

Using the Stack Overflow database, let’s say I wanted to see all posts for all users with more than 10,000 reputation points. I could write something like this;

Seems pretty simple, right? Well, the issue here is what goes on behind the scenes. That IN statement gets parsed out to something like this;

In the example query here, there would be 25,334 clauses in this statement. That’s a lot of data to parse through and that just won’t scale well within SQL Server. Joins would be the correct way of doing this. Whether a direct join is possible or whether the subquery is better to be separated and the result put into a temp table, that will all depend on the logic of your particular query.

Triggers

Aah, triggers, the silent killers. Sometimes necessary, always costly. The amount of times that we’ve seen poorly performing servers and there are triggers firing all over the place is definitely a non-zero number. They have the same issue as scalar functions, in which they are treated as a separate transaction and won’t be found in execution plans and the like.

Sometimes triggers are necessary, but it’s so easy to introduce them unnecessarily and then suffer the performance impact that follows. Need to do something upon or after insert? There are other ways of resolving those requirements without resorting to Triggers.

Synonyms

We’re glad that a lot of people don’t even know that synonyms are a thing in SQL Server, but when we do find them they can cause major confusion. While we totally understand that the business will often push for the simplest (cheapest) solution to resolve the issue of object renaming, it’ll definitely cause confusion when troubleshooting and ultimately become a maintenance issue.

Linked Servers used Incorrectly

Linked servers can be highly effective when used correctly. The issues that occur are with understanding how linked servers execute queries locally and remotely. If we just include a linked server using four part naming convention ([RemoteServerName].[Database].[Schema].[Object]), then our local server retrieves all data from the remote server and processes data locally. This may be fine if we’re doing a simple query, where we’re selecting all data in a single table. But once we get to adding in filtering and/or manipulating the data in any way (joining for example), then this can get inefficient very quickly.

There has been some change in recent versions of SQL Server on how it deals with these types of connections. Ultimately the better way of processing this data is to either push your results from the remote server and then deal with these results, or to use OPENROWSET. This avoids the issues above and represents best practice for querying linked servers.

Many Database Problem

For companies who are vendors or provide a service to multiple clients, it is not unusual to see a database created for each customer. The thinking here is that it silos each customer’s data into a separate database to ensure there is no risk of data being accessed by the wrong user. This makes sense, right? Sure, but there are definitely some issues that can arise from this:

1. Standardisation is a big one. Can you guarantee that your databases are identical to each other? Are they even meant to be? If not, you will have issues performing operations that have to update all databases. Do you need to push a maintenance patch or hotfix? What if some databases have incompatible schemas? You’re going to run into a big headache here really quick.
2. Context switching is absolutely an issue you should be considering in this scenario. Because the engine has to switch between databases constantly, it can put a hidden workload onto the server and cause some big issues if not monitored very closely.
3. Maintenance is also another thing to consider. Many databases will require many backups, will contain many indexes, and statistics that require maintenance (you’re doing this, right?). Everything becomes a list that needs sorting through before any potential issues can be resolved.

The two main options we have here are to consolidate your customers into fewer databases (we’re aware of how much work this can be to implement) or to simply split your server into smaller servers and host fewer customers on each. In today’s world of virtualisation, this is a lot simpler than it used to be. You just need to monitor them to ensure you don’t have all of your most busy customers on the same server.

Many Tenants on a Host

This one is tricky without having access to the virtualisation layer. We have absolutely seen issues with over provisioning hardware between multiple tenants. It seems obvious that you should consider doing this. Each server is not going to be using 100% of the resources allocated to it, so why not allow them all to have a bit more so you can utilise your hardware?

Let’s say we have 20 CPU cores and 1TB of ram on a server and we have 10 tenants on this host. If we were hard provisioning, we would allocate two CPU cores and 100GB of ram to each of these tenants. That way, everybody always has everything they want. But then we notice that the resources are not often used — we sit around with spare CPU going idle. Why not put a few more servers on this box so we can save money? Makes sense, right? The directors would love if we can provide a cost saving.

The issue comes when you have your busy periods, whether that be the holidays, or whether that’s Monday mornings when your customers come online. What then? Well, the tenants are going to fight for those resources. Your servers are going to think they each have two cores and can use them. But because we’ve now got 20 tenants on this host, they can not necessarily have that many resources at any time.

Seeing this inside SQL Server is extremely difficult and even experienced troubleshooters would find it hard to find that this was the problem.

A similar issue is that of “noisy neighbours”. What if one particular tenant on the host uses most of the resources? There’s not much the other tenants can do about it. They’re going to struggle to do what they need to and there’s nothing they can do about it.

Keeping Connections Open

When connecting to SQL Server, you’ll likely do multiple things, so it makes sense to keep the connection open until you finish the entire workload, right? That can come with some very expensive lessons to learn if this becomes commonplace in your environment.

There are a total of 32,767 user connections available in SQL Server. What happens when these are all used up? You simply cannot make new connections to the server. I will leave the consequences of that to your own imagination.

It’s very much best practice to only keep a connection open for as long as you absolutely require it. The resources required to open a new connection are negligible, true. But the long term consequences of keeping them open can be drastic. It’s also worth noting that you may not have an issue with this right now, but as databases and user counts grow, do you want to risk this or get it right from the beginning?

Debug Mode

It’s rare that a company will add the option to be able to debug SQL code. You may have the error message that something’s throwing, but where specifically is that error coming from? Most of the time somebody will need to extract the T-SQL code piece by piece and run it until you get to the problem code. This can be extremely time consuming and is luckily something that is absolutely avoidable.

By adding a @debug variable and the related code, it will save you so much time. The idea of @debug is that when you enable the flag, it will log relevant information into a logging table so you can analyse it separately.

Adding the variable and related logging isn’t free. It’ll take a little time to ensure it works, but by getting to the standard where code has a debug option, you will make the long-term viability and maintenance so much easier.

The post SQL Server Antipatterns: Common Mistakes with SQL Code appeared first on SQL Solutions Group.