Compile

A compile occurs when a query is given to the query optimiser and, when it does a lookup into the plan cache, no matching plan is found. The optimism must then compile the query, generating an execution plan, must add that plan to the plan cache (in most cases) and then must pass that plan onto the query execution engine so that the query can be executed. (http://technet.microsoft.com/en-us/library/Cc966425)

Recompile

A recompile is something slightly different. For a recompile, the optimiser must find a matching plan when it queries the plan cache, must hand that cached plan over to the query execution engine and then while doing validation checks the execution engine must determine that then query plan is no longer valid and request the optimiser to partially or completely recompile the query. (http://technet.microsoft.com/en-us/library/Cc966425)

Subtle difference. Both cases result in the optimiser generating an execution plan, but the reasons can be different. Also worth noting is that a compile results in a new plan in the cache, a recompile simply replaces an existing plan.

Another difference since SQL 2005 – a compile is for the entire batch, but a recompile can be for just a single statement within the batch.

Now the theory’s dealt with, let’s look at some examples and see how we can track these two events and try and get a better understanding of which occurs when and how they look.

The tools I’m going to use to track these are performance monitor with the compiles/sec and recompiles/sec counters and SQL Profiler with the event SP:StmtRecompile event (there’s no profiler event for compilation). I’ll also check what’s in the plan cache after each test.

The first one’s going to be very simplistic, a query run against an empty plan cache.

DBCC FREEPROCCACHE
GO

EXEC dbo.OutStandingTotalByStatus
GO

What we get from that is a non-zero value for SQL Compilations/sec (perfmon) and the following from profiler (The SQL Recompiles/sec remains 0)

and the plan cache now contains one plan with one use. (for more info on how the CacheMiss and CacheInsert events work, see http://sqlinthewild.co.za/index.php/2010/07/27/hit-and-miss/ and http://sqlinthewild.co.za/index.php/2010/08/31/come-and-gone/)

In this case, I hope it was clear, we had a compile occur (empty plan cache before, new plan added to cache).

Now what happens if, with no clearing of the cache nor anything else being done, I mark that procedure for recompile and run it again?

EXEC sp_recompile OutStandingTotalByStatus
GO

EXEC dbo.OutStandingTotalByStatus
GO

This is interesting. We don’t see a non-zero value for SQLRecompiles/sec (as might have been expected), instead, as with the previous case we get a spike for SQLCompiles/sec. The profiler output shows why

Note the SP:CacheRemove being run for the stored procedure’s plan. So sp_recompile does not mark a plan as invalid and needing recompilation. It removes it from the cache entirely. The next time that procedure runs SQL does a cache lookup, doesn’t find a matching plan and compiles a new one (compile, not recompile)

Onwards… What about if I alter the procedure?

I’m not going to clear the cache first, the procedure has a plan in there from the last test, we can use that to see the effects.

ALTER PROCEDURE OutStandingTotalByStatus
AS
-- This has changed!
SELECT 1;

SELECT o.OrderStatus, SUM(UnitPrice*Quantity) AS TotalOutstanding
FROM dbo.Orders o
INNER JOIN dbo.OrderDetails od ON o.OrderID = od.OrderID
WHERE ShippingDate IS NULL
GROUP BY OrderStatus;
GO

EXEC dbo.OutStandingTotalByStatus
GO

Again perfmon shows a bump in Compiles/sec and the recompiles/sec is still at 0. Again, profiler shows why.

Same as when I ran the sp_recompile, the ALTER PROCEDURE triggered a SP:CacheRemove. The plan was gone from cache at the point the stored procedure ran again, so this was considered a compile, not a recompile.

btw, for those curious, if I query the plan cache between altering the procedure and running it again, the procedure’s plan really has gone.

Let’s try something more complex, an alteration of one of the base tables. I’m going to clear the cache first, for reasons that will later be clear

DBCC FREEPROCCACHE
GO

EXEC dbo.OutStandingTotalByStatus
GO

ALTER TABLE dbo.OrderDetails ADD Filler CHAR(10)
GO

EXEC dbo.OutStandingTotalByStatus
GO

Ha! This time perfmon shows a non-zero value for SQLRecompiles/sec. The profiler trace has a different form too

There’s no SP:CacheRemove. No SP:CacheInsert either.

The alteration of the table did not result in SQL going through the plan cache and removing plans for dependant objects. That makes sense if you think about it. There could be millions of plans in cache, for SQL to check each and every one to see if it has a dependency on the table just changed would be terribly time consuming, so it’s not done. What happens instead is that the next time the procedure runs SQL looks in the cache, finds the procedure’s plan (the SP:CacheHit), does the pre-execution validations and finds that one of the base objects has changed since that plan was compiled. The plan is then sent to the optimiser to recompile that plan (the SP:Recompile event)

Here’s the other interesting thing, from a query of the plan cache

The usecounts is 2, not 1. I ran that procedure once before altering the table, once after. The alter of the table forced SQL to recompile the plan on the second execution, but the usecount for the plan reflects both executions, the one from before the recompile and the one from after.

In all the previous cases, when the plan was actually removed from cache and then compiled and inserted the usecount was only 1 because when the plan was removed all related information was removed with it. With a recompile it’s not removed, so the usecounts aren’t reset to 0.

This is important to note. There are lots of articles (and books even) that say to check plan usecounts as a low usecount is a sign of frequent recompiles. As we can now see, it’s not. It’s a sign of frequent compiles, of the plans being removed from cache and later re-added, recompiles don’t affect the usecount (well, at least not in this case, there may be cases where it does)

Now try a couple more common cases, see what they show, starting with rebuilding an index.

DBCC FREEPROCCACHE
GO

EXEC dbo.OutStandingTotalByStatus
GO

ALTER INDEX PK_OrderDetails ON dbo.OrderDetails REBUILD
GO

EXEC dbo.OutStandingTotalByStatus
GO

Any guesses?

It’s a recompile (and the perfmon counters confirm that). As with the previous case, the usecount shown in sys.dm_exec_cached_plans is 2 for the procedure’s plan, not 1

Lastly, a statistics update. Can anyone guess what’s going to happen here? Anyone?

DBCC FREEPROCCACHE
GO

EXEC dbo.OutStandingTotalByStatus
GO

-- Insert a few more rows into OrderDetails at this point

UPDATE STATISTICS dbo.OrderDetails WITH FULLSCAN
GO

EXEC dbo.OutStandingTotalByStatus
GO

Yup, it’s a recompile.

It is worth nothing that (in my tests anyway), if at least one row isn’t changed, the stats update has no effect. Perhaps an optimisation that checks whether there have been modifications or not before updating? I’m not sure what’s causing that.

There is a lot more I could test, SET option changes, memory pressure, configuration changes, etc, but I think I’ve done enough that anyone interested can play themselves for further info.

Ok, but what’s the use? Was this just a dive into internals with no practical usage? I don’t think so.

One thing that’s important when investigating SQL problems is to understand what the various counters, events and columns are saying and what they mean. If one is faced with a server with very high SQLCompiles/sec, it’s a waste of time to go looking at whether automatic or manual stats updates are happening too often. Similarly if the SQLRecompiles/sec is high, investigating the usage of WITH RECOMPILE and OPTION(RECOMPILE) is completely off track and is going to waste time and not help with solving the problem.

That’s why I believe it’s important to understand what the various counters, events and columns actually means and what occurrences within SQL affects them. None of us have time to spend a couple days busy investigating something that’s not related to what’s actually wrong with the server based on an incorrect understanding of what SQL is showing us.

When trying to check a query’s execution plan, the display execution plan option of Management Studio is usually adequate, however there are occasions where it’s either not feasible to run the query from Management Studio or the particular behaviour can’t be reproduced in Management Studio (perhaps because of different SET options). For cases like this it may be necessary to capture the execution plans via SQL Trace. Fortunately there are events for execution plans. Eight of them, to be precise, all under the Performance folder.

Great, so there’s no shortage of options available. But what are the differences between them?

Showplan All

According to Books Online:

The Showplan All event class occurs when Microsoft SQL Server executes an SQL statement. Include this event class to identify the Showplan operators on Microsoft SQL Server 2000 or Microsoft SQL Server 7.0. This event class will also work on SQL Server 2005 and later; however, the information included is a subset of the information available in the Showplan XML Statistics Profile or Showplan XML event class.

So in other words this one is not generally the event that we should be looking at on the latest versions of SQL. It’s not deprecated however (at least not in SQL 2008), so it is still usable on the later versions if you absolutely want.

The plan is in text format and hence it will be smaller than the XML format plan (like the one produced by Management Studio’s “include execution plan” option). Important to note is that the run-time statistics (eg “Actual row count”) are not present in this event. What is present is the text form of the plan, the estimated row counts for each operator, physical and logical operator names, any details of what the operator operated on, estimated rows, executions, IO and CPU costs, row size and some costs.

The event fires once per query executed, so if a single query is executed four times, the event will fire four times.

Also worth noting is that for all of the Showplan events, the BinaryData column must be included in the trace, as without that the plan will not be captured.

Showplan All for Query Compile

Very similar to the Showplan All event in appearance, the difference from the Showplan All event is when it fires. While Showplan All fires on query execution, Showplan All for Query Compile event fires when the query is optimised, not executed. Hence run the query four times and the Showplan All for Query Compile event will fire once, not four times (assuming that the plan is not already in cache and nothing forces a recompile on any of the executions)

Like with the Showplan event, there is no run-time information. This shouldn’t be surprising, since the event fires at query compile, not execution.

Showplan Statistics Profile

Again, very similar to the Showplan All event in appearance. As with Showplan All event, the Books Online entry indicates that this is more use against SQL 7 and 2000 servers.

This is the first event that we’ve looked at that does contain the run-time information, specifically the execution count and row count for each operator.

Showplan Text

This is the event that produced the least information of all of the plan events. Books Online has a similar, but longer, note as for Showplan All.

The Showplan Text event class occurs when Microsoft SQL Server executes an SQL statement. Include this event class to identify the Showplan operators on SQL Server 2000 or SQL Server 7.0. This event class works on SQL Server or later, however the information included is a subset of the information available in Showplan All, Showplan XML Statistics Profile or Showplan XML event class.

All that this event shows is the text form of the plan. No estimated or actual statistics are included, no costs are available, no details of operator names or details.

This may be useful if there’s a need for a long-term trace for execution plans, since it’s the smallest amount of information it will result in the smallest files and lowest impact.

Showplan Text (Unencoded)

This event shows exactly the same information as Showplan Text does. The only difference is that the plan is formatted as a string and placed into the TextData column instead of the BinaryData column as with previous events.

Showplan XML

The XML plan events are the ‘new’ events, first appearing in SQL 2005. One thing of interest with these events is that the resulting plans don’t have to be stored in the trace file, there is an option (with the profiler GUI that is) to save them into separate files. This is only an option when using the Profiler GUI, a server-side trace has no such capability.

With this event, the plan is stored in the textdata column, so the BinaryData column is not required, unlike with the Showplan All events.

Like with the Showplan All event, this event contains no run-time information. It has only the compile time information, much as the ‘Display Estimated Execution Plan’ option in Management Studio.

The event fires once per query executed, so if a single query is executed four times, the event will fire four times.

Showplan XML for Query Compile

Almost identical to the Showplan XML event, the difference is that this event fires on query compile rather than query execution. Hence run the query four times and the Showplan XML for Query Compile event will fire once, not four times.

Showplan XML Statistics Profile

This is pretty much the top of the execution plan events – the full XML execution plan, with all compile- and run-time information on each execution of the query.

Of course, as the largest of the plan events it’s going to have an impact on the server and as such should be traced with caution. It’s also going to result in large trace files if run for any length of time. Test first before running a trace against a busy production server.

Ok, it sounds reasonable, if the optimiser optimises for the execution path taken on first execution it’ll have to go back and optimise the other paths when they are executed.

But is that really how it works? Now that I’ve spend some time looking at how the cache monitoring trace events behave, it’s possible to find out using those.

Let’s start with a simple example

CREATE TABLE Tbl1 (
  ID INT
);

CREATE TABLE Tbl2 (
  ID VARCHAR(10),
  SomeDate DATETIME
);
GO

CREATE PROCEDURE TestingRecompiles (@SomeParam INT)
AS
IF (@SomeParam = 1)
  SELECT ID FROM Tbl1;
ELSE
  SELECT SomeDate FROM Tbl2;
GO

Simple enough. First execution will be with the parameter value of 1. I’m going to use Profiler to see what’s happening. Events traced are SP:CacheInsert, T-SQL:StmtRecompile and the XML Plan for query compile, so I can see exactly what plan was generated. I’m using the ‘For Query Compile’ event so that I can catch the plan at optimisation time, not at execution time.

With that trace running, I’m going to run the proc once with a value of 1 for the parameter. Important to note is that there is no cached plan for the procedure at this point.

I have two plan for query compile events, one cache insert for the procedure and a batch completed. I want to focus on the two compile events.

The first one shows a table scan on tbl1. The second shows another table scan, on tbl2. So it appears that both branches of the IF statement went through the optimiser and had plans created for them.

Before jumping to any conclusions, I want to see what’s sitting in the plan cache for this procedure.

One plan for that procedure and the plan shows a conditional (the if) and query operators for both branches.

So far it looks like the plan that was generated on the first execution contains plans for all of the statements and is sufficient for any execution, but to be sure, let’s see what happens when I run the query again with a different parameter.

No query compile, no cache insert, no recompile statement. So in this case there was no recompile from the IF.

Before drawing any conclusions, I want to check a more complex procedure, as the plans here were Trivial. Maybe full optimisation will show a different result.

USE Adventureworks
GO

CREATE PROCEDURE TestingRecompiles2 (@SomeParam INT)
AS
IF (@SomeParam = 1)
  SELECT p.[Title], p.[FirstName], p.[MiddleName], p.[LastName]
    FROM [HumanResources].[Employee] e
      INNER JOIN [Person].[Person] p ON p.[BusinessEntityID] = e.[BusinessEntityID]
      INNER JOIN [Person].[BusinessEntityAddress] bea ON bea.[BusinessEntityID] = e.[BusinessEntityID]
      INNER JOIN [Person].[Address] a ON a.[AddressID] = bea.[AddressID]
ELSE
  SELECT s.[Name], a.[AddressLine1],a.[AddressLine2],a.[City],a.[PostalCode]
    FROM [Sales].[Store] s
      INNER JOIN [Person].[BusinessEntityAddress] bea ON bea.[BusinessEntityID] = s.[BusinessEntityID]
      INNER JOIN [Person].[Address] a ON a.[AddressID] = bea.[AddressID]
GO

These queries are complex enough to go through full optimisation, but the profiler trace shows the same as before

Two showplan events prior to the completion of the first execution, one for each branch of the IF, one cache insert event and no showplan events, no recompile events when the second execution (with a different parameter value) began. (I’ll leave examining the plan in cache as an exercise to the reader)

One more, just to be thorough.

CREATE PROCEDURE TestingRecompiles3
AS
IF EXISTS (SELECT 1 FROM tbl1 WHERE ID>0)
 SELECT ID FROM tbl1
ELSE
 SELECT ID FROM tbl2
GO

INSERT INTO tbl1 (ID) -- Pre-populate with 1 row so that stats-based recompiles don't confuse the issue.
 VALUES (0)
GO

EXEC TestingRecompiles3
GO

INSERT INTO tbl1 (ID)
VALUES (1)
GO

EXEC TestingRecompiles3
GO

Much the same as we saw in the earlier tests. This time there are three showplan events, one for the subquery in the IF and one for each branch. Again there is one CacheInsert for the entire procedure and no recompile events on the second execution

So what is actually happening?

When the optimiser receives a batch or object to generate execution plans for it does not execute any portion of that code. That’s not its job, that’s what the Query Execution engine is there for. The optimiser’s job is to optimise each statement within the procedure or batch⁽¹⁾, to generate execution plans that are good enough and, in most cases, are reusable.

When it comes across an IF statement, the optimiser is not going to execute the expression to see which branch will be taken on this execution⁽²⁾. It’s going to optimise both branches because it has no idea which one will be executed on this or subsequent executions.

For each statement within the procedure, the optimiser generates the best plan it can find based on the parameter values for the current execution. This is not necessarily ideal as it can end up optimising queries for parameter values that they will never be called with. As can be imagined, this can generate some not-so-optimal execution plans. I wrote about this problem some time back, so I’m not going to touch on it again

So, in conclusion… Do IF statements cause recompiles? No. The optimiser processes all branches of a conditional no matter which branch will be taken during execution, even if the branch can never be executed. Use of IF statements can however result in some really awful plans and make you sincerely wish they had caused a recompile. Use with caution (or recompile hints) if plans can be radically different depending on parameter values on the first execution.

(1) There are cases where the optimiser won’t optimise a statement within a procedure during the initial optimisation and will leave the statement to be optimised later (called deferred compile), but this is not due to IF statements. The one thing that immediately comes to mind that does cause this behaviour is when the statement depends on an object that does not exist at the point that the first optimisation is done, such as when a table is created within the procedure and then used within the same procedure.

(2) As proof that the optimiser doesn’t evaluate conditional expressions, consider the following:

IF (42/0 = 137)
  SELECT name, sys.objects.type_desc FROM sys.objects

If that is executed, it fails with a divide by zero error (as expected). Requesting the estimated execution plan succeeds and returns a valid execution plan and executing it with the profiler trace that I used above (adding in the user error event) shows the Showplan for query compile (as the select within the IF is optimised), then a cache insert for that plan, and then only is the divide by zero error thrown, just before the SQL:BatchCompleted event.

There are two recompile events available in Profiler in SQL 2008

• SP:Recompile under Stored Procedures
• SQL:StmtRecompile under T-SQL

Which to use when?

Books Online has the following to say on the two events.

The SQL:StmtRecompile event class indicates statement-level recompilations caused by all types of batches: stored procedures, triggers, ad hoc batches, and queries. Starting in SQL Server 2005, the SQL:StmtRecompile event class should be used instead of the SP:Recompile event class.

and

The SP:Recompile event class indicates that a stored procedure, trigger, or user-defined function has been recompiled. In SQL Server 2005 and later, recompilations reported by this event class occur at the statement level, whereas those in SQL Server 2000 occurred at the batch level.

In SQL Server 2005 and later, the preferred way to trace statement-level recompilations is to use the SQL:StmtRecompile event class. Starting in SQL Server 2005, the SP:Recompile event class is deprecated.

So it appears that they show the same thing and SP:Recompile is deprecated. That simplifies the entire situation, the only one that I’m going to look at in that case is the SQL:StmtRecompile event.

So what does the event look like?

Not too different from the other cache events. One difference is that the ad-hoc SQL statement no longer has an Object ID, only the stored procedure has an Object ID and it matches (as one would expect) with the Object ID in the system tables.

Another difference is that if the statement being recompiled is part of a procedure, the stored procedure name does not appear in the TextData column, rather the statement does. The way to tell whether the recompile was ad-hoc code or part of a procedure is to look at the ObjectID and ObjectName columns, which are only populated if the recompiled statement was part of a procedure (or trigger or function, etc)

One column that’s important here is the EventSubClass. The different values of this column, nicely interpreted by Profiler, give the reason that the recompile was necessary.

As per Books Online, the EventSubClass can have the following values.

1. Schema changed
2. Statistics changed
3. Deferred compile
4. Set option changed
5. Temp table changed
6. Remote rowset changed
7. For Browse permissions changed
8. Query notification environment changed
9. Partition view changed
10. Cursor options changed
11. Option (recompile) requested

Some of those are not occurrences that will be encountered all that often, and I have found at least one that’s not on that list. (I remember seeing a ’12′ in the subclass column once, but can’t recall what the description was)

So, now that we’ve seen what the event looks like, let’s see when it fires. The event subclass gives a good list of the causes, I’m not going to go over all of the subclasses, just some of the more common (and easier to demo). I won’t guarantee that what follows is a comprehensive list of everything that causes the various recompiles, I’m sure to miss something.

Using the same example table as previous cache-investigation posts did.

Schema Changes

This recompile occurs when any of the base tables for the query have changed, when any of the indexes for the query have been rebuild or changed and when sp_recompile is run for any of the base tables.

It does not occur for a procedure if that procedure is altered, nor does it appear if sp_recompile is run for the procedure. Both of those result in a sp:CacheRemove event.

Statistics Changed

Fairly obvious. The recompile occurred because statistics that the statement in question used have changed.

The update of some rows is necessary to get the recompile. If the stats are updated and there were no data modifications made since the previous update stats, the recompile does not occur.

This recompile also occurs when an autoupdate of the stats that the query uses has occurred.

Deferred compile

A deferred compile occurs when an object referenced by a statement within the batch does not exist when the batch is first compiled.

CREATE PROCEDURE [dbo].[TestingCacheEvents2]
AS

SELECT ID, SomeDate, Status
FROM TestingCacheEvents
WHERE Status = 'G'

CREATE TABLE #Temp (Status char(2), CountStatus int)

INSERT INTO #Temp
SELECT status, count(*)
FROM TestingCacheEvents
GROUP BY Status

DROP TABLE #Temp

Now in SQL 2000 and earlier, this form of procedure would have recompiled (entirely) on every execution due to the creation of the temp table part way through. If the procedure ran frequently, this could have a devastating effect on performance. In SQL 2005 and above however this does not happen. As can be seen in the following screenshot, the second execution does not incur a recompile.

Set option changed

This one occurs when a set option is changed within the procedure, not outside the procedure.

CREATE PROCEDURE [dbo].[TestingCacheEvents3]
AS

SELECT ID, SomeDate, Status
FROM TestingCacheEvents
WHERE Status = 'G'

SET ARITHABORT OFF

SELECT DATEDIFF(dd,SomeDate,GETDATE())
FROM TestingCacheEvents
WHERE Status = 'G'

Option Recompile requested

If this one isn’t obvious, then I don’t know what could be.

CREATE PROCEDURE [dbo].[TestingCacheEvents5]
AS

SELECT ID, SomeDate, Status
FROM TestingCacheEvents
WHERE Status = 'G'

SELECT Status, COUNT(*)
FROM TestingCacheEvents
GROUP BY Status
OPTION (RECOMPILE)

Previously I took a look at the CacheHit and CacheMiss events to see how they behave and how to identify what’s been searched for in the cache. in this follow up, I want to take a similar look at the CacheInsert and CacheRemove events, see when they fire and how to identify the objects that they relate to.

Again, a word of caution, these can be frequently occurring events on busy servers and so traces should be kept short and to a minimum of events and columns. That said, these should occur a lot less often than the CacheHit and CacheMiss events. If they are occurring very frequently it may indicate that the SQL Server is not reusing plans efficiently.

CacheInsert

The CacheInsert event fires after a CacheMiss. The search for a matching plan in the cache failed, firing a CacheMiss event. Since there’s no plan, the optimiser is invoked to generate one and then that plan is inserted into the plan cache before the Query Execution engine begins execution.

The event is fairly simple, though of course there are a few surprises (what in SQL doesn’t have?).

Exec FireCacheEvents
GO

SELECT ID, SomeDate, Status
FROM TestingCacheEvents
WHERE Status =  'C'

Three CacheInsert events for two batches. The first is simple enough, it’s the insert of the plan for the stored procedure. The procedure name is in the TextData column and the procedure’s ID is in the ObjectID column. The ObjectName column is not populated for this event.

The second and third Cacheinsert events are the interesting ones. The second one shows a parameterised version of the ad-hoc SQL statement, while the third shows an unparameterised version of the same ad-hoc SQL statement. Clearly this query was simple enough to qualify for auto-parameterisation. So, are there two plans for this been inserted into cache? To answer that one, I need to switch over to SSMS and query the plan cache.

select usecounts, size_in_bytes, cacheobjtype, objtype, st.text,  qp.query_plan
 from sys.dm_exec_cached_plans cp
 cross apply  sys.dm_exec_sql_text(cp.plan_handle) st
 cross apply  sys.dm_exec_query_plan(cp.plan_handle) qp
 where st.text not like  '%sys.dm_exec%'

Well, there are three entries and the Cache Object Type is listed as compiled plan for all three, but there’s something different between them. Take a look at the size in bytes for the ad-hoc and prepared statements. The ad-hoc is less than half the size of the prepared. That’s not the only thing different. If I open up the execution plans, the exec plan for the prepared statement looks normal. There’s a clustered index scan (there are no nonclustered indexes on this table yet) and a select operator. The plan for the adhoc statement however…

There’s something missing here, like the rest of the plan. Ok, so the entry with the ad-hoc type is not a complete plan, so what is it?

There doesn’t seem to be anything on MSDN on this (at least nothing I could find), but two of Kalen’s books¹. (Inside Microsoft SQL Server 2005: Query Tuning and Optimisation, page 283; Microsoft SQL Server 2008 Internals, page 533) mention this. These ad-hoc entries are shell queries, cached just to make the parameterised form of the query easier to find. All the execution plan contains for these shell queries is a pointer to the plan for the parameterised version of the query.

For the ad-hoc query that gets inserted as a shell plan and the parameterised query, the objectID is, as with the CacheHit and CacheMiss events, a hash of the query text and does not match to any object in the database. The full text of the batch is then given in the TextData column.

I think that’s enough for CacheInsert. On to it’s opposite.

CacheRemove

The CacheRemove event fires, as one would expect, when a plan is removed from cache. The interesting question is, when does that occur?

Is it every event that could possibly result in an inefficient or invalid plan, or just memory pressure and explicit cache flushes that trigger the CacheRemove event?

Let’s see…

As an aside, I’m going to add a nonclustered index to the table that I’m using, so that the update of statistics will have an effect. Otherwise the only execution path possible is a table scan, and changing statistics won’t affect that.

CREATE NONCLUSTERED INDEX idx_TestingCacheEvents_Status
 ON  TestingCacheEvents (Status)
GO

Right, onto the testing.

Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO
DBCC FREEPROCCACHE -- Gone.  All gone
WAITFOR DELAY '00:00:05'
GO

Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO
DBCC  FREEPROCCACHE(0x0500060063A9355540614285000000000000000000000000) -- Just one removed, the proc
WAITFOR DELAY '00:00:05'
GO

Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO
exec sp_recompile  'TestingCacheEvents' -- recompile on the table, both proc and ah-hoc plans  invalidated.
WAITFOR DELAY '00:00:05'
GO

Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO
UPDATE TOP(1)  TestingCacheEvents
 SET Status = 'C'
 WHERE Status = 'B'
UPDATE STATISTICS TestingCacheEvents WITH FULLSCAN
WAITFOR DELAY  '00:00:05'
GO

Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO
ALTER TABLE  TestingCacheEvents ALTER COLUMN FILLER CHAR(325)
WAITFOR DELAY '00:00:05'
GO
Exec FireCacheEvents
GO
SELECT ID, SomeDate, Status
FROM  TestingCacheEvents
WHERE Status = 'C'
GO

Well, this is interesting…

The only things that caused the CacheRemove event to fire were the two DBCC FreeProcCache, the first that, as can be seen from the EventSubClass, cleared the entire plan cache, and the second that cleared a single plan.

If I go back and look at the whitepaper Plan Caching in SQL Server 2008, it differentiates between removing a plan from cache (which memory pressure and cache flushes do) and invalidating the plan (which recompiles, schema changes, stats changes and the like do). Based on that whitepaper and these results, I would conclude that the CacheRemove event only fires when the plan is actually removed from cache, not when it’s just invalidated. The invalidation of a plan simply results in a recompile next time the plan is needed.

Now that the question of when it appears is answered, let’s finish with what it looks like.

There are two possible formats for this event, and it’s the EventSubClass that shows which of the two the particular event is.

The first possibility is when all of the plan cache has been flushed, identified by a EventSubClass of “2 – ProcCacheFlush”. This will be a result of a DBCC FREEPROCCACHE and some server reconfiguration events. In this case, there will be no ObjectID and the text data will simply state that the cache has been flushed.

The second possibility is more interesting, especially if monitoring plans being aged out of memory. An EventSubClass of “1 – Compplan Remove” identifies a single plan being removed from cache. These are fired when either a single plan or part of the plan cache is cleared. Examples here are DBCC FREEPROCCACHE with a plan or SQL handle passed to it, DBCC FLUSHPROCINDB, database reconfigurations (restore, detach, offline)

With the one, the ObjectID does have a value and it’s the ObjectID of the pbject whose plan is being thrown out of cache. As with all the other cache events, for ad-hoc queries the ObjectID is just a hash of the query text and only really useful for matching to CacheMiss, CacheHit and CacheInsert events and the object name or ad-hoc batch appears in the TextData

That’s more than I planned to write, but CacheRemove turned out to be a little more complex than I expected. Next up, the last part of this sort series – the recompile events.

For people interested in the details of how SQL is using and reusing execution plans, there are some useful events in profiler for watching this in detail, under the Stored procedure group:

• SP:CacheMiss
• SP:CacheInsert
• SP:CacheHit
• SP:CacheRemove
• SP:Recompile
• SP:StmtRecompile

Additionally there’s the SQL:StmtRecompile event under the TSQL group.

For now, I just want to look briefly at the CacheMiss and CacheHit events.

One word of caution early on, these are frequently occurring events and it may not be a good idea to trace these on busy production servers. If you do need to, keep the duration of the trace short and the columns to a minimum.

CacheMiss

The cache miss event fires any time SQL looks for the execution plans for an object or ad-hoc batch and does not find it in the plan cache.

For an object (scalar function, multi-statement table-valued function, stored procedure or trigger) the match is done on the object ID (along with some of the connection’s SET options and possibly the database user and c couple other factors¹). For an ad-hoc batch, the match is done on a hash of the text of the batch (along with some of the connection’s SET options and possibly the database user)

When testing stored procedures from Management Studio (or another SQL querying tool), two CacheMiss events will appear in the trace.

What’s going on here?

Let’s start from the bottom and work up. The last event there, the SP:Completed records the completion of the stored procedure and lists both the ObjectID and ObjectName and, if I check in the database, ObjectID 1301579675 does indeed belong to the stored procedure FireCacheEvents.

The second event (the second cache miss) has the same ObjectID and ObjectName. So this is the failed cache lookup for the stored procedure (failed because this is the first time I ran it and the plan was, as a result, not in cache).

The first entry is the one that’s curious. The ObjectName is not populated and the ObjectID doesn’t match anything in the database. So what is the cache lookup trying to find?

The TextData column give a hint. What it’s trying to find is a cached plan for the ad-hoc batch containing the whole of the text submitted to SQL (In this case just ‘EXEC FireCacheEvents’). This too could contain queries (though it doesn’t in this case) and needs a plan lookup. With just an EXEC in the batch, it won’t have a plan and hence won’t be found in cache, but there’s still a lookup for it.

If there was any SELECT, INSERT, UPDATE or DELETE (or MERGE on SQL 2008) statement within that ad-hoc batch, the batch would also be cached and future cache lookups would succeed but, since it’s just an EXEC, there’s no plan to cache.

This additional cache lookup won’t occur if the execution is via RPC (e.g. A .Net call with the SQLCommand.CommandType = CommandType.StoredProcedure) but it will any time there is an ad-hoc SQL batch submitted (e.g. from Management Studio or .Net if the SQLCommand.CommandType is set to CommandType.Text)

That’s ad-hoc batches and stored procs. Let’s try the third possibility – a parameterised query.

This looks semi-familiar. The first CacheMiss is again for the entire of the ad-hoc batch (consisting of the DECLARE and the EXEC in this case). The second looks similar, something in TextData and no ObjectName. This is the parameterised query, identifiable as parameterised by the (@Status Char(1)) parameter definition right at the beginning. Again here the ObjectID is a hash of the text and doesn’t match to a real object in the database.

It is worth noting that, if both the CacheMiss and CacheInsert events are traced, the CacheMiss events will only appear if there is no subsequent CacheInsert for the same ObjectID. If I ran the exact same script as above, but had the SP:Completed, SP:CacheMiss and SP:CacheInsert events in the trace, there would still be only three events recorded, the cache miss for the ad-hoc batch (as that’s not cached there’s no matching CacheInsert event), the CacheInsert for the stored proc and then the SP:Completed. The CacheMiss for the procedure wouldn’t appear, though it’s presence can be intuited from the presence of the CacheInsert.

That should about cover it for the CacheMiss. If the failed cache lookup is for is an ad-hoc batch or parameterised query, the TextData column will be populated with the contents of the batch or query and the ObjectID will be a hash of the text (and shouldn’t match any object in the database). If the failed cache lookup is for a procedure (or function or trigger), the object name column is usually populated (not always) with the name of the object, TextData is blank and the ObjectID matches the ObjectID of the object in the database

CacheHit

Onto the CacheHit event. This, as its name implies, is the opposite to the CacheMiss. The CacheMiss indicates that a lookup to the plan cache failed to find a matching plan. The CacheHit indicates that a lookup to the plan cache succeeded in finding a matching plan (based on object id, set options, maybe user, and various other conditions).

It’s not certain, even if the cache lookup succeeds, that the plan will indeed be used for the execution of the query/batch/procedure as there are a number of stability and optimality related checks that will be done before the plan is used.

So let’s see how this event looks.

EXEC FireCacheEvents
GO

SELECT ID, SomeDate, Status
FROM dbo.TestingCacheEvents
WHERE Status = 'B'
GO 

Two ad-hoc batches, first with just a stored procedure call, second with just an ad-hoc SQL statement.

No big surprises here, not after looking at the CacheMiss events. The first is the CacheMiss for the first of the ad-hoc batches, the one with just the EXEC in it. Since that is not cached, there will be a CacheMiss every time that executes.

The first of the CacheHit events is for the stored procedure. As with the CacheMiss, this CacheHit for the stored procedure has an ObjectID that matches the ObjectID for that procedure in the database, and the ObjectName column is populated with (surprise) the object name while the TextData is blank.

The second CacheHit is for the ad-hoc batch with the SELECT statement. As is probably expected by this point, the ObjectID there is just a hash of the text, the ObjectName is blank and the TextData is populated with the full text of the batch.

In Conclusion

If you’re monitoring cache usage with the CacheMiss and CacheHit events, there are two different ways to identify what the lookup was looking for.

If the lookup was for the plan of a stored procedure, trigger or function, the ObjectID column contains a value that matches the ObjectID for that object in the database. The TextData column is blank and the ObjectName column is (usually) populated with the name of the object. I did encounter a couple of cases where the ObjectName was blank for a CacheMiss event for a stored procedure, not quite sure why. More investigation is necessary.

If the lookup was for the plan of an ad-hoc batch or parameterised query, the ObjectID contains a meaningless value, the ObjectName column is blank and the TextData contains the entire of the batch/query.

It may also be worth mentioning that the DatabaseID column is populated for all CacheMiss and CacheHit events, regardless of what the lookup is looking for. Additionally the DatabaseName column is populated for all CacheHit events (but is not an available column for the CacheMiss)

(1) For anyone who wants more information, there are two excellent resources available from Microsoft:

• Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
• Plan Caching in SQL Server 2008

Both go extremely deep into caching, what influences matching of plans, plan reuse, recompilations and the plan cache itself.

Reproduction code:

CREATE TABLE TestingCacheEvents (
 ID INT IDENTITY PRIMARY KEY,
 SomeDate DATETIME,
 Status CHAR(1),
 Filler CHAR(300) DEFAULT ' '
);
GO

INSERT INTO TestingCacheEvents (SomeDate, Status)
SELECT TOP (10000)
 DATEADD(dd,FLOOR(RAND(a.object_id+b.column_id*5000)*500),'2000/01/01'),
 CHAR(65+FLOOR(RAND(b.object_id+a.column_id*5000)*10))
 FROM master.sys.columns a CROSS JOIN master.sys.columns b;
GO

CREATE PROCEDURE FireCacheEvents
AS
 SELECT ID, SomeDate, Status
 FROM TestingCacheEvents
 WHERE Status = 'G'

GO

A trivial plan is essentially a plan for a query where a specific plan will always be the most optimal way of executing it. If we consider something like SELECT * FROM SomeTable then there’s only one real way to execute it, a scan of the cluster/heap.

The trivial plan is somewhat of a query optimiser optimisation. If the query qualifies for a trivial plan (and there are lots of restrictions) then the full optimisation process doesn’t need to be started and so the query’s execution plan can be generated quicker and with less overhead. The fact that a query has a trivial plan at one point doesn’t necessarily mean that it will always have a trivial plan, indexes may be added that make the selection of plan less of a sure thing and so the query must go for full optimisation, rather than getting a trivial plan

Nice theory, but how does one tell if a particular query has a trivial execution plan? The information is found within the execution plan, the properties of the highest-level operator has an entry ‘Optimisation level’ For a trivial plan this will read ‘TRIVIAL’

It’s also, for the brave people who like XML, found within the xml form of the plan

<StmtSimple StatementCompId='1' StatementEstRows='11' StatementId='1' StatementOptmLevel='TRIVIAL' StatementSubTreeCost='0.0032941' StatementText='SELECT * FROM forums' StatementType='SELECT' QueryHash='0xB38EBF594006422E' QueryPlanHash='0xCC9AB99E7081C81D'>
<!-- most of the rest of the plan here -->
</StmtSimple>

So, are they cached? The way to find that out is to run a variety of queries and see what’s sitting in the cache afterwards.

I’m going to clear the procedure cache then run a couple queries against the AdventureWorks database, checking the graphical execution plan for each one. After they’ve all been run, I’ll query the plan cache and check the optimisation level of the cached plans,

Query 1:

SELECT FirstName, LastName
    FROM Person.Person
    WHERE BusinessEntityID = 42

According to the graphical plan, this query has a trivial plan.

Query 2:

SELECT TOP (10) Name FROM Production.Product

According to the graphical plan, this query also has a trivial plan.

Query 3:

SELECT * FROM Sales.SalesOrderHeader sh
    INNER JOIN sales.SalesOrderDetail sd
        ON sh.SalesOrderID = sd.SalesOrderID
    WHERE sh.ShipDate > '2008/05/25'

According to the graphical plan, this query has a non-trivial plan, the optimisation level is listed as full.

SELECT st.text, qp.query_plan,
     qp.query_plan.value('
       declare default element
       namespace "http://schemas.microsoft.com/sqlserver/2004/07/showplan";
       (//StmtSimple/@StatementOptmLevel)[1]','varchar(20)') AS OptimisationLevel
    FROM sys.dm_exec_cached_plans cp
        CROSS APPLY sys.dm_exec_query_plan(cp.plan_handle) qp
        CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st
    WHERE text not like '%sys.dm_exec_cached_plans%'

This returns 4 rows. All three of the queries that I ran against AdventureWorks, two with optimisation levels of Trivial, one with an optimisation level of Full, and the unparameterised ‘shell’ of the first of the queries.

So it appears that some trivial plans are indeed cached.

Let’s take a look at an example. (table creation code at the end of the post)

select bt.id, bt.SomeColumn, st.SomeArbDate
from dbo.BigTable bt
inner join dbo.SmallerTable st on bt.SomeColumn = st.LookupColumn
where bt.id between 5000 and 5100

Estimated rows = 1, actual rows = 101. That’s a large discrepancy, but what caused it? It’s not out of date statistics (a usual cause) because the table has only just been created, so why is the estimation so far from the actual.

Let’s take a closer look at that seek. The seek predicate is an equality match on LookupColumn. That can only return 1 row because when the table was populated it was populated with unique values for that column (though the index on that column is not defined unique). So the estimated row count is dead-on, the index seek will return  a single row. Now the question is where that 101 for the actual comes from.

This seek is on the inner table of a nested loop join. The way the nested loop join works is to query the outer table of the join and then to query the inner table once for each row returned by the outer table. A look at the details of the clustered index scan that defines the outer table of the nested loop shows that it returns 101 rows.

Since the outer table returns 101 rows, the seek on the inner table must be done 101 times. That’s supported by the execution count shown on the inner seek. That’s where the discrepancy between actual and estimated rows comes from.

When an operator is executed multiple times as part of the query execution, the estimated row count refers to the number of rows that the optimiser estimates will be affected per execution. The actual row count refers to the total number of rows that the operator affected, cumulative over all executions. So when checking to see if there’s a major discrepancy between estimated and actual rows counts, the actual row count has to be divided by the number of executions.

That’s fine when using SQL 2008′s management studio, which exposes the execution count of an operator in the tooltip of that operator. SQL 2005′s management studio did not display the execution count anywhere convenient, though it is present in the XML of the plan. This is purely a feature of the version of Management Studio. The SQL 2008 management studio will display the execution count regardless of whether it’s connected to SQL 2005 or to SQL 2008.

For those still using SQL 2005′s tools, if you want the execution count, this is where to look:

<RelOp AvgRowSize="15" EstimateCPU="0.0001581" EstimateIO="0.003125" EstimateRebinds="85.432" EstimateRewinds="0" EstimateRows="1" LogicalOp="Index Seek" NodeId="2" Parallel="false" PhysicalOp="Index Seek" EstimatedTotalSubtreeCost="0.0480212" TableCardinality="3956">
<OutputList>
<ColumnReference Database="[Testing]" Schema="[dbo]" Table="[SmallerTable]" Alias="[st]" Column="SomeArbDate" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="101" ActualEndOfScans="101" ActualExecutions="101" />
</RunTimeInformation>

The number of executions, along with the actual row count is contained within the XML node RunTimeInformation. Obviously this node will not be present when looking at an estimated execution plan or an execution plan retrieved from the plan cache, as neither contains any run-time information.

Table creation code.

Edit: In the original post I left 2 indexes out of the table creation, which changed the behaviour of the query completely (hash joins instead of nested loop). If you tried to reproduce my results and couldn’t, you should be able to now with the correct indexes.

Create Table BigTable (
id int identity primary key,
SomeColumn char(4),
Filler char(100)
)

Create Table SmallerTable (
id int identity primary key,
LookupColumn char(4),
SomeArbDate Datetime default getdate()
)

INSERT INTO BigTable (SomeColumn)
SELECT top 250000
char(65+FLOOR(RAND(a.column_id *5645 + b.object_id)*10)) + char(65+FLOOR(RAND(b.column_id *3784 + b.object_id)*12)) +
char(65+FLOOR(RAND(b.column_id *6841 + a.object_id)*12)) + char(65+FLOOR(RAND(a.column_id *7544 + b.object_id)*8))
from master.sys.columns a cross join master.sys.columns b

INSERT INTO SmallerTable (LookupColumn)
SELECT DISTINCT SomeColumn
FROM BigTable TABLESAMPLE (25 PERCENT)
GO

CREATE NONCLUSTERED INDEX [idx_BigTable_SomeColumn]
  ON [dbo].[BigTable] ([SomeColumn] ASC)
GO

CREATE NONCLUSTERED INDEX [idx_SmallerTable_LookupColumn]
  ON [dbo].[SmallerTable] ([LookupColumn] ASC)
  INCLUDE ( [SomeArbDate])
GO

The only thing estimated about the estimated execution plan is the rowcounts, costs and row size. The plan itself isn’t an estimate. It’s not as if the optimiser, when asked for an estimated plan, does a less thorough job than when asked to compile a plan for a query’s execution.

The two forms of execution plan are better described as ‘execution plan with run-time information’ and ‘execution plan without run-time information’

When, in Management Studio, someone clicks the ‘display estimated execution plan’ button, the query is submitted to SQL Server, parsed and bound, algebratised and optimised just as if it was going to be executed. But the query is not executed, and as such, the plan when returned contains no run time information.

If there is a matching cached query plan, that cached plan is what’s returned and no optimisation is done. This can be seen by using profiler with the Cache hit, cache miss and cache insert events being traced.

When, in Management Studio, the query is run with the execution plan option enabled, the query is submitted to SQL Server, parsed and bound, algebratised, optimised and executed. The returned plan does contain the run-time for that specific execution, hence the plan contains things like ‘actual row count, actual IO cost’, etc

If there’s a matching query plan in cache then that cached plan will be used for the query’s execution and will be the one returned, though with the run-time information added

When a plan is cached, only the compile-time information is cached. The detailed run-time information on the actual number of rows, costs and the like is (I believe) discarded after updating the aggregated query stats. Hence, when you retrieve a query from the plan cache, it will not contain the run-time information. Consider a plan that’s been used 20 times. Which execution’s run-time information would it contain? Remember that there’s only one plan in cache per procedure.

Hence, a plan fetched from will be identical to the plan returned by requesting the estimated execution plan for a specific query (Providing there’s nothing happening to invalidate the cached plan)

Profiler can capture (depending on event) the plan without the run-time information or the plan with the run-time information. There’s a nice table in chapter 2 of one of Itzik’s books that shows the various profiler events, when they fire and what they return.

The operator that I’m talking about is the one that fetches extra columns from the clustered index when the nonclustered index that’s used to retrieve the rows doesn’t have all of the columns required.

In SQL 2000, that operator appeared in the execution plan as a bookmark lookup and it appeared as follows:

In SQL 2005, the name was changed, and the bookmark lookup no longer appeared in the execution plan. In it’s place appeared a clustered index seek, joined back to the original index seek by a nested loop join. It appeared as follows (the highlighted operator is the ‘bookmark lookup’)

This change made it harder to see what was going on as clustered index seeks are usually considered ‘good’. The trick to see if it is actually a ‘bookmark lookup’ is to look at the objects involved. When the nonclustered index seek and the clustered index seek are both on the same table, then it’s a ‘bookmark lookup’

I don’t recall what this appeared as when the base table was a heap, not a cluster.

In SQL 2005 SP2, the name of the operator was changed again, now appearing as a key lookup (when the base table has a clustered index) or a RID lookup (when the base table is a heap). It now looks like this:

The thing to note is that it’s not the version of the server that’s important. The format of the XML for the exec plan has not changed since SQL 2005 RTM (I can and have created a .sqlplan file from SQL 2008 and opened that file in SQL 2005′s management studio).

It’s the version of management studio that affects how the execution plans are displayed. If the server is SQL 2005 SP3, but the client tools are still RTM, the bookmark lookup will appear as a clustered index seek. Another reason to patch the client as well as the server

I hope that clears up some of the confusion around the naming. So, in future, what should I refer to this as? A bookmark lookup? A Key lookup?