Well, it’s been a while since I did my initial review of some simple Linq performance tests. Since then, I’ve done a bit more testing of Linq performance and I’d like to share that. The results are enlightening, to say the least. I did this because I’ve gotten a lot of questions regarding the performance of Linq and, in particular, Linq to Sql – something that is common whenever there is a new data-oriented API. Now, let me also say that performance isn’t the only consideration … there are also considerations of functionality and ease of use, as well as the overall functionality of the API and its applicability to a wide variety of scenarios. I used the same methodology that I detailed in this previous post.

Now, all of the tests were against the AdventureWorks sample database’s Person.Contact table with some 20,000 rows. Not the largest table in the world, but it’s also a good deal larger that the much-beloved Northwind database. I also decided to re-run all of the tests a second time on my home PC (rather than my laptop) as the client and one of my test servers as the database server. The specs are as follows:

So, with that out of the way, let’s discuss the first test.

Simple Query

This is a simple “SELECT * FROM Person.Contact” query … nothing special or funky. From there, as with all of the tests, I loop through the results and assign them to temporary, local variables. An overview of the tests is below:

int firstName = rdr.GetOrdinal("FirstName");
int lastName = rdr.GetOrdinal("LastName"); 
while (rdr.Read())
{
    string fullName = rdr.GetString(firstName) + rdr.GetString(lastName);
}
rdr.Close();

while (rdr.Read())
{
    string fullName = rdr.GetString(0) + rdr.GetString(1);
}
rdr.Close();

while (rdr.Read())
{
    string fullName = (string)rdr["FirstName"] + (string)rdr["LastName"];
}
rdr.Close();

var contactNames = from c in dc.Contacts
                   select new { c.FirstName, c.LastName };
foreach (var contactName in contactNames)
{
    string fullName = contactName.FirstName + contactName.LastName;
}

IQueryable<AdvWorksName> contactNames = from c in dc.Contacts
                   select new AdvWorksName()
                    {FirstName= c.FirstName, LastName= c.LastName };
foreach (var contactName in contactNames)
{
    string fullName = contactName.FirstName + contactName.LastName;
}

DataSet ds = new DataSet();
                ds.RemotingFormat = SerializationFormat.Binary; 
                SqlDataAdapter adp = new SqlDataAdapter(cmd);
                adp.Fill(ds);
                foreach (DataRow dr in ds.Tables[0].Rows)
                {
                    string fullName = dr.Field<String>("FirstName") + dr.Field<String>("LastName"); 
                }
                cnct.Close();

IQueryable<AdvWorksNameProps> contactNames = from c in dc.Persons
                                        select new AdvWorksNameProps() { FirstName = c.FirstName, LastName = c.LastName };
foreach (var contactName in contactNames)
{
    string fullName = contactName.FirstName + contactName.LastName;
}

IQueryable<AdvWorksNameProps> contactNames = from c in dc.Persons
                                        select new AdvWorksNameProps(c.FirstName,  c.LastName);
foreach (var contactName in contactNames)
{
    string fullName = contactName.FirstName + contactName.LastName;
}

If you are wondering why the different “flavors” of Linq … it’s because, when I first started re-running these tests for the blog, I got some strange differences that I hadn’t seen before between (what is now) LinqAnonType and LinqClassField. On examination, I found that these things made a difference and wanted to get a more rounded picture of what we were looking at here … so I added a couple of tests.

And the results …

These results are actually quite different from what I saw when I ran the tests on a single machine … which is quite interesting and somewhat surprising to me. Linq still does very well when compared to DataReaders … depending on exactly how you implement the class. I didn’t expect that the version using the constructor would turn out to be the one that had the worst performance … and I’m not really sure what to make of that. I was surprised to see the DataSet do so well … it didn’t on previous tests, but in those cases, I also didn’t change the remoting format to binary; this does have a huge impact on the load performance, especially as the datasets get larger (XML gets pretty expensive when it starts getting big).

I’ve got more tests, but due to the sheer length of this post, I’m going to post them separately.

OK, well, I think it's cool (and since the mind is its own place ...). I've been a big fan of ASP.net's cache API since I found out it way back in the 1.0 beta. It certainly solves something that was problematic in ASP "Classic" in a clean, elegant and darn easy to use way. Unfortunately, not a lot of folks seem to know about it. So I'll start with a little overview of ASP.net caching.

As the name implies, it's a cache that sits server side. All of the relevant, .Net-supplied classes are in the System.Web.Caching namespace and the class representing the cache itself is System.Web.Caching.Cache. You can access it from the current HttpContext (which you'll see). The management of the cache is handled completely by ASP.net ... you just have to add objects to it and then read from it. When you add to the cache, you can set options like dependencies, expiration, priority and a delegate to call when the item is removed from the cache. Dependencies are interesting ... they will automatically invalidate (and remove) the cache item based on notification from the dependency. ASP.net 1.x had only 1 cache dependency class (System.Web.Caching.CacheDependency) that allowed you to have a dependency on a file, another cache item, and array of them or another CacheDependency. Framework 2.0 introduced System.Web.Caching.SqlCacheDependency for database dependencies and System.Web.Caching.AggregateCacheDependency for multiple, related dependencies. With the AggregateCacheDependency, if one of the dependencies changes, it item is invalidated and tossed from the cache. Framework 2.0 also (finally) "unsealed" the CacheDependency class, so you could create your own cache dependencies. With expiration, you can have an absolute expiration (specific time) or a sliding expiration (TimeSpan after last access). Priority plays into the clean-up algorithm; the Cache will remove items that haven't expired if the cache taking up too much memory/resources. Items with a lower priority are evicted first. Do yourself a favor and make sure that you keep your cache items reasonable. Your AppDomain will thank you for it.

ASP.net also provides page and partial-page caching mechanisms. That, however, is out of our scope here. For the adventurous among that don't know what that is ...

So ... the cache ... mmmmm ... yummy ... gooooood. It's golly-gee-gosh-darn useful for items that you need on the site, but don't change often. Those pesky drop-down lookup lists that come from the database are begging to be cached. It takes a load off the database and is a good way to help scalability - at the cost of server memory, of course. (There ain't no free lunch.) Still, I'm a big fan of appropriate caching.

So ... what's the technique I mentioned that this post is title after? Well, it's actually quite simple. It allows you to have 1 single common method to add and retrieve items from the cache ... any Linq item, in fact. You don't need to know anything about the cache ... just the type that you want and the DataContext that it comes from. And yes, it's one method to rule them all, suing generics (generics are kewl!) and the Black Voodoo Majick goo. From there, you can either call it directly from a page or (my preferred method) write a one-line method that acts as a wrapper. The returned objects are detached from the DataContext before they are handed back (so the DataContext doesn't need to be kept open all) and returned as a generic list object. The cache items are keyed by the type name of the DataContext and the object/table so that it's actually possible to have the same LinqToSql object come from two different DataContexts and cache both of them. While you can load up the cache on application start up, I don't like doing that ... it really is a killer for the app start time. I like to lazy load on demand. (And I don't wanna hear any comments about the lazy.)

Here's the C# code:

/// <summary>
/// Handles retrieving and populating Linq objects in the ASP.NET cache
/// </summary>
/// <typeparam name="LinqContext">The DataContext that the object will be retrieved from.</typeparam>
/// <typeparam name="LinqObject">The object that will be returned to be cached as a collection.</typeparam>
/// <returns>Generic list with the objects</returns>
public static List<LinqObject> GetCacheItem<LinqContext, LinqObject>()
    where LinqObject : class
    where LinqContext : System.Data.Linq.DataContext, new()
{
    //Build the cache item name. Tied to context and the object.
    string cacheItemName = typeof(LinqObject).ToString() + "_" + typeof(LinqContext).ToString();
    //Check to see if they are in the cache. 
    List<LinqObject> cacheItems = HttpContext.Current.Cache[cacheItemName] as List<LinqObject>;
    if (cacheItems == null)
    {
        //It's not in the cache -or- is the wrong type. 
        //Create a new list.
        cacheItems = new List<LinqObject>();
        //Create the contect in a using{} block to ensure cleanup. 
        using (LinqContext dc = new LinqContext())
        {
            try
            {
                //Get the table with the object from the data context.
                System.Data.Linq.Table<LinqObject> table = dc.GetTable<LinqObject>();
                //Add to the generic list. Detaches from the data context. 
                cacheItems.AddRange(table);
                //Add to the cache. No absolute expirate and a 60 minute sliding expiration
                HttpContext.Current.Cache.Add(cacheItemName, cacheItems, null,
                    System.Web.Caching.Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(60),
                    System.Web.Caching.CacheItemPriority.Normal, null);
            }
            catch (Exception ex)
            {
                //Something bad happened.
                throw new ApplicationException("Could not retrieve the request cache object", ex);
            }
        }
    }
    //return ... 
    return cacheItems;
}

And in VB (see, I am multi-lingual!) ...

''' <summary>
''' Handles retrieving and populating Linq objects in the ASP.NET cache
''' </summary>
''' <typeparam name="LinqContext">The DataContext that the object will be retrieved from.</typeparam>
''' <typeparam name="LinqObject">The object that will be returned to be cached as a collection.</typeparam>
''' <returns>Generic list with the objects</returns>
Public Shared Function GetCacheItem(Of LinqContext As {DataContext, New}, LinqObject As Class)() As List(Of LinqObject)
    Dim cacheItems As List(Of LinqObject)

    'Build the cache item name. Tied to context and the object.
    Dim cacheItemName As String = GetType(LinqObject).ToString() + "_" + GetType(LinqContext).ToString()
    'Check to see if they are in the cache. 
    Dim cacheObject As Object = HttpContext.Current.Cache(cacheItemName)

    'Check to make sure it's the correct type.
    If cacheObject.GetType() Is GetType(List(Of LinqObject)) Then
        cacheItems = CType(HttpContext.Current.Cache(cacheItemName), List(Of LinqObject))
    End If

    If cacheItems Is Nothing Then
        'It's not in the cache -or- is the wrong type. 
        'Create a new list.
        cacheItems = New List(Of LinqObject)()
        'Create the contect in a using   block to ensure cleanup. 
        Using dc As LinqContext = New LinqContext()
            Try
                'Get the table with the object from the data context.
                Dim table As Linq.Table(Of LinqObject) = dc.GetTable(Of LinqObject)()
                'Add to the generic list. Detaches from the data context. 
                cacheItems.AddRange(table)
                'Add to the cache. No absolute expirate and a 60 minute sliding expiration
                HttpContext.Current.Cache.Add(cacheItemName, cacheItems, Nothing, _
                    Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(60), _
                   CacheItemPriority.Normal, Nothing)
            Catch ex As Exception
                'Something bad happened.
                Throw New ApplicationException("Could not retrieve the request cache object", ex)
            End Try
        End Using
    End If

    'return ... 
    Return cacheItems
End Function

The comments, I think, pretty much say it all. It is a static method (and the class is a static class) because it's not using any private fields (variables). This does help performance a little bit and, really, there is no reason to instantiate a class if it's not using any state. Also, note the generic constraints - these are actually necessary and make sure that we aren't handed something funky that won't work. These constraints are checked and enforced by the compiler.

Using this to retrieve cache items is now quite trivial. The next example shows a wrapper function for an item from the AdventureWorks database. I made it a property but it could just as easily be a method. We won't get into choosing one over the other; that gets religious.

public static List<StateProvince> StateProvinceList
{
    get
    {
        return GetCacheItem<AdvWorksDataContext, StateProvince>(); 
    }
}

And VB ...

Public ReadOnly Property StateProvinceList() As List(Of StateProvince)
    Get
        Return GetCacheItem(Of AdvWorksDataContext, StateProvince)()
    End Get
End Property

Isn't that simple? Now, if you only have one DataContext type, you can safely code that type into the code instead of taking it as a generic. However, looking at this, you have to admit ... you can use this in any ASP.net project where you are using Linq to handle the cache. I think it's gonna go into my personal shared library of tricks.

As I think you can tell, I'm feeling a little snarky. It's Friday afternoon so I have an excuse. BTW ... bonus points to whoever can send me an email naming the lit reference (and finish it!) in this entry. Umm, no it isn't Lord of the Rings.