Generic caching framework for Sitecore with relevant clearance

Caching with clearance on a relevant publish

As you all know caching is important and used by Sitecore a lot. But after a publish Sitecore clears caches rather like an elephant - just remove it no matter what was published. We have been using a custom framework for quite a while that allows us to use granular caching of data - data that we have put in our own cache that is, not the actual Sitecore caches - and clearance only when relevant.
It's a long post.. but mainly just a lot of code ;)

The idea

The idea behind it is rather simple: you can put objects in the cache and you add a removal strategy for that cache entry. After a publish we run a cache clear process that calls all the removal strategies. Each strategy can decide on the information from the publish if the cache entry should be removed or not. This way we make each cache entry responsible for its own removal logic. Which is good as it makes it very flexible and we keep the logic together with the actual data. Note that the removal strategy is not limited to removing the object from the cache - it might also perform other actions if needed.

The code

Before we take a look at the code, let's check the dependency graph:

Let 's start with the CacheEntry class. This is the object we will store in the cache. It contains the value (actual object you want to cache) and the PublishRemoveStrategy. It also contains the function to invoke the removal strategy.

public class CacheEntry : ICacheEntry
{
  public CacheEntry(object valueToCache)
  {
    CachedValue = valueToCache;
  }

  public CacheEntry(object valueToCache, PublishRemoveStrategy publishRemoveStrategy)
  {
    CachedValue = valueToCache;
    PublishRemoveStrategy = publishRemoveStrategy;
  }

  public object CachedValue { get; }

  public PublishRemoveStrategy PublishRemoveStrategy { get; }

  /// <summary>
  /// Runs the on-publish remove strategy for the entry.
  /// The strategy (if provided) is expected to do the necessary work such as actually removing the entry from the cache.
  /// </summary>
  /// <param name="cache">Cache instance</param>
  /// <param name="key">Cache entry key</param>
  /// <param name="language">Publish language</param>
  /// <param name="database">Target Database</param>
  /// <param name="rootNode">Root node of the publish action</param>
  /// <param name="publishMode">Publish mode</param>
  public virtual void RunOnPublishRemoveStrategy(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    PublishRemoveStrategy?.Invoke(cache, key, language, database, rootNode, publishMode);
  }
}

The PublishRemoveStrategy strategy itself is a delegate that requires the language, the target database, the root node and the mode (of the publish):

public delegate void PublishRemoveStrategy(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);

The core of the module is the actual SitecoreCache class:

using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class SitecoreCache : ISitecoreCache
{
  private const string LanguageIndependentKey = "xx";

  public virtual void Add(string key, ICacheEntry cacheEntry, string language)
  {
    HttpRuntime.Cache.Insert(BuildKey(key, language), cacheEntry);
  }

  public virtual void Add(string key, ICacheEntry cacheEntry)
  {
    Add(key, cacheEntry, null);
  }

  public virtual object GetObject(string key, string language)
  {
    var cacheObject = HttpRuntime.Cache[BuildKey(key, language)] as ICacheEntry;
    return cacheObject?.CachedValue;
  }

  public virtual object GetObject(string key)
  {
    return GetObject(key, null);
  }

  public virtual T GetObject<T>(string key, string language) where T : class
  {
    return GetObject(key, language) as T;
  }

  public virtual T GetObject<T>(string key) where T : class
  {
    return GetObject<T>(key, null);
  }

  public virtual void Remove(string key, string language)
  {
    HttpRuntime.Cache.Remove(BuildKey(key, language));
  }

  public virtual void Remove(string key)
  {
    Remove(key, null);
  }

  public virtual void RemoveItemsOnPublish(Database database, Item rootNode, PublishMode publishMode, Language language)
  {
    if (language == null)
    {
    return;
    }

    var prefixLanguage = BuildKey(string.Empty, language.Name);
    var entriesInLanguage = from DictionaryEntry entry in HttpRuntime.Cache
          let key = entry.Key as string
          where IsValidKey(key, prefixLanguage)
          select entry;

    foreach (var entry in entriesInLanguage)
    {
    if (!(entry.Value is ICacheEntry cacheObject))
    {
      continue;
    }

    var key = ExtractKey(entry.Key.ToString());
    cacheObject.RunOnPublishRemoveStrategy(this, key, language, database, rootNode, publishMode);
    }
  }

  public virtual void Clear()
  {
    var keys = (from DictionaryEntry entry in HttpRuntime.Cache where entry.Value is ICacheEntry select entry.Key).ToList();

    foreach (string key in keys)
    {
      Remove(key);
    }
  }

  private static string BuildKey(string key, string language)
  {
    var prefix = language?.ToLowerInvariant() ?? LanguageIndependentKey;
    return prefix + "|" + key;
  }

  private static string ExtractKey(string compositeKey)
  {
    return compositeKey.Substring(compositeKey.IndexOf('|') + 1);
  }

  private static bool IsValidKey(string key, string prefixLanguage)
  {
    return !string.IsNullOrEmpty(key)
    && (key.StartsWith(prefixLanguage, StringComparison.OrdinalIgnoreCase) || key.StartsWith(LanguageIndependentKey, StringComparison.OrdinalIgnoreCase));
  }
}

The SitecoreCache class has functions to:

• Add entries: with or without language (some entries might be language dependent, others are not)
• Get entries: again with or without the language, but also a (generic) version that will try to cast to a given type
• Remove entries or Clear all entries
• Handle the publish event: this gets all entries from the cache and calls their remove strategy

Entries are stored with a key (any string -should be unique- given by the using application that is combined with the language).

Clearing the cache

Clearing the cache is done in the CacheClearer class that we will attach to publish events:

using Sitecore.Common;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Events;
using Sitecore.Events;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class CacheClearer
{
  private readonly ISitecoreCache sitecoreCache;

  public CacheClearer(ISitecoreCache sitecoreCache)
  {
    this.sitecoreCache = sitecoreCache;
  }

  public event EventHandler SitecoreCacheClearedHandler;

  public virtual void ClearCache(object sender, EventArgs args)
  {
    var publishingOptions = FetchPublishingOptions(args).ToList();

    if (!publishingOptions.Any())
    {
      return;
    }

    if (sitecoreCache == null)
    {
      return;
    }

    RemoveFromCache(publishingOptions);
  }

  protected virtual void AfterSitecoreCacheCleared(EventArgs e)
  {
    if (SitecoreCacheClearedHandler == null)
    {
      return;
    }

    SitecoreCacheClearedHandler(this, e);
  }

  protected virtual IEnumerable<DistributedPublishOptions> FetchPublishingOptions(EventArgs args)
  {
    var sitecoreRemoteArgs = args as PublishCompletedRemoteEventArgs;
    if (sitecoreRemoteArgs != null)
    {
      if (sitecoreRemoteArgs.PublishOptions != null)
      {
        return sitecoreRemoteArgs.PublishOptions;
      }

      return new List<DistributedPublishOptions>();
    }

    var sitecoreArgs = args as SitecoreEventArgs;
    if (sitecoreArgs == null)
    {
      return new List<DistributedPublishOptions>();
    }

    var publishingParameter = sitecoreArgs.Parameters[0] as IEnumerable<DistributedPublishOptions>;

    if (publishingParameter != null)
    {
      return publishingParameter;
    }

    return new List<DistributedPublishOptions>();
  }

  protected virtual void RemoveFromCache(IEnumerable<DistributedPublishOptions> publishingOptions)
  {
    foreach (var publishingOption in publishingOptions)
    {
      var database = GetDatabase(publishingOption.TargetDatabaseName);
      if (database == null)
      {
        continue;
      }

      RemoveFromCache(database, publishingOption.RootItemId.ToID(), publishingOption.LanguageName, publishingOption.Mode);
    }

    AfterSitecoreCacheCleared(EventArgs.Empty);
  }

  protected virtual void RemoveFromCache(Database database, ID id, string languageName, PublishMode mode)
  {
    Language publishingLanguage;
    if (!Language.TryParse(languageName, out publishingLanguage))
    {
      return;
    }

    var rootItem = database.GetItem(id, publishingLanguage);
    sitecoreCache.RemoveItemsOnPublish(database, rootItem, mode, publishingLanguage);
  }

  protected virtual Database GetDatabase(string name)
  {
    if (!Factory.GetDatabaseNames().Contains(name, StringComparer.OrdinalIgnoreCase))
    {
      return null;
    }

    return Factory.GetDatabase(name);
  }
}

In this class we are actually not doing much more than fetching the publish options and getting the parameters to call the RemoveItemsOnPublish from our SitecoreCache: the target database, the root item, the language and the publish mode. When fetching the publish options, we are trying twice to support local and remote events. We could also split the functions but decided not to. The SitecoreCacheClearedHandler event can be used to inject extra code after the clear.

Attaching the clearance code to the events is done with configuration:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <events>
      <event name="publish:complete">
        <handler type="XX.Caching.CacheClearer, XX" method="ClearCache" resolve="true" />
      </event>
      <event name="publish:complete:remote">
        <handler type="XX.Caching.CacheClearer, XX" method="ClearCache" resolve="true" />
      </event>
    </events>
  </sitecore>
</configuration>

Note that the clear code is attached to the local and the remote publish event to support scaled architectures.

Cache removal strategies

The last part of the puzzle is the actual cache removal strategies. You can easily write your own one if needed and use that, but we provided some that cover most of our needs. Each removal strategy class must implement the ICacheRemoval interface which has two functions: one for clearing the cache with a language and one without:

public interface ICacheRemovalStrategy where T : class
{
  void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);

  void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);
}

Four examples of a removal strategy:
1. Always: this strategy will always clear the cache, no matter what the options are.

public class Always : ICacheRemovalStrategy<object>
{
  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache != null && language != null)
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache != null)
    {
      cache.Remove(key);
    }
  }
}

2. On root node: this will remove the entry if the root node of the publish is the item or one of its descendants of the given parent - the parent is injected through a method that fetches the item based on the database.

using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class OnRootNode : ICacheRemovalStrategy<Item>
{
  public OnRootNode(Func<Database, Item> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<Database, Item> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    if (IsDescendant(database, rootNode))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    if (IsDescendant(database, rootNode))
    {
      cache.Remove(key);
    }
  }

  private bool IsDescendant(Database database, Item publishRootNode)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var parent = GetDependencies?.Invoke(database);
    
    if (parent == null)
    {
      return false;
    }

    return publishRootNode.ID.Guid.Equals(parent.ID.Guid) || publishRootNode.Paths.IsDescendantOf(parent);
  }
}

3. On templates: this will remove the entry if the id of the template of the root node of the publish is amongst the given template id's

using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class OnTemplates : ICacheRemovalStrategy<IEnumerable<Guid>>
{
  public OnTemplates(Func<IEnumerable<Guid>> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<IEnumerable<Guid>> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    if (IsTemplateBased(rootNode))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    if (IsTemplateBased(rootNode))
    {
      cache.Remove(key);
    }
  }

  private bool IsTemplateBased(Item publishRootNode)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var templates = GetDependencies?.Invoke();
    
    if (templates == null)
    {
      return false;
    }

    return templates.ToList().Contains(publishRootNode.TemplateID.Guid);
  }
}

4. On root node & template: combination of the 2 above

using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class TemplatesRootNode
{
  public Item RootNode { get; set; }
  public IEnumerable<Guid> Templates { get; set; }
}

public class OnTemplatesRootNode : ICacheRemovalStrategy<TemplatesRootNode>
{
  public OnTemplatesRootNode(Func<TemplatesRootNode> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<TemplatesRootNode> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    var dependency = Dependencies();
    
    if (IsTemplateBased(rootNode, dependency) && IsDescendant(rootNode, dependency))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    var dependency = Dependencies();
    
    if (IsTemplateBased(rootNode, dependency) && IsDescendant(rootNode, dependency))
    {
      cache.Remove(key);
    }
  }

  private TemplatesRootNode Dependencies()
  {
    return GetDependencies?.Invoke();
  }

  private static bool IsTemplateBased(Item publishRootNode, TemplatesRootNode dependency)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    if (dependency == null)
    {
      return false;
    }

    return dependency.Templates.ToList().Contains(publishRootNode.TemplateID.Guid);
  }

  private static bool IsDescendant(Item publishRootNode, TemplatesRootNode dependency)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var parent = dependency?.RootNode;

    if (parent == null)
    {
      return false;
    }

    return publishRootNode.ID.Guid.Equals(parent.ID.Guid) || publishRootNode.Paths.IsDescendantOf(parent);
  }
}

Usage

If you're still here reading after such a long post, you probably also want to see an example of how to use this:

var rootNode = .. // get the parent item
var removeStrategy = new OnTemplatesRootNode(() => GetRemovalStrategy(rootNode));
sitecoreCache.Add(CacheKey, new CacheEntry(myObject, removeStrategy.RemoveInLanguage), language);

private TemplatesRootNode GetRemovalStrategy(Item rootItem)
{
  return new TemplatesRootNode
  {
    RootNode = rootItem,
    Templates = ..  // get the id's of the templates
  };
}

In this example I am caching a language dependent object and am using the combination of templates and a root node. The calls to fetch the parent node and the ids of the templates are left blank as those are really dependent on your solution.

Conclusion

I hope this all made some sense (and I didn't forget anything). It has proven to be a rather powerful way of caching data and clearing the data based on relevant Sitecore publish events. It's not perfect yet -there are still some ideas to extend and improve- but it's a good start. So if you did get this far I hope you enjoyed it - any feedback is welcome.

One last remark: be aware that objects returned from the cache are references - if you change them you also change the cached value! If you don't want this, take a copy first.

Thanks to Özkan Sayın for reading and correcting the draft version.