Showing posts with label publish. Show all posts
Showing posts with label publish. Show all posts

Monday, May 13, 2019

Generic caching framework for Sitecore with relevant clearance

Caching with clearance on a relevant publish

As you all know caching is important and used by Sitecore a lot. But after a publish Sitecore clears caches rather like an elephant - just remove it no matter what was published. We have been using a custom framework for quite a while that allows us to use granular caching of data - data that we have put in our own cache that is, not the actual Sitecore caches - and clearance only when relevant.
It's a long post.. but mainly just a lot of code ;)

The idea

The idea behind it is rather simple: you can put objects in the cache and you add a removal strategy for that cache entry. After a publish we run a cache clear process that calls all the removal strategies. Each strategy can decide on the information from the publish if the cache entry should be removed or not. This way we make each cache entry responsible for its own removal logic. Which is good as it makes it very flexible and we keep the logic together with the actual data. Note that the removal strategy is not limited to removing the object from the cache - it might also perform other actions if needed.

The code

Before we take a look at the code, let's check the dependency graph:
Let 's start with the CacheEntry class. This is the object we will store in the cache. It contains the value (actual object you want to cache) and the PublishRemoveStrategy. It also contains the function to invoke the removal strategy.
public class CacheEntry : ICacheEntry
{
  public CacheEntry(object valueToCache)
  {
    CachedValue = valueToCache;
  }

  public CacheEntry(object valueToCache, PublishRemoveStrategy publishRemoveStrategy)
  {
    CachedValue = valueToCache;
    PublishRemoveStrategy = publishRemoveStrategy;
  }

  public object CachedValue { get; }

  public PublishRemoveStrategy PublishRemoveStrategy { get; }

  /// <summary>
  /// Runs the on-publish remove strategy for the entry.
  /// The strategy (if provided) is expected to do the necessary work such as actually removing the entry from the cache.
  /// </summary>
  /// <param name="cache">Cache instance</param>
  /// <param name="key">Cache entry key</param>
  /// <param name="language">Publish language</param>
  /// <param name="database">Target Database</param>
  /// <param name="rootNode">Root node of the publish action</param>
  /// <param name="publishMode">Publish mode</param>
  public virtual void RunOnPublishRemoveStrategy(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    PublishRemoveStrategy?.Invoke(cache, key, language, database, rootNode, publishMode);
  }
}
The PublishRemoveStrategy strategy itself is a delegate that requires the language, the target database, the root node and the mode (of the publish):
public delegate void PublishRemoveStrategy(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);

The core of the module is the actual SitecoreCache class:
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class SitecoreCache : ISitecoreCache
{
  private const string LanguageIndependentKey = "xx";

  public virtual void Add(string key, ICacheEntry cacheEntry, string language)
  {
    HttpRuntime.Cache.Insert(BuildKey(key, language), cacheEntry);
  }

  public virtual void Add(string key, ICacheEntry cacheEntry)
  {
    Add(key, cacheEntry, null);
  }

  public virtual object GetObject(string key, string language)
  {
    var cacheObject = HttpRuntime.Cache[BuildKey(key, language)] as ICacheEntry;
    return cacheObject?.CachedValue;
  }

  public virtual object GetObject(string key)
  {
    return GetObject(key, null);
  }

  public virtual T GetObject<T>(string key, string language) where T : class
  {
    return GetObject(key, language) as T;
  }

  public virtual T GetObject<T>(string key) where T : class
  {
    return GetObject<T>(key, null);
  }

  public virtual void Remove(string key, string language)
  {
    HttpRuntime.Cache.Remove(BuildKey(key, language));
  }

  public virtual void Remove(string key)
  {
    Remove(key, null);
  }

  public virtual void RemoveItemsOnPublish(Database database, Item rootNode, PublishMode publishMode, Language language)
  {
    if (language == null)
    {
    return;
    }

    var prefixLanguage = BuildKey(string.Empty, language.Name);
    var entriesInLanguage = from DictionaryEntry entry in HttpRuntime.Cache
          let key = entry.Key as string
          where IsValidKey(key, prefixLanguage)
          select entry;

    foreach (var entry in entriesInLanguage)
    {
    if (!(entry.Value is ICacheEntry cacheObject))
    {
      continue;
    }

    var key = ExtractKey(entry.Key.ToString());
    cacheObject.RunOnPublishRemoveStrategy(this, key, language, database, rootNode, publishMode);
    }
  }

  public virtual void Clear()
  {
    var keys = (from DictionaryEntry entry in HttpRuntime.Cache where entry.Value is ICacheEntry select entry.Key).ToList();

    foreach (string key in keys)
    {
      Remove(key);
    }
  }

  private static string BuildKey(string key, string language)
  {
    var prefix = language?.ToLowerInvariant() ?? LanguageIndependentKey;
    return prefix + "|" + key;
  }

  private static string ExtractKey(string compositeKey)
  {
    return compositeKey.Substring(compositeKey.IndexOf('|') + 1);
  }

  private static bool IsValidKey(string key, string prefixLanguage)
  {
    return !string.IsNullOrEmpty(key)
    && (key.StartsWith(prefixLanguage, StringComparison.OrdinalIgnoreCase) || key.StartsWith(LanguageIndependentKey, StringComparison.OrdinalIgnoreCase));
  }
}

The SitecoreCache class has functions to:
  • Add entries: with or without language (some entries might be language dependent, others are not)
  • Get entries: again with or without the language, but also a (generic) version that will try to cast to a given type
  • Remove entries or Clear all entries
  • Handle the publish event: this gets all entries from the cache and calls their remove strategy
Entries are stored with a key (any string -should be unique- given by the using application that is combined with the language).

Clearing the cache

Clearing the cache is done in the CacheClearer class that we will attach to publish events: 
using Sitecore.Common;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Events;
using Sitecore.Events;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class CacheClearer
{
  private readonly ISitecoreCache sitecoreCache;

  public CacheClearer(ISitecoreCache sitecoreCache)
  {
    this.sitecoreCache = sitecoreCache;
  }

  public event EventHandler SitecoreCacheClearedHandler;

  public virtual void ClearCache(object sender, EventArgs args)
  {
    var publishingOptions = FetchPublishingOptions(args).ToList();

    if (!publishingOptions.Any())
    {
      return;
    }

    if (sitecoreCache == null)
    {
      return;
    }

    RemoveFromCache(publishingOptions);
  }

  protected virtual void AfterSitecoreCacheCleared(EventArgs e)
  {
    if (SitecoreCacheClearedHandler == null)
    {
      return;
    }

    SitecoreCacheClearedHandler(this, e);
  }

  protected virtual IEnumerable<DistributedPublishOptions> FetchPublishingOptions(EventArgs args)
  {
    var sitecoreRemoteArgs = args as PublishCompletedRemoteEventArgs;
    if (sitecoreRemoteArgs != null)
    {
      if (sitecoreRemoteArgs.PublishOptions != null)
      {
        return sitecoreRemoteArgs.PublishOptions;
      }

      return new List<DistributedPublishOptions>();
    }

    var sitecoreArgs = args as SitecoreEventArgs;
    if (sitecoreArgs == null)
    {
      return new List<DistributedPublishOptions>();
    }

    var publishingParameter = sitecoreArgs.Parameters[0] as IEnumerable<DistributedPublishOptions>;

    if (publishingParameter != null)
    {
      return publishingParameter;
    }

    return new List<DistributedPublishOptions>();
  }

  protected virtual void RemoveFromCache(IEnumerable<DistributedPublishOptions> publishingOptions)
  {
    foreach (var publishingOption in publishingOptions)
    {
      var database = GetDatabase(publishingOption.TargetDatabaseName);
      if (database == null)
      {
        continue;
      }

      RemoveFromCache(database, publishingOption.RootItemId.ToID(), publishingOption.LanguageName, publishingOption.Mode);
    }

    AfterSitecoreCacheCleared(EventArgs.Empty);
  }

  protected virtual void RemoveFromCache(Database database, ID id, string languageName, PublishMode mode)
  {
    Language publishingLanguage;
    if (!Language.TryParse(languageName, out publishingLanguage))
    {
      return;
    }

    var rootItem = database.GetItem(id, publishingLanguage);
    sitecoreCache.RemoveItemsOnPublish(database, rootItem, mode, publishingLanguage);
  }

  protected virtual Database GetDatabase(string name)
  {
    if (!Factory.GetDatabaseNames().Contains(name, StringComparer.OrdinalIgnoreCase))
    {
      return null;
    }

    return Factory.GetDatabase(name);
  }
}
In this class we are actually not doing much more than fetching the publish options and getting the parameters to call the RemoveItemsOnPublish from our SitecoreCache: the target database, the root item, the language and the publish mode. When fetching the publish options, we are trying twice to support local and remote events. We could also split the functions but decided not to. The SitecoreCacheClearedHandler event can be used to inject extra code after the clear.

Attaching the clearance code to the events is done with configuration: 
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <events>
      <event name="publish:complete">
        <handler type="XX.Caching.CacheClearer, XX" method="ClearCache" resolve="true" />
      </event>
      <event name="publish:complete:remote">
        <handler type="XX.Caching.CacheClearer, XX" method="ClearCache" resolve="true" />
      </event>
    </events>
  </sitecore>
</configuration>
Note that the clear code is attached to the local and the remote publish event to support scaled architectures.


Cache removal strategies

The last part of the puzzle is the actual cache removal strategies. You can easily write your own one if needed and use that, but we provided some that cover most of our needs. Each removal strategy class must implement the ICacheRemoval interface which has two functions: one for clearing the cache with a language and one without: 
public interface ICacheRemovalStrategy where T : class
{
  void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);

  void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode);
}

Four examples of a removal strategy:
1. Always: this strategy will always clear the cache, no matter what the options are.
public class Always : ICacheRemovalStrategy<object>
{
  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache != null && language != null)
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache != null)
    {
      cache.Remove(key);
    }
  }
}

2. On root node: this will remove the entry if the root node of the publish is the item or one of its descendants of the given parent - the parent is injected through a method that fetches the item based on the database.
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class OnRootNode : ICacheRemovalStrategy<Item>
{
  public OnRootNode(Func<Database, Item> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<Database, Item> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    if (IsDescendant(database, rootNode))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    if (IsDescendant(database, rootNode))
    {
      cache.Remove(key);
    }
  }

  private bool IsDescendant(Database database, Item publishRootNode)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var parent = GetDependencies?.Invoke(database);
    
    if (parent == null)
    {
      return false;
    }

    return publishRootNode.ID.Guid.Equals(parent.ID.Guid) || publishRootNode.Paths.IsDescendantOf(parent);
  }
}

3. On templates: this will remove the entry if the id of the template of the root node of the publish is amongst the given template id's
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class OnTemplates : ICacheRemovalStrategy<IEnumerable<Guid>>
{
  public OnTemplates(Func<IEnumerable<Guid>> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<IEnumerable<Guid>> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    if (IsTemplateBased(rootNode))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    if (IsTemplateBased(rootNode))
    {
      cache.Remove(key);
    }
  }

  private bool IsTemplateBased(Item publishRootNode)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var templates = GetDependencies?.Invoke();
    
    if (templates == null)
    {
      return false;
    }

    return templates.ToList().Contains(publishRootNode.TemplateID.Guid);
  }
}

4. On root node & template: combination of the 2 above
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Publishing;

public class TemplatesRootNode
{
  public Item RootNode { get; set; }
  public IEnumerable<Guid> Templates { get; set; }
}

public class OnTemplatesRootNode : ICacheRemovalStrategy<TemplatesRootNode>
{
  public OnTemplatesRootNode(Func<TemplatesRootNode> dependency)
  {
    GetDependencies = dependency;
  }

  public Func<TemplatesRootNode> GetDependencies
  {
    get;
    set;
  }

  public virtual void RemoveInLanguage(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key) || language == null)
    {
      return;
    }

    var dependency = Dependencies();
    
    if (IsTemplateBased(rootNode, dependency) && IsDescendant(rootNode, dependency))
    {
      cache.Remove(key, language.Name);
    }
  }

  public virtual void RemoveLanguageIndependent(ISitecoreCache cache, string key, Language language, Database database, Item rootNode, PublishMode publishMode)
  {
    if (cache == null || string.IsNullOrEmpty(key))
    {
      return;
    }

    var dependency = Dependencies();
    
    if (IsTemplateBased(rootNode, dependency) && IsDescendant(rootNode, dependency))
    {
      cache.Remove(key);
    }
  }

  private TemplatesRootNode Dependencies()
  {
    return GetDependencies?.Invoke();
  }

  private static bool IsTemplateBased(Item publishRootNode, TemplatesRootNode dependency)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    if (dependency == null)
    {
      return false;
    }

    return dependency.Templates.ToList().Contains(publishRootNode.TemplateID.Guid);
  }

  private static bool IsDescendant(Item publishRootNode, TemplatesRootNode dependency)
  {
    if (publishRootNode == null)
    {
      return false;
    }

    var parent = dependency?.RootNode;

    if (parent == null)
    {
      return false;
    }

    return publishRootNode.ID.Guid.Equals(parent.ID.Guid) || publishRootNode.Paths.IsDescendantOf(parent);
  }
}

Usage

If you're still here reading after such a long post, you probably also want to see an example of how to use this:
var rootNode = .. // get the parent item
var removeStrategy = new OnTemplatesRootNode(() => GetRemovalStrategy(rootNode));
sitecoreCache.Add(CacheKey, new CacheEntry(myObject, removeStrategy.RemoveInLanguage), language);

private TemplatesRootNode GetRemovalStrategy(Item rootItem)
{
  return new TemplatesRootNode
  {
    RootNode = rootItem,
    Templates = ..  // get the id's of the templates
  };
}
In this example I am caching a language dependent object and am using the combination of templates and a root node. The calls to fetch the parent node and the ids of the templates are left blank as those are really dependent on your solution.

Conclusion

I hope this all made some sense (and I didn't forget anything). It has proven to be a rather powerful way of caching data and clearing the data based on relevant Sitecore publish events. It's not perfect yet -there are still some ideas to extend and improve- but it's a good start. So if you did get this far I hope you enjoyed it - any feedback is welcome.

One last remark: be aware that objects returned from the cache are references - if you change them you also change the cached value! If you don't want this, take a copy first.

Thanks to Özkan Sayın for reading and correcting the draft version.


Posted by at No comments:
Labels: ,

Thursday, April 25, 2019

Publishing target in Sitecore SXA

Publishing target

Sitecore has the notion of publish targets that you can use to publish to different databases. This can be several live databases if you need that, or in many cases this is used for a "preview" database in order to have your editors publish to this preview database and verify the results before actually publishing to the live environment. 
Creating a publishing target has been documented already, so I will be very brief on this. Sitecore describes on their documentation how you can use this in combination with workflows. You need to add an actual extra database (tip: copy your web database) to work with and configure this one. Note that a few things changed in Sitecore 9 - as noted here on SSE.  A good start is the blog post of Peter Procházka on how to add a publish target.

Preview

For the scope of this article, I am focusing on using a publish target as a preview target. So the idea is that editors can publish to this target and see what they have created before publishing to the live site. This has some benefits over using the preview mode in Sitecore as that might not behave exactly as it will on the live environment. 
Assume our new target is called "preview" and we have created the database and all configs as mentioned in the above posts.

Indexes

One thing I am missing in a lot of posts on the subject is an extra index. Your code might rely on indexes (it should) but if you don't have an index to match that preview you will get strange results. So, create an extra index called sitecore_preview_index. You can copy the configuration from your web_index as it should act the same. Just don't forget to change the database in the crawler. 

SXA

The last step in configuring a preview target is normally to point a site to the preview database. That will be our test site. Normally I just use the ContentManagement site for this. In standard Sitecore this can be easily done in the configuration but in SXA I had to use a different approach.

First of all though: don't forget SXA also has it's own index..  so create a sitecore_sxa_preview_index as well (copy the config from your sxa_web_index, not from the other preview index as the sxa indexes have some different types).

Using SXA sites

In SXA you have a site definition in Sitecore itself (/sitecore/content/.../Settings/Site Grouping). This means you cannot have different values for ContentManagement and ContentDelivery in the same item (in config files that was easy). But there are solutions...  let's start by creating a second site item next to the one you already have (duplicate it to save time). 

The new site item should have all the same properties as the original one, except:
  • set the database to "preview"  (if your preview database is in the config, it should be in the list) 
  • add the preview index in the index config 

Routing the sites

Now we need to make sure that the CM is using the new site and the CD keeps using the original one. First thing to do is set the hostnames. The new site should not pick the CD urls and the original one doesn't need the CM url anymore. In some cases, this can be sufficient - it depends on the urls you have. In some cases however you might have wildcards and such that make this a first good step but not enough to make the CM go to the preview site.

SXA Environment parameter

There is one more parameter we can use - the "environment". By default this will be set to "*" on your site item. Meaning the site definition is valid for all environments. We can use this to distinguish between CM and CD environments! The value for the environment is not the same one as used in the Sitecore 9 configuration (the "role") - it is a value from a setting that we can patch on CM:
<settings>
  <setting name="XA.Foundation.Multisite.Environment" value="PREVIEW" role:require="ContentManagement" />
</settings>

So lets alter the values on our site items:
On the new preview site item we set the value to PREVIEW (same name as used in the setting). I noticed that I still had issues when the original one was left on *, so I changed that to LIVE - which is the default value of the environment parameter.

And yeah.. it works.
Not sure if this is the only way or the best way to do this, so if anyone has alternative ways to have publish targets combined with SXA please let me know ;)

Posted by at No comments:
Labels: ,
Subscribe to: Posts (Atom)