SXA cache multilingual datasources

Sitecore SXA datasource caching

I bumped into an issue a while ago related to datasources and how they are handled in a Sitecore SXA environment. This is an overview of our setup:

• using SXA (Sitecore Experience Accellerator) for Sitecore 9.2
• a PageList component with a variant
• an Item Query that refers to custom code (a query starting with "code:" - more info here)
• custom code that return a list of items and is language dependent (meaning: the list of items is different per language)  

That last one is actually the trigger for my issue as I had used custom coded Item Queries before and they seemed to work fine. Even in a multilingual setup. But in this particular case the results per language were actually different items - not just the same items with translated content.

I noticed that there was some caching issue and in the end I contacted Sitecore Support and placed some information on Sitecore StackExchange: "Custom SXA PageList datasource is not working in multilingual environment".

ResolveRenderingDatasourceCache

With some research I found there was an issue with the ResolveRenderingDatasourceCache. That cache is not language dependent and so my results were wrong.

A first possible solution would be to disable that cache by patching the config setting XA.Foundation.LocalDatasources.ResolveRenderingDatasourceCache.Enabled to false. This works, but of course that turns off this cache completely.

I noticed that the resolveRenderingDatasource pipeline has 2 relevant functions:

• Sitecore.XA.Foundation.LocalDatasources.Pipelines.ResolveRenderingDatasource.GetFromCache
• Sitecore.XA.Foundation.LocalDatasources.Pipelines.ResolveRenderingDatasource.SetCache

This brings us to solution two, which would disable the cache only for datasources that start with "code:": we can override the process method in the SetCache class and add code like this:

if (args.DefaultDatasource.StartsWith("code")) { return; }

This works as well, we keep the cache for most of our datasources but it is still disabled for the coded ones. And so I thought of another final solution: let's check the generation of the cache key!

BuildCacheKey

Both methods (GetFromCache and SetCache) are using a BuildCacheKey method from an underlying ResolveRenderingDatasourceCacheBase class. This method seemed not to be used anywhere else so I decided to override that one and use it in both the GetFromCache and SetCache classes.

protected override string BuildCacheKey(ResolveRenderingDatasourceArgs args)
{
    var key = base.BuildCacheKey(args);
    if (!string.IsNullOrEmpty(key) && args.DefaultDatasource.StartsWith("code:", StringComparison.OrdinalIgnoreCase))
    {
        key += Sitecore.Context.Language.Name;
    }

    return key;
}

This solution makes the cache work in all situations. We keep the current implementation for most datasources (as those are not language dependent) and add the language to the key for those that might (the ones starting with "code:" and you might also add those starting with "query:" if needed).

 

Patch the processors as:

<sitecore>
  <pipelines>
    <resolveRenderingDatasource>
      <processor type="Sitecore.XA.Foundation.LocalDatasources.Pipelines.ResolveRenderingDatasource.GetFromCache, Sitecore.XA.Foundation.LocalDatasources" resolve="true">
        <patch:attribute name="type">Feature.Caching.Rendering.Datasources.GetFromCache, Feature.Caching</patch:attribute>
      </processor>
      <processor type="Sitecore.XA.Foundation.LocalDatasources.Pipelines.ResolveRenderingDatasource.SetCache, Sitecore.XA.Foundation.LocalDatasources" resolve="true">
        <patch:attribute name="type">Feature.Caching.Rendering.Datasources.SetCache, Feature.Caching</patch:attribute>
      </processor>
    </resolveRenderingDatasource>
  </pipelines>
</sitecore>

A change request has been made to adjust this in a future version of SXA - but in the meantime you can easily patch it yourself if needed. Keep the cache and enable different results in different languages!

 

Kid Puzzle GIF from Kid GIFs

Local datasources on standard values with branch templates

Local datasources module

The local datasource module on the Sitecore marketplace has a second version. The first version managed to create local datasource items when editing in the experience editor. In this second version we anticipated on renderings that are added by default on a template.

Branch templates with "standard values"

The idea was simple: 

1. create a branch template based on the page template
2. edit the created item (in the branch template - probably called $name) in the experience editor
3. add a rendering with a local datasource (the local datasource module will create the children - a data folder and the datasource item)
4. repeat 4 if wanted (add more renderings)
5. use the branch template when creating items

Sounds simple, but of course when creating the items the datasource value of the rendering is still set to the item in the branch templates section - not to the child item we just created.

To fix that we added some code that I'll explain here. Code can be found on GitHub.

Event

I used the item:added event and not the item:created event. We need to be able to detect whether the item was created from a branch and can use the BranchId property of the Item for that. The item:created event however comes too soon to do that as the BranchId is not yet set at that moment. In the item:added event, we can check the BranchId. 

So we create a handler to add to item:added and patch it as first handler: patch:before="*[1]".

OnItemAdded

Our main function starts with some checks to get out as fast as possible when it is not applicable. When it is, we call our action method for the current item and all descendants. 

In case of a branch, the item:added event is called only once for the main item. 

But as this is a branch, we might also have a child with local datasources set. So we check all children - they are already created at this time.

Correct the datasources

For each item with a layout (item.Visualization.Layout != null) we grab the renderings and change the datasource value if it should be local.

var layout = LayoutDefinition.Parse(item[Sitecore.FieldIDs.LayoutField]);
var devices = layout.Devices;
var changed = devices.Cast<DeviceDefinition>().Sum(device => SetDatasourcesToLocal(device, item));
if (changed > 0)
{
    UpdateLayout(item, layout, Sitecore.FieldIDs.LayoutField);
}

We use the Parse method to get a LayoutDefinition of the "Shared layout" field. This does mean indeed that we only cover renderings placed on the shared layout. Doing the same for a rendering on the final layout is for a future version...

We loop through the devices and call a function to change the datasource values. This function loops over all renderings and checks the datasource item. If the datasource items path starts with /sitecore/template we know this was meant to be a local one. To find the correct local item, we explicitly go into the local data folder because a search over all descendants might also find a similar datasource item on another page in the branch. Once found, we set the value to the rendering in the layout.

Saving the changes

We keep track of the changes made because they are not automatically saved back into the item. The "layout" object has all the changes, but the item does not. So if we had any changes, we save the new value back into the shared layout field:

item.Editing.BeginEdit();
item.Fields[Sitecore.FieldIDs.LayoutField].Value = layout.ToXml();
item.Editing.EndEdit(false, false);

Conclusion

Version 2 is now published on the marketplace, and to be honest version 3 will not be for any time soon. But if you like the module, I'm always open for comments, ideas and contributions...

Enjoy the local datasources!

Local Datasources module for Sitecore Experience Editor

Local Datasources marketplace module

On the Sitecore marketplace a new module was recently added to help editors with the concept of "local" datasources. The module can be found here. 

The source code is available on GitHub. Instruction on installing and using the module are included in the readme file.

We assume people working with Sitecore and reading this know what datasources are. Some of you might even work with a "local" datasources. Local meaning that the datasource item is coupled with the "page" item where it is used in a hierarchical way by storing the datasource item underneath the page item - in a "local" data folder.

We tried to automate this proces for datasources that should not be shared amongst "pages" while working in the Experience Editor.

The solution now consists of 2 parts.

• The first part will create the actual datasource item (and the data folder if that does not yet exists - datafolder will be pushed as latest child). The template name of the required datasource is used as base for the item name, combined with a number.
• A second part will prevent the "Select the associated content" dialog from appearing and is discussed already in a previous post: Preventing the Sitecore "select datasource" dialog.

I wrote another blog about the functionalities of the module - here I will go deeper into the code.

Creating the datasource

Creating the datasource item is done in a processor in the getRenderingDatasource pipeline. This pipeline is started within the AddRendering command (see step 2 - the dialog). 

We check the datasource location of the rendering and determine whether it is "local". If so we start the item creation:

1. Create the parent item for the datasource item if it does not exist 
• based on a provided template
• underneath the context item, and pushed down to the last child position
• name is based on the datasource location (we should add some more data verification here)
2. Add this parent item to the datasource roots
3. Create the datasource item underneath the parent
• based on the template set on the rendering as datasource template
• using this template name of the datasource as base for the item name (combined with a number)
4. Set the datasource item FullPath as CurrentDatasource in the pipeline arguments

A few things to notice here:

• We use a SecurityDisabler to make sure the item can be created.
• We use a SiteContextSwitcher (to "system") to make sure the page does not refresh (if you don't do this, the page detects a new item and refreshes - causing errors)

The datasource dialog

You can read here how we prevented the dialog from appearing and still setting the datasource value when the arguments were set correctly in the previous step using a custom AddRendering command.

As we do override the AddRendering command, the module needs a Sitecore version with the exact same code as the one we used in that command to keep all renderings without local datasources to work as expected.

The Sitecore Experience Accelerator (SXA) has it's own version of the AddRendering command - for the same reason btw. 

This means that this module should not be installed on instances were SXA is also present!

A future enhancement could be to add SXA support to the module, although it might be a better option to try to use SXA functionality in your non-SXA site - but that is a whole other story...

Future enhancements

We released this module as version 1.0.  It is working on Sitecore 8.2 and has some functionalities already but we are aware that there are still quite some areas of improvement. As mentioned here before, we might need to add some extra data checks to be more robust. But also on a functional level we still have ideas:

• a solution for renderings that are already set on the standard values of a template (using branches maybe?)
• remove datasource items if the rendering is removed (and no other links are found)
• content editor support? (although we're not quite sure we actually want this)

Feel free to use the module, share more ideas for enhancements, contribute to the code...

Preventing the "Select Datasource" dialog in the Sitecore Experience Editor

Select the Associated Content dialog

Every editor working in the Sitecore Experience Editor has undoubtedly seen this screen before:

The dailog lets you select a Datasource for a rendering. Or create a new one...  The dialog will appear if your rendering has a datasource location set on its definition item:

 

But now we had a case where we wanted this dialog not to appear. The datasource was set in a custom processor in the getRenderingDatasource pipeline so it was already available. The dialog knows about this value as it will select it by default for you but it still appears.

Rendering "Data source" field

A first attempt was successful by accident. Wouldn't be the first achievement made like this... If a datasource is set in the "Data source" field of the rendering definition item, the dialog will not appear. Due to a flaw in the code setting the datasource the value was written there instead of the actual item containing the rendering. Works like a charm! But not thread safe. You only need two editors to place that rendering at the same time and your datasources will get mixed up. 

The AddRendering command

With a little help we found the right command to override (thanks to Support and Alan Płócieniak): webedit:addrendering.

It is quite obvious the class is not meant to be overridden, so you might need a debugging tool to get it all right. Let's start by creating a class that overrides the Sitecore.Shell.Applications.WebEdit.Commands.AddRendering. The function we need is "void Run(ClientPipelineArgs args)".  Best start is to copy the code from the original class and get rid of the function you do not need. As you will notice (I am writing based on Sitecore 8.2 initial btw) you do need to keep a few private functions in your own code as well to keep the "Run" function working.

In the Run-code, locate the call to the getRenderingDatasource pipeline:

CorePipeline.Run("getRenderingDatasource", renderingDatasourceArgs);

Just below you'll see

if (!string.IsNullOrEmpty(renderingDatasourceArgs.DialogUrl) && !AddRendering.IsMorphRenderingsRequest(args))

  {
    ...
    args.Parameters["OpenProperties"] = flag.ToString().ToLowerInvariant();
    SheerResponse.ShowModalDialog(renderingDatasourceArgs.DialogUrl, "1200px", "700px", string.Empty, true);
    args.WaitForPostBack();
  }

This is the spot where the dialog would open. As we don't want this, we add an if-statement within the if-statement from above:

if (!string.IsNullOrEmpty(renderingDatasourceArgs.DialogUrl) && !IsMorphRenderingsRequest(args))
  {
    if (!string.IsNullOrEmpty(renderingDatasourceArgs.CurrentDatasource))
    {
      var datasourceItem = Client.ContentDatabase.GetItem(renderingDatasourceArgs.CurrentDatasource);
      WebEditResponse.Eval(FormattableString.Invariant(
           $"Sitecore.PageModes.ChromeManager.handleMessage('chrome:placeholder:controladded', 
              {{ id: '{itemNotNull.ID.Guid.ToString("N").ToUpperInvariant()}', 
                 openProperties: {flag.ToString().ToLowerInvariant()}, 
                 dataSource: '{datasourceItem.ID.Guid.ToString("B").ToUpperInvariant()}' }});"));
    }
  ...
  }

What are we doing here? First we check if the datasource was set in the arguments before. If so, we want our code and no dialog anymore. We fetch the datasource item from the content database. And then the magic: we send the message to Sitecore editor that the control was added, just the way it does when a rendering was completely set (see the code a few lines below in the original AddRendering). The trick is to make sure all parameters are set correctly:

• the ID of the rendering that was added as digits ("N")
• a value indicating whether the properties window should be opened - this is true/false and is send without any quote
• the ID of the datasource item as guid with braces ("B")

If any of the parameters are faulty, you will get an error on the call to "Palette.aspx" and the rendering will not be added (you will also see an error popup in the browser).

As we send the value for the properties window as for all other renderings, we can keep that functionality as is. It will react on the value set when the rendering was selected (which can be tweaked on the rendering definition item itself if you don't want to leave that choice to your editors).

Configuration

You need to tell Sitecore you have a new AddRendering command. Patch the config by creating a new config file in the include folder (make sure it gets added in the end):

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="webedit:addrendering" type="YourNamespace.YourClass, YourSolution" patch:instead="*[@name='webedit:addrendering']" />
    </commands>
  </sitecore>
</configuration>

Wrap it up

This code will probably (as it was for me) be a part of a bigger solution and is not a fully working code set. If you have any questions, you can find me on Sitecore Slack or sitecore.stackexchange.com ;)

One question that apparently does pop up is how to set the datasource. Setting the datasource when a rendering is added can be done in the getRenderingDatasource pipeline, which is called in the command as we saw. Create an extra processor and add the datasource item to the arguments:

getRenderingDatasourceArgs.CurrentDatasource = datasourceItem.Paths.FullPath;

One thing to remember though: as we did overwrite some piece of Sitecore code here, you do need to check on every upgrade that all is still valid and you might need to adapt the code. 

Multiple datasources for Associated Content

Associated Content

In the definition of a sublayout in Sitecore you can select a Datasource location and a Datasource template. This makes it easier for your editors to pick the right datasource when using this sublayout. But what if you want to allow more than one datasource template? The answer was found somewhere well hidden on the Sitecore developers site:

Please refer to the following Release note, which describes the officially recommended way to implement such behavior: http://sdn.sitecore.net/Products/Sitecore%20V5/Sitecore%20CMS%206/ReleaseNotes/ChangeLog.aspx
The "Select Associated Content" dialog has been made more extensible. (339396)
Developers can now specify multiple allowed templates by hooking into the <getRenderingDatasource> pipeline and populating the new TemplatesForSelection property.
The default behavior is to use value of the "Datasource Template" field on the rendering item.

How to use multiple datasource templates?

So, all you need to do is create a config file and place it in the include folder (do not alter the web.config):

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
 <sitecore>
   <!-- PIPELINES -->
   <pipelines>
     <getRenderingDatasource>
      <processor patch:after="*[@type='Sitecore.Pipelines.GetRenderingDatasource.GetDatasourceTemplate, 
              Sitecore.Kernel']" type="Demo.SitecorePipelines.GetMultipleDataSourceTemplates, Demo"/>
     </getRenderingDatasource>
   </pipelines>
 </sitecore>
</configuration>

and create your class:

public class GetMultipleDataSourceTemplates
{
    public void Process(GetRenderingDatasourceArgs args)
    {
        Assert.ArgumentNotNull(args, "args");
        if (args.Prototype != null)
        {
            return;
        }

        var data = args.RenderingItem["Datasource Template"];
        if (string.IsNullOrEmpty(data))
        {
            return;
        }

        var templates = data.Split('|');
        foreach (var name in templates)
        {
            var item = args.ContentDatabase.GetItem(name);
            if (item == null)
            {
                continue;
            }
            var template = TemplateManager.GetTemplate(item.ID, args.ContentDatabase);
            if (template != null)
            {
                args.TemplatesForSelection.Add(template);
            }
        }
    }
}

Note that we use "|" as separator here. That is just as an example.
Ok, that's it. Now we can use it like this in Sitecore:

When saving your sublayout item, Sitecore will complain because the field in the datasource template cannot be linked to an item. Don't worry about this. Your editors won't notice and it works.

Template updates

One drawback is that Sitecore is not able to link the template to the field anymore. This means that if you move or rename the template the field will not be updated (as a normal datasource template field will).