Thursday, March 9, 2017

Variants on your custom Sitecore SXA renderings

Sitecore Hackathon 2017

One of the possible ideas of the 2017 Sitecore Hackathon was the Sitecore Experience Accelerator (SXA). With our "No Weekend 4 Us" team we took this challenge and created a custom component that works with SXA Variants (and did other fancy stuff, but that will be explained in another blog post). [Using SXA 1.2 on Sitecore 8.2-upd2]

Sitecore Experience Accelerator Variants

SXA comes with a set of default renderings and rendering variants. Rendering variants are configurable adaptations of the default renderings. To further encourage reusability, you can also create new rendering variants. This gives authors more options in the way they present their content.
Info on creating variants can be found on the SXA dev documentation, but what if you create a custom component for SXA and want to use this technology as well. I'll try to provide a step by step overview of the things to do. Our component was about blogs, so you might find that in the code examples...

Variant model repository

We created a repository based on the VariantsRepository of SXA:
using Sitecore.XA.Foundation.RenderingVariants.Repositories;

public class BlogVariantRepository : VariantsRepository, IBlogVariantRepository
{
  public T GetModel<T>(object model) where T : IRenderingModelBase
  {
    FillBaseProperties(model);
    return (T)model;
  }
}
The reason to do this is to get a variants-aware model later on. The repository has an interface, so we will inject it with dependency injection. We will do this the SXA way, by using their <ioc> pipeline.

Dependency Injection

<pipelines>
  <ioc>
    <processor type="X.Features.Blog.Pipelines.IoC.RegisterBlogServices, X.Features.Blog" />
  </ioc>
</pipelines>
public class RegisterBlogServices : IocProcessor
{
  public override void Process(IocArgs args)
  {
    args.ServiceCollection.AddTransient<IBlogVariantRepository, BlogVariantRepository>();
    ...
  }
}
We are using build-in DI of Sitecore 8.2 as used by SXA this way. It's actually a very easy way to insert your stuff to the container. As an extra there is no need to register your controller - SXA will take care of that for you.

Model - View - Controller


Model

Let's start with the model. 
public class BlogsModel : VariantsRenderingModel
{
    ..  // custom properties
}
The model inherits from the VariantsRenderingModel. This base class provides some interesting (and needed) properties. It actually is the RenderingModelBase (with info on Item, Rendering ...) extended with VariantFields. We will use the model repository we created earlier to fill the properties of this base class. Of course, it can be extended with any properties and function.

Controller

public class BlogsController : VariantsController
{
  private readonly IBlogVariantRepository blogVariantRepository;
  ...

  public BlogsController(IBlogVariantRepository blogVariantRepository, ...)
  {
    this.blogVariantRepository = blogVariantRepository;
    ...
  }

  public ActionResult BlogList()
  {
    var model = blogVariantRepository.GetModel<BlogsModel>(new BlogsModel());
    model.X = ...;
    return View(model);
   }
}
The controller inherits from VariantsController. Our repository is injected as explained earlier. We call the GetModel on our repository to get our base model with filled base properties. We can set values for other custom properties on the retrieved model object.

View

<div @Html.Sxa().Component("bloglist", Model.Attributes)>
  <div class="component-content">
    <div class="blog-list">
      @foreach (var item in Model.Blogs)
      {
        <div data-itemid="@item.Blog.ID.Guid">
        @foreach (var variantField in Model.VariantFields)
        {
          @Html.RenderingVariants().RenderVariant(variantField, item.Blog, false)
        }
        </div>
      }
    </div>
  </div>
</div>

This is just an example of what might be possible in the view - a bit more bound to the model we used as a test (with a list of blogs and each blog object included the underlying Sitecore item).

You can see here that we loop over all variant fields and simply display them. This means of course that we need a default variant to show anything.

That's it for the code.. now lets move on to Sitecore.

Sitecore (SXA) configuration

Rendering parameters template

First step is to create a rendering parameters template to use for your rendering. The template should inherit from /sitecore/templates/Foundation/Experience Accelerator/Rendering Variants/Rendering Parameters/IComponentVariant (don't forget to also inherit /sitecore/templates/System/Layout/Rendering Parameters/Standard Rendering Parameters instead of the Standard Template).

Using this base template will give you the variants dropdown in the rendering properties. By the way, SXA has a similar base template for Styling (/sitecore/templates/Foundation/Experience Accelerator/Presentation/Rendering Parameters/IStyling).








Rendering

In /sitecore/layout/Renderings/Feature/Experience Accelerator/[Feature-Name]/[Rendering-Name] create a Controller Rendering and fill in the usual fields (Controller, Action ..). And a tip to make it complete: give your rendering an icon ;)

To make the rendering available in the toolbox, go to the Presentation part of the SXA website (will be on the same level as the Home page). Under Available Renderings create a new Available Renderings item (might want to name if like your feature again). Add your newly created rendering in the "Renderings" box. Now we have a rendering in the toolbox - lets continue by creating the variants we need.

Variants

Variants are also defined in the "Presentation" section of your site. Under Rendering Variants create a new Item of type Variants and name it exactly as your rendering (important!). Under this Variants, create a Variant Definition. You might want to call the first one "default". Under the Variant Definition, create an Item of type Field. Choose a tag (e.g. "div"), fill in the Field name and set the other fields as desired (e.g. check "Is link" if you want a link to be generated on the field). Continue creating "Fields" as much as you want. And that is it. When you add the rendering from the Toolbox, you will see your variant in the dropdown and all the fields you created will be rendered. 

To make it really useful, you might want to create more Variant Definitions with different sets of Fields. This way you can easily create different outputs of the same rendering (and even more if you combine this with Styling).

More information on what you can do with the variants and their properties in the link mentioned above on the Sitecore doc site.

ps: thx to Dawid Rutkowski for pointing us in the right direction!

Wednesday, March 1, 2017

Local datasources on standard values with branch templates

Local datasources module

The local datasource module on the Sitecore marketplace has a second version. The first version managed to create local datasource items when editing in the experience editor. In this second version we anticipated on renderings that are added by default on a template.

Branch templates with "standard values"

The idea was simple: 
  1. create a branch template based on the page template
  2. edit the created item (in the branch template - probably called $name) in the experience editor
  3. add a rendering with a local datasource (the local datasource module will create the children - a data folder and the datasource item)
  4. repeat 4 if wanted (add more renderings)
  5. use the branch template when creating items
Sounds simple, but of course when creating the items the datasource value of the rendering is still set to the item in the branch templates section - not to the child item we just created.

To fix that we added some code that I'll explain here. Code can be found on GitHub.

Event

I used the item:added event and not the item:created event. We need to be able to detect whether the item was created from a branch and can use the BranchId property of the Item for that. The item:created event however comes too soon to do that as the BranchId is not yet set at that moment. In the item:added event, we can check the BranchId. 

So we create a handler to add to item:added and patch it as first handler: patch:before="*[1]".

OnItemAdded

Our main function starts with some checks to get out as fast as possible when it is not applicable. When it is, we call our action method for the current item and all descendants. 
In case of a branch, the item:added event is called only once for the main item. 
But as this is a branch, we might also have a child with local datasources set. So we check all children - they are already created at this time.

Correct the datasources

For each item with a layout (item.Visualization.Layout != null) we grab the renderings and change the datasource value if it should be local.
var layout = LayoutDefinition.Parse(item[Sitecore.FieldIDs.LayoutField]);
var devices = layout.Devices;
var changed = devices.Cast<DeviceDefinition>().Sum(device => SetDatasourcesToLocal(device, item));
if (changed > 0)
{
    UpdateLayout(item, layout, Sitecore.FieldIDs.LayoutField);
}
We use the Parse method to get a LayoutDefinition of the "Shared layout" field. This does mean indeed that we only cover renderings placed on the shared layout. Doing the same for a rendering on the final layout is for a future version...

We loop through the devices and call a function to change the datasource values. This function loops over all renderings and checks the datasource item. If the datasource items path starts with /sitecore/template we know this was meant to be a local one. To find the correct local item, we explicitly go into the local data folder because a search over all descendants might also find a similar datasource item on another page in the branch. Once found, we set the value to the rendering in the layout.

Saving the changes

We keep track of the changes made because they are not automatically saved back into the item. The "layout" object has all the changes, but the item does not. So if we had any changes, we save the new value back into the shared layout field:
item.Editing.BeginEdit();
item.Fields[Sitecore.FieldIDs.LayoutField].Value = layout.ToXml();
item.Editing.EndEdit(false, false);

Conclusion

Version 2 is now published on the marketplace, and to be honest version 3 will not be for any time soon. But if you like the module, I'm always open for comments, ideas and contributions...

Enjoy the local datasources!

Tuesday, February 14, 2017

Local Datasources module for Sitecore Experience Editor

Local Datasources marketplace module

On the Sitecore marketplace a new module was recently added to help editors with the concept of "local" datasources. The module can be found here

The source code is available on GitHub. Instruction on installing and using the module are included in the readme file.

We assume people working with Sitecore and reading this know what datasources are. Some of you might even work with a "local" datasources. Local meaning that the datasource item is coupled with the "page" item where it is used in a hierarchical way by storing the datasource item underneath the page item - in a "local" data folder.

We tried to automate this proces for datasources that should not be shared amongst "pages" while working in the Experience Editor.

The solution now consists of 2 parts.
  • The first part will create the actual datasource item (and the data folder if that does not yet exists - datafolder will be pushed as latest child). The template name of the required datasource is used as base for the item name, combined with a number.
  • A second part will prevent the "Select the associated content" dialog from appearing and is discussed already in a previous post: Preventing the Sitecore "select datasource" dialog.

I wrote another blog about the functionalities of the module - here I will go deeper into the code.

Creating the datasource

Creating the datasource item is done in a processor in the getRenderingDatasource pipeline. This pipeline is started within the AddRendering command (see step 2 - the dialog). 

We check the datasource location of the rendering and determine whether it is "local". If so we start the item creation:
  1. Create the parent item for the datasource item if it does not exist 
    • based on a provided template
    • underneath the context item, and pushed down to the last child position
    • name is based on the datasource location (we should add some more data verification here)
  2. Add this parent item to the datasource roots
  3. Create the datasource item underneath the parent
    • based on the template set on the rendering as datasource template
    • using this template name of the datasource as base for the item name (combined with a number)
  4. Set the datasource item FullPath as CurrentDatasource in the pipeline arguments

A few things to notice here:
  • We use a SecurityDisabler to make sure the item can be created.
  • We use a SiteContextSwitcher (to "system") to make sure the page does not refresh (if you don't do this, the page detects a new item and refreshes - causing errors)

The datasource dialog

You can read here how we prevented the dialog from appearing and still setting the datasource value when the arguments were set correctly in the previous step using a custom AddRendering command.

As we do override the AddRendering command, the module needs a Sitecore version with the exact same code as the one we used in that command to keep all renderings without local datasources to work as expected.

The Sitecore Experience Accelerator (SXA) has it's own version of the AddRendering command - for the same reason btw. 
This means that this module should not be installed on instances were SXA is also present!
A future enhancement could be to add SXA support to the module, although it might be a better option to try to use SXA functionality in your non-SXA site - but that is a whole other story...

Future enhancements

We released this module as version 1.0.  It is working on Sitecore 8.2 and has some functionalities already but we are aware that there are still quite some areas of improvement. As mentioned here before, we might need to add some extra data checks to be more robust. But also on a functional level we still have ideas:
  • a solution for renderings that are already set on the standard values of a template (using branches maybe?)
  • remove datasource items if the rendering is removed (and no other links are found)
  • content editor support? (although we're not quite sure we actually want this)
Feel free to use the module, share more ideas for enhancements, contribute to the code...

Saturday, February 4, 2017

Preventing the "Select Datasource" dialog in the Sitecore Experience Editor

Select the Associated Content dialog

Every editor working in the Sitecore Experience Editor has undoubtedly seen this screen before:

The dailog lets you select a Datasource for a rendering. Or create a new one...  The dialog will appear if your rendering has a datasource location set on its definition item:

 

But now we had a case where we wanted this dialog not to appear. The datasource was set in a custom processor in the getRenderingDatasource pipeline so it was already available. The dialog knows about this value as it will select it by default for you but it still appears.

Rendering "Data source" field

A first attempt was successful by accident. Wouldn't be the first achievement made like this... If a datasource is set in the "Data source" field of the rendering definition item, the dialog will not appear. Due to a flaw in the code setting the datasource the value was written there instead of the actual item containing the rendering. Works like a charm! But not thread safe. You only need two editors to place that rendering at the same time and your datasources will get mixed up. 

The AddRendering command

With a little help we found the right command to override (thanks to Support and Alan Płócieniak): webedit:addrendering.

It is quite obvious the class is not meant to be overridden, so you might need a debugging tool to get it all right. Let's start by creating a class that overrides the Sitecore.Shell.Applications.WebEdit.Commands.AddRendering. The function we need is "void Run(ClientPipelineArgs args)".  Best start is to copy the code from the original class and get rid of the function you do not need. As you will notice (I am writing based on Sitecore 8.2 initial btw) you do need to keep a few private functions in your own code as well to keep the "Run" function working.

In the Run-code, locate the call to the getRenderingDatasource pipeline:
CorePipeline.Run("getRenderingDatasource", renderingDatasourceArgs);

Just below you'll see
if (!string.IsNullOrEmpty(renderingDatasourceArgs.DialogUrl) && !AddRendering.IsMorphRenderingsRequest(args))

  {
    ...
    args.Parameters["OpenProperties"] = flag.ToString().ToLowerInvariant();
    SheerResponse.ShowModalDialog(renderingDatasourceArgs.DialogUrl, "1200px", "700px", string.Empty, true);
    args.WaitForPostBack();
  }

This is the spot where the dialog would open. As we don't want this, we add an if-statement within the if-statement from above:

if (!string.IsNullOrEmpty(renderingDatasourceArgs.DialogUrl) && !IsMorphRenderingsRequest(args))
  {
    if (!string.IsNullOrEmpty(renderingDatasourceArgs.CurrentDatasource))
    {
      var datasourceItem = Client.ContentDatabase.GetItem(renderingDatasourceArgs.CurrentDatasource);
      WebEditResponse.Eval(FormattableString.Invariant(
           $"Sitecore.PageModes.ChromeManager.handleMessage('chrome:placeholder:controladded', 
              {{ id: '{itemNotNull.ID.Guid.ToString("N").ToUpperInvariant()}', 
                 openProperties: {flag.ToString().ToLowerInvariant()}, 
                 dataSource: '{datasourceItem.ID.Guid.ToString("B").ToUpperInvariant()}' }});"));
    }
  ...
  }

What are we doing here? First we check if the datasource was set in the arguments before. If so, we want our code and no dialog anymore. We fetch the datasource item from the content database. And then the magic: we send the message to Sitecore editor that the control was added, just the way it does when a rendering was completely set (see the code a few lines below in the original AddRendering). The trick is to make sure all parameters are set correctly:

  • the ID of the rendering that was added as digits ("N")
  • a value indicating whether the properties window should be opened - this is true/false and is send without any quote
  • the ID of the datasource item as guid with braces ("B")
If any of the parameters are faulty, you will get an error on the call to "Palette.aspx" and the rendering will not be added (you will also see an error popup in the browser).

As we send the value for the properties window as for all other renderings, we can keep that functionality as is. It will react on the value set when the rendering was selected (which can be tweaked on the rendering definition item itself if you don't want to leave that choice to your editors).

Configuration

You need to tell Sitecore you have a new AddRendering command. Patch the config by creating a new config file in the include folder (make sure it gets added in the end):
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="webedit:addrendering" type="YourNamespace.YourClass, YourSolution" patch:instead="*[@name='webedit:addrendering']" />
    </commands>
  </sitecore>
</configuration>

Wrap it up


This code will probably (as it was for me) be a part of a bigger solution and is not a fully working code set. If you have any questions, you can find me on Sitecore Slack or sitecore.stackexchange.com ;)

One question that apparently does pop up is how to set the datasource. Setting the datasource when a rendering is added can be done in the getRenderingDatasource pipeline, which is called in the command as we saw. Create an extra processor and add the datasource item to the arguments:
getRenderingDatasourceArgs.CurrentDatasource = datasourceItem.Paths.FullPath;

One thing to remember though: as we did overwrite some piece of Sitecore code here, you do need to check on every upgrade that all is still valid and you might need to adapt the code. 

Wednesday, November 16, 2016

Sitecore Experience Accelerator: SXA and Search

First encounters with SXA: Search components

I was investigating the possibilities of SXA, the new Sitecore Experience Accelerator, 1.1. Questions were put on the Sitecore Stack Exchange site - not only by myself of course, also by many others - and that made some things clear. One of the subject I was testing was the search functionality, using the standard components and not adding any code. As it was all on a small test setup, I did not go as far as testing performance and stability, the focus was more on functionality.

But here they are - my first encounters with SXA to share.

Setup a search box and results page

First thing to do was setting up a search result page. I created a page just like any other and added the "Search Results" component from the Toolbox.

Next step was to  have a search box to feed the search page. I placed this one on a partial design to include it on all pages.  As I am still using the built-in basic theme, the search box looks like this:


The labels can be altered. The search box has some properties that can be set: first one seems quite important is the search results page. So we select the page that we created before. We also set the maximum number of predictive results shown in the dropdown. Yes, that's right - the search box has predictive results:



We save, publish it all. And there is our search! As we had some content pages, we get results from the index in our search box and we get send to the search results page as expected. The request to the search results page has the search query in the url query parameters, so analytics tools can pick it up if needed.

Some extra's

Of course, a page with just search results is not very user friendly so we add some more components. Let's add the "Results Count" on top, a dropdown "Filter" as well and the "Page Selector" at the bottom. We will go for a fixed page size, and have set it's value in the Search Results component. We now have something like this:


We have set the page size to 2, which is not realistic but managed to show the pagers working without having to add too much demo content pages.

So now we have a search box, results page with counter and paging and no letter code. 

Facets

Our filter is still empty now. The pages in SXA have tagging by default and we did tag all of them with one or more tag values. So lets try to filter on those tags!

First we create a facet "Tag".
In the facet item we set the name and display name and the field name. This is the lower case name of the field that is used in the index and that the facet is based on, in our case it's "sxatags". If you don't know the exact name of the fields you can always find this by looking at the templates used by the (SXA) items.

Once we have the facet item, we create a datasource for our Filter component and set the Facet field to the one we just created. Et voila, we publish and see:

facets with numbers in the dropdown filter box. By selecting one of them, the search results get filtered on the tag. 

Still no line of code...

And further... a Tag Cloud

We also added a Tag Cloud component - and noticed it works very well with our newly created search. 
As it is not styled, it looks like ****, but it works. I get my tags as used in the site and when I select my search results page as a parameter of the tag cloud components, clicking the tags gets me to my search results page with a search on the tag.


Customizing

You can alter the search results look by creating a rendering variant. The default will show the Title or display name of the items, but you can add fields to the default variant, or create a new one and select this custom one in your search results component. You need to create VariantField items to achieve this. 

Note that you can not alter the results look per template type (e.g. add an image to results of type 'Product') but you can add extra fields for it, and if the fields does not exist it will just be skipped.

Issues and/or questions - The conclusion

Questions will rise and problems as well. What is for me still unclear is the way the index behind the search engine works. The search uses one field, "SxaContent" - which is a computed field. But for now it's not very clear what is included in that field (next to the default title and content) and what is not - or how to get things included. 

Another mystery is the updates. As I did some research to using Lucene/Solr indexes for a full site search in the past, I am always curious to see how the updates are handled in index based search solutions. How does a "page" know that some content in a datasource is updated? 
From the tests I made, updating a shared datasource seems to work - all pages are updated. But I had some trouble when I added a new component with an existing datasource to a page - that did not get updated in the index.  That will require some more investigation.

Conclusion

The conclusion is that we have a working search without coding and in a very short time. Does it have all the features that "a customer" might want? No, maybe not. But do compared to the effort needed to put it on the site it's definitely worth considering! A promising first encounter with SXA...

Wednesday, October 5, 2016

Custom Sitecore index crawler

Why? - The case

We needed to find items in the master database quickly, based on some criteria. Language didn't matter and we didn't need the actual item, a few properties would do. So we decided to create a custom index with the fields we needed indexed (the criteria) and/or stored (the properties). All fine, but as we were using the master database we had different versions of each item in the index. As we needed only the latest version in only one language we though we could optimize the index to only contain those versions.

First attempt

We had to override the SitecoreItemCrawler. That was clear. The first attempt was creating a custom IsExcludedFromIndex function that would stop all entries not in English and not the latest version. Pretty simple, but does not work.
First of all, all entries were in English.. this function is only called per item and not per version. So actually, we could not use this. Furthermore, we did not take into account the fact that when adding a new version, we have to remove the previously indexed one.

Don't index multiple versions

I started searching the internet and found this post by Martin English on (not) indexing multiple versions. Great post, and pointing as well towards a solution with inbound filters. But as those filters work for every index, that would be no solution here. I needed it configurable per index. So back to Martins' post. We had to overwrite DoAdd and DoUpdate.

A custom index crawler

The result was a bit different as I was using Sitecore 8.1 and also wanted to include a language filter. I checked the code from the original SitecoreItemCrawler, created a class overwriting it and adapted where needed.

Language

I made the language configurable by putting it into a property:
private string indexLanguage;

public string Language
{
  get  {  return !string.IsNullOrEmpty(indexLanguage) ? indexLanguage : null; }
  set  {  indexLanguage = value; }
}

DoAdd

The DoAdd method was changed by adding an early check in the language-loop to get out when not the requested language. I also removed the version-loop with a request for the latest version so that only that version gets send to the index.
protected override void DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
{
  Assert.ArgumentNotNull(context, "context");
  Assert.ArgumentNotNull(indexable, "indexable");
  using (new LanguageFallbackItemSwitcher(context.Index.EnableItemLanguageFallback))
  {
    Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:adding", context.Index.Name, indexable.UniqueId, indexable.AbsolutePath);
    if (!IsExcludedFromIndex(indexable, false))
    {
      foreach (var language in indexable.Item.Languages)
      {
        // only include English
        if (!language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
        {
          continue;
        }

        Item item;
        using (new WriteCachesDisabler())
        {
          item = indexable.Item.Database.GetItem(indexable.Item.ID, language, Version.Latest);
        }

        if (item == null)
        {
          CrawlingLog.Log.Warn(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : AddItem : Could not build document data {0} - Latest version could not be found. Skipping.", indexable.Item.Uri));
        }
        else
        {
          SitecoreIndexableItem sitecoreIndexableItem;
          using (new WriteCachesDisabler())
          {
            // only latest version
            sitecoreIndexableItem = item.Versions.GetLatestVersion();
          }

          if (sitecoreIndexableItem != null)
          {
            IIndexableBuiltinFields indexableBuiltinFields = sitecoreIndexableItem;
            indexableBuiltinFields.IsLatestVersion = indexableBuiltinFields.Version == item.Version.Number;
            sitecoreIndexableItem.IndexFieldStorageValueFormatter = context.Index.Configuration.IndexFieldStorageValueFormatter;
            Operations.Add(sitecoreIndexableItem, context, index.Configuration);
          }
        }
      }
    }

    Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:added", context.Index.Name, indexable.UniqueId, indexable.AbsolutePath);
  }
}


DoUpdate

For the DoUpdate method I did something similar although I had to change a bit more here.
protected override void DoUpdate(IProviderUpdateContext context, SitecoreIndexableItem indexable, IndexEntryOperationContext operationContext)
{
  Assert.ArgumentNotNull(context, "context");
  Assert.ArgumentNotNull(indexable, "indexable");
  using (new LanguageFallbackItemSwitcher(Index.EnableItemLanguageFallback))
  {
    if (IndexUpdateNeedDelete(indexable))
    {
      Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:deleteitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      Operations.Delete(indexable, context);
    }
    else
    {
      Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updatingitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      if (!IsExcludedFromIndex(indexable, true))
      {
        if (operationContext != null && !operationContext.NeedUpdateAllLanguages)
 {
   if (!indexable.Item.Language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
   {
     CrawlingLog.Log.Debug(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : Update : English not requested {0}. Skipping.", indexable.Item.Uri));
            return;
   }
        }
     
 Item item;
 var languageItem = LanguageManager.GetLanguage(indexLanguage);
 using (new WriteCachesDisabler())
 {
   item = indexable.Item.Database.GetItem(indexable.Item.ID, languageItem, Version.Latest);
 }

 if (item == null)
 {
    CrawlingLog.Log.Warn(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : Update : Latest version not found for item {0}. Skipping.", indexable.Item.Uri));
 }
 else
 {
   Item[] versions;
   using (new SitecoreCachesDisabler())
   {
     versions = item.Versions.GetVersions(false);
   }

   foreach (var version in versions)
   {
     if (version.Version.Equals(item.Version))
     {
       UpdateItemVersion(context, version, operationContext);
     }
     else  
     {
       Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:deleteitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
       Delete(context, ((SitecoreIndexableItem)version).UniqueId);
     }
   }
 }
    
 Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updateditem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      }


      if (!DocumentOptions.ProcessDependencies)
      {
        return;
      }

      if (indexable.Item.Language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
      {
        Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updatedependents", index.Name, indexable.UniqueId, indexable.AbsolutePath);
 UpdateDependents(context, indexable);
      }
    }
  }
}

I did a few things here:
  • if the operationContext is not asking to update all languages, I check the language and get it out if it is not the index language
  • I get all versions, loop trough them and update the latest - other versions get a delete instruction
    • not sure if this is really needed as it might be sufficient to delete only the previous one
  • the call to update the dependent items was put in a language condition so that it was only executed when the requested language is the index language

Testing

And I started testing. Rebuild. Add versions. Update items. Constantly using Luke to investigate the index. It all seemed to work. 
Until I tried to add a new version in a language that was not supposed to be in the index. The new version was not send to the index, but it's previous version was. I tried to figure out what was happening and by following the flow through the existing SitecoreItemCrawler I found some options in the "IndexEntryOperationContext" that were used in the base Update function.

Update

So we also override the Update method:
public override void Update(IProviderUpdateContext context, IIndexableUniqueId indexableUniqueId, IndexEntryOperationContext operationContext, IndexingOptions indexingOptions = IndexingOptions.Default)
{
  operationContext.NeedUpdatePreviousVersion = false;
  base.Update(context, indexableUniqueId, operationContext, indexingOptions);
}

What I'm doing here is actually quite simple: I tell the crawler that he does not need to update previous versions, no matter what. As I am already updating all versions in the DoUpdate this seemed ok to do. By doing this, the problem was fixed and I did not had to copy too much code anymore.

Conclusion

The custom crawler works and does what it is supposed to do. It would have been nice though if the functions in the crawler provided by Sitecore were cut into smaller pieces to make it easier to override the pieces we want to change. I remember reading somewhere that Pavel Veller already managed to get this on a roadmap, so I hope that is true...

But for now, this worked for me. Glad to hear any remarks, suggestions, ...

Wednesday, September 21, 2016

Sitecore Symposium 2016 (New Orleans)

Sitecore Symposium 2016 : a developer view

New Orleans: city of lively music, amusing drinks, po-boy beignets and the 2016 Sitecore Symposium. 

In this post I will share some of my thoughts on attended sessions from a developers point of view. A bit more in-depth information on some topics will follow: blogged and presented on SUG Belux.

Starting the event with a brass band immediately set the tone: this was going to be an experience! Where did we here that word before... and again. In the keynote, between the announcements of the Sitecore Experience Accelerator (SXA), serious updates to Path Analyzer and EXM, or the new Commerce vNext. Not to forget Sitecore on Azure.

Day 1

The developer keynote was a good news show for developers with the "best practices" called Helix (Habitat is the example), a view on the open standards being used, .net core being embraced, a Sitecore nuget, and a move to more open source projects (Rocks, PathFinder, SIM, ...). But the main focus was community. Cool!
The community website should be known to all. The Slack channels get more and more users and usually someone is always there to answer your Sitecore question (in between the small talk). We should not forget Twitter, the many blogs out there and the user groups all over the world - including the amazing SugCon in Copenhagen this year. And recently a dedicated StackExchange site went in to beta. Proud to be part of all this!





xConnect

My first session was about xConnect. A promising new framework that should give us an (easy) API to work with analytics data. Sitecore will be using the framework itself in several modules. We will be able to use it to integrate all kinds of external data into xDB (write) or get all sorts of combined data out of xDB (read) to be used in code or external tools (e.g. Power BI). xConnext is not yet available, but will be shortly.




Commerce vNext

"Sitecore Commerce vNext is a brand new, state of the art commerce engine, designed from the ground up as an integral part of the Sitecore Experience Platform, built on ASP.NET Core." That's what they say. Sounds good though, and in just a few more months  we should be able to test if it. The demo showed a nicely integrated platform with lots of options in the box, but also an architecture that should provide lots of integration points where custom development - 'Plugins" -  can be hooked into the processes.



Sitecore Experience Accelerator (SXA)

Mentioned in two keynotes, so this SXA must be the new hype in Sitecore land. As it all looks nice with lots of "buts", I can't make up my mind on this one. It's something to look at and investigate further, that's for sure. It has some great features to rapidly create new sites and/or pages without the need for a developer. Front-end developers will be pleased to hear that in a future version we should be able to choose the grid system. It's a module with great opportunities but also with some challenges for implementation partners. Update-1 should be available in the very near future...


We finished our day with a warm-up in the partner pavillion, getting ready for the party in the House of Blues. This turned out to be quite an experience (didn't use that word enough so far). But this is a developers view so nor drinks nor a stage filled with Sitecore-women sounds interesting, right?




Day 2

The keynote of day 2 can be captured in: "Play hard. Work hard.". Sounds like a plan! 

Multi-lingual

I started the sessions with a view on how to conquer languages in Sitecore. We got an interesting round-up of what the current language fallback can do and how to use it. Unfortunately I got the feeling that the audience (including myself) hoped for some more real-life solutions to multi-lingual (sometimes combined with multi-site) issues. I hope I could help some people by pointing them to our EasyLingo module on the marketplace, and would be pleased to get feedback.

Security

After missing Bas Lijten's session at SugCon Copenhagen this year, I had to attend this time. Although his demo got pwned the session still provided some good information on presumably basic security settings that are unfortunately still not always tackled. I would like to point to Bas' project on GitHub on security in Sitecore projects. I must admit I didn't find the time to really check it out but it is definitely on my radar and should be on yours too.

Testing

Next in line was Dan Solovay - a fully seated session on unit testing in a Sitecore project.  Based on Sitecore 8.2, which made the session very interesting. The majority of the audience was already aware of FakeDB and unit testing principles, but Dan managed to point out some tips and tricks not everyone knew. And the new capabilities in Sitecore caused by the move towards interfaces and virtual functions are very promising for test-driven developers. If you are interested in unit-testing, make sure to check the information on Dan's blog.


Publishing

They saved the best for last... the new publishing service presented by Stephen Pope looks very good. It's a complete new way of handling the publishing, making it a separate service that handles it's stuff in bulk operations making it very fast and robust.

It's a very lightweight process on .net core. An interface was created in the Sitecore admin to see what is going on in the publish service, and developers will be pleased to know that there is also a debug mode allowing you to follow all the steps in a console window. The service is a first version now in Sitecore 8.2 and will evolve as some topics are still in the backlog, but it's already a huge change in a part of the framework that hadn't truly changed since..  well, a long time. Sites that need to handle large publishes will surely benefit and it will become easier for content editors as they don't need to know about different publishes nor need to wait for a counter that goes up to..  And as icing on the cake: "publish site" is gone! (except for admins who know where they have hidden the button).


Final

The final keynote gave us a sneak of the future, but if you want to know about that: start saving for Las Vegas 2017!




Conclusion

It was a great event. An experience (did I use it enough now?). It truly was. A symposium is a place to meet people. Which I did. A chance to say hi to guys you only see online the rest of the year because they reside in Sri Lanka (Chaturanga) or in the far away Netherlands (Bas, Robbert, ...) and to meet new people - and I surely still missed some guys.

A symposium is also a place to catch up on new stuff, get some insights, tips and tricks to create even better Sitecore solutions. And I did.

Although it might look quite impossible to make the next symposium even better, I would like to give one remark for the organizing team. Based upon feedback that I heard from several people after different sessions I would suggest to give session some sort of "level" indication. Now the sessions are divided between developers, marketers and business people which is good but in quite a few cases attendants came out of a session disappointed because they learned (almost) nothing. Sessions at a lower level are very useful and needed for a part of the audience, but there are quite a few experts as well - so it would be nice to know upfront what the target of the session is. It's probably more difficult then it sounds but I'm quite sure it would make the experience even better.

Experience!