Sitecore Experience Accelerator: SXA and Search

First encounters with SXA: Search components

I was investigating the possibilities of SXA, the new Sitecore Experience Accelerator, 1.1. Questions were put on the Sitecore Stack Exchange site - not only by myself of course, also by many others - and that made some things clear. One of the subject I was testing was the search functionality, using the standard components and not adding any code. As it was all on a small test setup, I did not go as far as testing performance and stability, the focus was more on functionality.

But here they are - my first encounters with SXA to share.

Setup a search box and results page

First thing to do was setting up a search result page. I created a page just like any other and added the "Search Results" component from the Toolbox.

Next step was to  have a search box to feed the search page. I placed this one on a partial design to include it on all pages.  As I am still using the built-in basic theme, the search box looks like this:

The labels can be altered. The search box has some properties that can be set: first one seems quite important is the search results page. So we select the page that we created before. We also set the maximum number of predictive results shown in the dropdown. Yes, that's right - the search box has predictive results:

We save, publish it all. And there is our search! As we had some content pages, we get results from the index in our search box and we get send to the search results page as expected. The request to the search results page has the search query in the url query parameters, so analytics tools can pick it up if needed.

Some extra's

Of course, a page with just search results is not very user friendly so we add some more components. Let's add the "Results Count" on top, a dropdown "Filter" as well and the "Page Selector" at the bottom. We will go for a fixed page size, and have set it's value in the Search Results component. We now have something like this:

We have set the page size to 2, which is not realistic but managed to show the pagers working without having to add too much demo content pages.

So now we have a search box, results page with counter and paging and no letter code. 

Facets

Our filter is still empty now. The pages in SXA have tagging by default and we did tag all of them with one or more tag values. So lets try to filter on those tags!

First we create a facet "Tag".

In the facet item we set the name and display name and the field name. This is the lower case name of the field that is used in the index and that the facet is based on, in our case it's "sxatags". If you don't know the exact name of the fields you can always find this by looking at the templates used by the (SXA) items.

Once we have the facet item, we create a datasource for our Filter component and set the Facet field to the one we just created. Et voila, we publish and see:

facets with numbers in the dropdown filter box. By selecting one of them, the search results get filtered on the tag. 

Still no line of code...

And further... a Tag Cloud

We also added a Tag Cloud component - and noticed it works very well with our newly created search. 

As it is not styled, it looks like ****, but it works. I get my tags as used in the site and when I select my search results page as a parameter of the tag cloud components, clicking the tags gets me to my search results page with a search on the tag.

Customizing

You can alter the search results look by creating a rendering variant. The default will show the Title or display name of the items, but you can add fields to the default variant, or create a new one and select this custom one in your search results component. You need to create VariantField items to achieve this. 

Note that you can not alter the results look per template type (e.g. add an image to results of type 'Product') but you can add extra fields for it, and if the fields does not exist it will just be skipped.

Issues and/or questions - The conclusion

Questions will rise and problems as well. What is for me still unclear is the way the index behind the search engine works. The search uses one field, "SxaContent" - which is a computed field. But for now it's not very clear what is included in that field (next to the default title and content) and what is not - or how to get things included. 

Another mystery is the updates. As I did some research to using Lucene/Solr indexes for a full site search in the past, I am always curious to see how the updates are handled in index based search solutions. How does a "page" know that some content in a datasource is updated? 

From the tests I made, updating a shared datasource seems to work - all pages are updated. But I had some trouble when I added a new component with an existing datasource to a page - that did not get updated in the index.  That will require some more investigation.

Conclusion

The conclusion is that we have a working search without coding and in a very short time. Does it have all the features that "a customer" might want? No, maybe not. But do compared to the effort needed to put it on the site it's definitely worth considering! A promising first encounter with SXA...

Custom Sitecore index crawler

Why? - The case

We needed to find items in the master database quickly, based on some criteria. Language didn't matter and we didn't need the actual item, a few properties would do. So we decided to create a custom index with the fields we needed indexed (the criteria) and/or stored (the properties). All fine, but as we were using the master database we had different versions of each item in the index. As we needed only the latest version in only one language we though we could optimize the index to only contain those versions.

First attempt

We had to override the SitecoreItemCrawler. That was clear. The first attempt was creating a custom IsExcludedFromIndex function that would stop all entries not in English and not the latest version. Pretty simple, but does not work.
First of all, all entries were in English.. this function is only called per item and not per version. So actually, we could not use this. Furthermore, we did not take into account the fact that when adding a new version, we have to remove the previously indexed one.

Don't index multiple versions

I started searching the internet and found this post by Martin English on (not) indexing multiple versions. Great post, and pointing as well towards a solution with inbound filters. But as those filters work for every index, that would be no solution here. I needed it configurable per index. So back to Martins' post. We had to overwrite DoAdd and DoUpdate.

A custom index crawler

The result was a bit different as I was using Sitecore 8.1 and also wanted to include a language filter. I checked the code from the original SitecoreItemCrawler, created a class overwriting it and adapted where needed.

Language

I made the language configurable by putting it into a property:

private string indexLanguage;

public string Language
{
  get  {  return !string.IsNullOrEmpty(indexLanguage) ? indexLanguage : null; }
  set  {  indexLanguage = value; }
}

DoAdd

The DoAdd method was changed by adding an early check in the language-loop to get out when not the requested language. I also removed the version-loop with a request for the latest version so that only that version gets send to the index.

protected override void DoAdd(IProviderUpdateContext context, SitecoreIndexableItem indexable)
{
  Assert.ArgumentNotNull(context, "context");
  Assert.ArgumentNotNull(indexable, "indexable");
  using (new LanguageFallbackItemSwitcher(context.Index.EnableItemLanguageFallback))
  {
    Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:adding", context.Index.Name, indexable.UniqueId, indexable.AbsolutePath);
    if (!IsExcludedFromIndex(indexable, false))
    {
      foreach (var language in indexable.Item.Languages)
      {
        // only include English
        if (!language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
        {
          continue;
        }

        Item item;
        using (new WriteCachesDisabler())
        {
          item = indexable.Item.Database.GetItem(indexable.Item.ID, language, Version.Latest);
        }

        if (item == null)
        {
          CrawlingLog.Log.Warn(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : AddItem : Could not build document data {0} - Latest version could not be found. Skipping.", indexable.Item.Uri));
        }
        else
        {
          SitecoreIndexableItem sitecoreIndexableItem;
          using (new WriteCachesDisabler())
          {
            // only latest version
            sitecoreIndexableItem = item.Versions.GetLatestVersion();
          }

          if (sitecoreIndexableItem != null)
          {
            IIndexableBuiltinFields indexableBuiltinFields = sitecoreIndexableItem;
            indexableBuiltinFields.IsLatestVersion = indexableBuiltinFields.Version == item.Version.Number;
            sitecoreIndexableItem.IndexFieldStorageValueFormatter = context.Index.Configuration.IndexFieldStorageValueFormatter;
            Operations.Add(sitecoreIndexableItem, context, index.Configuration);
          }
        }
      }
    }

    Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:added", context.Index.Name, indexable.UniqueId, indexable.AbsolutePath);
  }
}

DoUpdate

For the DoUpdate method I did something similar although I had to change a bit more here.

protected override void DoUpdate(IProviderUpdateContext context, SitecoreIndexableItem indexable, IndexEntryOperationContext operationContext)
{
  Assert.ArgumentNotNull(context, "context");
  Assert.ArgumentNotNull(indexable, "indexable");
  using (new LanguageFallbackItemSwitcher(Index.EnableItemLanguageFallback))
  {
    if (IndexUpdateNeedDelete(indexable))
    {
      Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:deleteitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      Operations.Delete(indexable, context);
    }
    else
    {
      Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updatingitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      if (!IsExcludedFromIndex(indexable, true))
      {
        if (operationContext != null && !operationContext.NeedUpdateAllLanguages)
 {
   if (!indexable.Item.Language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
   {
     CrawlingLog.Log.Debug(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : Update : English not requested {0}. Skipping.", indexable.Item.Uri));
            return;
   }
        }
     
 Item item;
 var languageItem = LanguageManager.GetLanguage(indexLanguage);
 using (new WriteCachesDisabler())
 {
   item = indexable.Item.Database.GetItem(indexable.Item.ID, languageItem, Version.Latest);
 }

 if (item == null)
 {
    CrawlingLog.Log.Warn(string.Format(CultureInfo.InvariantCulture, "SitecoreItemCrawler : Update : Latest version not found for item {0}. Skipping.", indexable.Item.Uri));
 }
 else
 {
   Item[] versions;
   using (new SitecoreCachesDisabler())
   {
     versions = item.Versions.GetVersions(false);
   }

   foreach (var version in versions)
   {
     if (version.Version.Equals(item.Version))
     {
       UpdateItemVersion(context, version, operationContext);
     }
     else  
     {
       Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:deleteitem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
       Delete(context, ((SitecoreIndexableItem)version).UniqueId);
     }
   }
 }
    
 Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updateditem", index.Name, indexable.UniqueId, indexable.AbsolutePath);
      }


      if (!DocumentOptions.ProcessDependencies)
      {
        return;
      }

      if (indexable.Item.Language.Name.Equals(indexLanguage, StringComparison.OrdinalIgnoreCase))
      {
        Index.Locator.GetInstance<IEvent>().RaiseEvent("indexing:updatedependents", index.Name, indexable.UniqueId, indexable.AbsolutePath);
 UpdateDependents(context, indexable);
      }
    }
  }
}

I did a few things here:

• if the operationContext is not asking to update all languages, I check the language and get it out if it is not the index language
• I get all versions, loop trough them and update the latest - other versions get a delete instruction
• not sure if this is really needed as it might be sufficient to delete only the previous one
• the call to update the dependent items was put in a language condition so that it was only executed when the requested language is the index language

Testing

And I started testing. Rebuild. Add versions. Update items. Constantly using Luke to investigate the index. It all seemed to work. 

Until I tried to add a new version in a language that was not supposed to be in the index. The new version was not send to the index, but it's previous version was. I tried to figure out what was happening and by following the flow through the existing SitecoreItemCrawler I found some options in the "IndexEntryOperationContext" that were used in the base Update function.

Update

So we also override the Update method:

public override void Update(IProviderUpdateContext context, IIndexableUniqueId indexableUniqueId, IndexEntryOperationContext operationContext, IndexingOptions indexingOptions = IndexingOptions.Default)
{
  operationContext.NeedUpdatePreviousVersion = false;
  base.Update(context, indexableUniqueId, operationContext, indexingOptions);
}

What I'm doing here is actually quite simple: I tell the crawler that he does not need to update previous versions, no matter what. As I am already updating all versions in the DoUpdate this seemed ok to do. By doing this, the problem was fixed and I did not had to copy too much code anymore.

Conclusion

The custom crawler works and does what it is supposed to do. It would have been nice though if the functions in the crawler provided by Sitecore were cut into smaller pieces to make it easier to override the pieces we want to change. I remember reading somewhere that Pavel Veller already managed to get this on a roadmap, so I hope that is true...

But for now, this worked for me. Glad to hear any remarks, suggestions, ...

Sitecore Symposium 2016 (New Orleans)

Sitecore Symposium 2016 : a developer view

New Orleans: city of lively music, amusing drinks, po-boy beignets and the 2016 Sitecore Symposium. 

In this post I will share some of my thoughts on attended sessions from a developers point of view. A bit more in-depth information on some topics will follow: blogged and presented on SUG Belux.

Starting the event with a brass band immediately set the tone: this was going to be an experience! Where did we here that word before... and again. In the keynote, between the announcements of the Sitecore Experience Accelerator (SXA), serious updates to Path Analyzer and EXM, or the new Commerce vNext. Not to forget Sitecore on Azure.

Day 1

The developer keynote was a good news show for developers with the "best practices" called Helix (Habitat is the example), a view on the open standards being used, .net core being embraced, a Sitecore nuget, and a move to more open source projects (Rocks, PathFinder, SIM, ...). But the main focus was community. Cool!

The community website should be known to all. The Slack channels get more and more users and usually someone is always there to answer your Sitecore question (in between the small talk). We should not forget Twitter, the many blogs out there and the user groups all over the world - including the amazing SugCon in Copenhagen this year. And recently a dedicated StackExchange site went in to beta. Proud to be part of all this!

xConnect

My first session was about xConnect. A promising new framework that should give us an (easy) API to work with analytics data. Sitecore will be using the framework itself in several modules. We will be able to use it to integrate all kinds of external data into xDB (write) or get all sorts of combined data out of xDB (read) to be used in code or external tools (e.g. Power BI). xConnext is not yet available, but will be shortly.

Commerce vNext

"Sitecore Commerce vNext is a brand new, state of the art commerce engine, designed from the ground up as an integral part of the Sitecore Experience Platform, built on ASP.NET Core." That's what they say. Sounds good though, and in just a few more months  we should be able to test if it. The demo showed a nicely integrated platform with lots of options in the box, but also an architecture that should provide lots of integration points where custom development - 'Plugins" -  can be hooked into the processes.

Sitecore Experience Accelerator (SXA)

Mentioned in two keynotes, so this SXA must be the new hype in Sitecore land. As it all looks nice with lots of "buts", I can't make up my mind on this one. It's something to look at and investigate further, that's for sure. It has some great features to rapidly create new sites and/or pages without the need for a developer. Front-end developers will be pleased to hear that in a future version we should be able to choose the grid system. It's a module with great opportunities but also with some challenges for implementation partners. Update-1 should be available in the very near future...


We finished our day with a warm-up in the partner pavillion, getting ready for the party in the House of Blues. This turned out to be quite an experience (didn't use that word enough so far). But this is a developers view so nor drinks nor a stage filled with Sitecore-women sounds interesting, right?

Day 2

The keynote of day 2 can be captured in: "Play hard. Work hard.". Sounds like a plan! 

Multi-lingual

I started the sessions with a view on how to conquer languages in Sitecore. We got an interesting round-up of what the current language fallback can do and how to use it. Unfortunately I got the feeling that the audience (including myself) hoped for some more real-life solutions to multi-lingual (sometimes combined with multi-site) issues. I hope I could help some people by pointing them to our EasyLingo module on the marketplace, and would be pleased to get feedback.

Security

After missing Bas Lijten's session at SugCon Copenhagen this year, I had to attend this time. Although his demo got pwned the session still provided some good information on presumably basic security settings that are unfortunately still not always tackled. I would like to point to Bas' project on GitHub on security in Sitecore projects. I must admit I didn't find the time to really check it out but it is definitely on my radar and should be on yours too.

Testing

Next in line was Dan Solovay - a fully seated session on unit testing in a Sitecore project.  Based on Sitecore 8.2, which made the session very interesting. The majority of the audience was already aware of FakeDB and unit testing principles, but Dan managed to point out some tips and tricks not everyone knew. And the new capabilities in Sitecore caused by the move towards interfaces and virtual functions are very promising for test-driven developers. If you are interested in unit-testing, make sure to check the information on Dan's blog.

Publishing

They saved the best for last... the new publishing service presented by Stephen Pope looks very good. It's a complete new way of handling the publishing, making it a separate service that handles it's stuff in bulk operations making it very fast and robust.

It's a very lightweight process on .net core. An interface was created in the Sitecore admin to see what is going on in the publish service, and developers will be pleased to know that there is also a debug mode allowing you to follow all the steps in a console window. The service is a first version now in Sitecore 8.2 and will evolve as some topics are still in the backlog, but it's already a huge change in a part of the framework that hadn't truly changed since..  well, a long time. Sites that need to handle large publishes will surely benefit and it will become easier for content editors as they don't need to know about different publishes nor need to wait for a counter that goes up to..  And as icing on the cake: "publish site" is gone! (except for admins who know where they have hidden the button).

Final

The final keynote gave us a sneak of the future, but if you want to know about that: start saving for Las Vegas 2017!

Conclusion

It was a great event. An experience (did I use it enough now?). It truly was. A symposium is a place to meet people. Which I did. A chance to say hi to guys you only see online the rest of the year because they reside in Sri Lanka (Chaturanga) or in the far away Netherlands (Bas, Robbert, ...) and to meet new people - and I surely still missed some guys.

A symposium is also a place to catch up on new stuff, get some insights, tips and tricks to create even better Sitecore solutions. And I did.

Although it might look quite impossible to make the next symposium even better, I would like to give one remark for the organizing team. Based upon feedback that I heard from several people after different sessions I would suggest to give session some sort of "level" indication. Now the sessions are divided between developers, marketers and business people which is good but in quite a few cases attendants came out of a session disappointed because they learned (almost) nothing. Sessions at a lower level are very useful and needed for a part of the audience, but there are quite a few experts as well - so it would be nice to know upfront what the target of the session is. It's probably more difficult then it sounds but I'm quite sure it would make the experience even better.

Experience!

EasyLingo 2.1: multi-site, multi-lingual and now with XP editor support

EasyLingo 2.1

EasyLingo 2.1 was just released to the marketplace. After version 1 we promised to include support for the experience editor and so we did. Kris wrote a nice post about the how-to.

Once in the experience editor, in the view tab amongst the other "bars" you can select the "Language bar" and you will get the same EasyLingo experience as in the content editor.

We also added one more (optional) version section: in Sitecore you can create versions of an item in languages that are not registered in the systems settings. These versions are now listed as well -separately- by the module. Note that these versions will always use a general flag as we cannot detect it from the system items.

More information on this great module in my previous post. You can find the code and all documentation files on GitHub.  The module itself is available on the Sitecore marketplace.

EasyLingo

EasyLingo, our first module on the Sitecore marketplace

Why

This language expansion module was designed to simplify content editing when using a large amount of languages in your Sitecore environments. When you have multiple sites with different language sets (e.g. a Belgian site in Dutch/French, a Swiss site in fr/de/it and a Danish site in dk/en) or simply a large volume of available languages, managing these languages in Sitecore can become cumbersome.

Sitecore has no out of the box functionality that restricts or manages the languages used per node/site and only offers the languages list to navigate through the available languages. The language list presented by Sitecore comprises of both those with one or more versions as well as those that have no version available or are irrelevant for that context/site.

This module was created on a Sitecore 8.1 platform together with my colleague and Sitecore MVP Kris Verheire and is now available on the Sitecore marketplace.

What

This module consists of 2 parts, which can be activated separately.

Extra languages section in the content editor

The bar consists of 3 language lists:

• On the first line, two of the lists are displayed. On the left hand side, the first list shows all the languages that have an actual version available. 
• The right side shows the second list with all the languages for which a fallback version exists (working through Sitecore’s item fallback available as of version 8.1). 
• The second line hold the third list that shows languages which are available on this node/site, but for which no version exists. This line will disappear if there are no such languages.

The versions with fallback on the right are rendered in italic. The current language on the Sitecore context is visualized in bold.

Languages in all the lists are clickable so the editor can easily switch languages without having to go through the whole list.

In order to get a nice view like in the screenshots it is necessary to manually set the icons of the languages in the system section of Sitecore (as Sitecore does not use these icons anymore in the list, they are not set by default. Flag-icons are still available though).

How are “available” languages detected?

We loop through the configured sites and check whether the current item is part of it (based on the
path in the content tree and the “rootPath” of the site). For each site we check if a custom property “allowedLanguages” is available in the SiteSettings specific site node. The distinct union of all these languages will be the list of allowed languages for the item. If the list is empty, all available languages in the Sitecore instance are allowed.

How to use the “allowedLanguages” setting?

The parameter will be part of a <site /> definition (just like hostname, database, rootPath and so on). It is not obligatory – the module works perfectly without it. It can be used for multi-site solution with different languages. For our example mentioned above we could have something like:

<site name=”Belgian” allowedLanguages="nl,fr-BE" … />
<site name=”Swiss” allowedLanguages="fr,de,it" … />
<site name=”Danish” allowedLanguages="dk,en" … />

The language should match the used language exactly (so “fr-BE” ≠ “fr”) and can be completely different or overlap. For the second part of the module it is important though to put the “default” language first.

Second part: request resolver

The second part of the module consists of a request resolver that is put in the httpRequestBegin pipeline after the LanguageResolver.
The current requested URL is checked to disable the functionality for Sitecore URL’s or when not in normal page mode. The current context site is checked for the ‘allowedLanguages’ parameter. If found, and the current language is not part of the ‘allowedLanguages’ set we will redirect the user to homepage in the default language (default is the first language in the set).

Future features

The EasyLingo bar is now only available in the Content Editor. The next step would be to introduce a language bar in Sitecore’s Experience Editor that allows users to quickly switch between the available and allowed languages for the content they are editing. (is already in progress)
We are also working on adding versions of items in languages that are not listed in the system languages.

More information

• Sitecore marketplace
• GitHub

Sitecore WFFM: act on success

The question I had to tackle was to change the redirect after a successful form submit based on a form value. By doing so I learned we can do a lot on success (or on error). Using Sitecore 8.1...

Success (and error) WFFM pipelines

All comes down to finding the correct pipelines - as usual in Sitecore. In our case we found a pipeline in case of success and one in case of error. But be aware: there is a big difference between the webforms and mvc solution.

Webforms

The pipelines in case of webforms are:

• <successAction>
• <errorSubmit>

These can be found in Sitecore.Forms.config.

Mvc

The pipelines in case of mvc are:

• <wffm.success>
• <wffm.error>

These can be found in Sitecore.MvcForms.config.

Custom redirect after success submit

In case of webforms, we could write a processor that redirects to the location we want and place it in the successAction pipeline as first processor (just before the original 'SuccessRedirect').

In case of MVC, it's a different story.

MVC solution 

The wffm.success is by default empty. The actual redirect happens in another pipeline later on. So redirecting here is not the right thing to do. I started writing the processor and noticed that I had a lot of information in a Sitecore.Forms.Mvc.Models.FormModel.

You have access to all the fields, and to the SuccessRedirectUrl property. At this point this is filled with the url as set in Sitecore (which is very useful as a base) and you can alter it as you please. The actual redirect will happen later but it will use the value you have set here.

Code example

public class SuccessRedirect : FormProcessorBase<IFormModel>
{
    public override void Process(FormProcessorArgs<IFormModel> args)
    {
        Assert.ArgumentNotNull((object)args, "args");
        var model = args.Model as FormModel;
        if (model == null)
            return;
         model.SuccessRedirectUrl = model.SuccessRedirectUrl + "?val=" + model.Results[0].Value;
    }
 }

This example shows you to inherit from Sitecore.Forms.Mvc.Pipelines.FormProcessorBase and implement the Process method. Cast the arguments Model property to a FormModel and start using it. I add a querystring to the redirect url with the value of the first field - sounds useless but it's just an example.

Conclusion

As often in WFFM there is a big difference between Webforms and Mvc. I tried the mvc way, and found that you can actually do a lot before Sitecore handles the model. You can change the redirect url, but there are probably also other useful possibilities here for scenarios I can't think of now...

Sitecore Index dependencies

I recently stumbled upon a question on how to trigger re-indexing of related content in a Sitecore (Lucene) index. Different answers were given and I got the feeling that not everyone already knows about the getDependencies pipeline. So we write a blog post...

Re-index related content

As I mentioned, there are other solutions that could do the trick. 

• Custom update strategy

  You could write your own update strategy and include your dependency logic in there. This approach has the benefit that you can use it in one index only without affecting others.

• Custom save handler

  With a custom save handler you could detect save actions, get the dependent items and register them as well for index updating. I'm not convinced that this will work in all update strategy scenario's but if you have working code, feel free to share ;)

These are probably also valid solutions, but I'll leave those to others as I want to show the Sitecore pipeline that looks like the ideal candidate for the job.

getDependencies pipeline

There is a pipeline.. there always is. One drawback I'll mention already is that the pipeline is for all indexes and so far I have not found a way to trigger it for one index only (see update below on disabling). I also tried to get the index (name or anything) in the code but that didn't work out either. We could get the name of the job, but that was only relevant for the first batch of items - after that, multiple jobs were started and the name became meaningless. 

Anyway, the pipeline. In the Sitecore.ContentSearch.config you'll find this:

<!-- INDEXING GET DEPENDENCIES
  This pipeline fetches dependant items when one item is being index. Useful for fetching related or connected items that also
  need to be updated in the indexes.
  Arguments: (IQueryable) Open session to the search index, (Item) The item being indexed.
  Examples: Update clone references.
  Update the data sources that are used in the presentation components for the item being indexed.
-->

<indexing.getDependencies help="Processors should derive from Sitecore.ContentSearch.Pipelines.GetDependencies.BaseProcessor">
  <!-- When indexing an item, make sure its clones get re-indexed as well -->
  <!--<processor type="Sitecore.ContentSearch.Pipelines.GetDependencies.GetCloningDependencies, Sitecore.ContentSearch"/>-->
  <!-- When indexing an item, make sure its datasources that are used in the presentation details gets re-indexed as well -->
  <!--<processor type="Sitecore.ContentSearch.Pipelines.GetDependencies.GetDatasourceDependencies, Sitecore.ContentSearch"/>-->
</indexing.getDependencies>

As you can see, some processors are in the box, but in comments. You can simply enable them if you want your clones and/or datasources to be indexed with the main items.

And you can write your own processor of course. An example:

public class GetPageDependencies : Sitecore.ContentSearch.Pipelines.GetDependencies.BaseProcessor
{
    public override void Process(GetDependenciesArgs context)
    {
        Assert.IsNotNull(context.IndexedItem, "indexed item");
        Assert.IsNotNull(context.Dependencies, "dependencies");
            
        var scIndexable = context.IndexedItem as SitecoreIndexableItem;
        if (scIndexable == null) return;
            
        var item = scIndexable.Item;
        if (item == null) return;
            
        // optimization to reduce indexing time by skipping this logic for items not in the Web database
        if (!string.Equals(item.Database.Name, "web", StringComparison.OrdinalIgnoreCase)) return;
            
        if (!item.Paths.IsContentItem) return;
            
        if (item.Name.Equals("__Standard Values", StringComparison.OrdinalIgnoreCase)) return;
            
        if (Sitecore.Context.Job == null) return;
            
        // logic here - example = get first child
        if (!item.HasChildren) return;
            
        var dependency = item.Children[0];
        var id = (SitecoreItemUniqueId)dependency.Uri;
        if (!context.Dependencies.Contains(id))
        {
            context.Dependencies.Add(id);
        }
    }
}

In the example here we keep it simple and just add the first child (if any). That logic can contain anything though.

As you can see we try to get out of the processor as fast as possible. You can add even more checks based on template and so on. Getting out fast if you don't want the dependencies is important!

The benefit of the solution is that the pipeline is executed when the indexing starts but before the list of items to index is finalized - which is the best moment for this task. All "extra" items are added to the original list so they are executed (indexed) by the same job and we let the Sitecore handle them they way it was meant.

Performance might not seem an issue, but when having quite some items and dependencies, and these get updated frequently it will be. You might be triggering way too much items towards the index, so be careful (no matter what solution you go for). The indexing is be a background job but if it goes berserk you will notice.
Note that it is a good thing that your dependencies don't have to go through all kind of processes before being added, they are just "added to the list".

I found this pipeline solution very useful in scenario's where the amount of dependent items that actually got added was not too big. Don't forget you can also disable the pipeline processor temporarily (and perform a rebuild) if needed.

How to Enable/Disable 

(from the Sitecore Search and Indexing on SDN) - thx jammykam for the info

The pipeline is executed from within each crawler if the crawler’s ProcessDependencies property is set to true, which is the default. To disable this feature, add the following parameter to the appropriate index under the <Configuration /> section.

<index id="content" ...>
 ...
 <Configuration type="...">
...
 <ProcessDependencies>false</ProcessDependencies>

Alternatively, if the indexes don’t override default configuration with a local one, you can also globally change this setting in the DefaultIndexConfiguration.

Known issues with the indexing.getDependencies pipeline

https://kb.sitecore.net/articles/116076