Thursday, January 11, 2018

Sitecore Experience Accelerator SXA 1.6: Snippets

Sitecore Experience Accelerator 1.6

Together with Sitecore 9.0 update-1 we welcomed SXA 1.6.
One of the things I was really looking forward to are Snippets:
The Snippet rendering lets you create a reusable group of renderings. It is a composite rendering that consists of several renderings that can be designed separately in the Experience Editor.
Our analists asked for this a while ago as it makes life for editors potentially a lot easier. Partial designs are nice, but their content is fixed - you cannot alter anything on the page itself, not even switch datasources. With snippets you can now create your own "composite rendering" (in the end, that is indeed what it is) and reuse this on lots of pages. Unlike partials designs, you do need to place the snippet on the page yourself - it doesn't come automatically with the page design.

Create a grid based on (multiple) splitters and add other renderings in it. Save this bunch, reuse it, and be able to still adapt the content if needed. Oh yes, this will be used!

I installed SXA 1.6 on a vanilla Sitecore 9.0-1 (installing is still a piece of cake btw) and tried them out.

Creating a snippet

To add a snippet on your page, select Snippet from the Composites section in the toolbox (experience editor). Just as you would add any other rendering. Select the location -placeholder- where you want the snippet to appear and drag & drop.

You will get the screen to select the "Associated Content" or datasource. This is an item of type Snippet. You get 2 possible locations presented: a global folder for your site called Snippets and the local Data folder (which is located as a child underneath your current item). People working with Sitecore will recognize such a screen. 

If you want to reuse your snippet on other pages, make sure to put it in the global Snippets folder as your local Data one will not be available on other pages in your site. 

The screenshot shows an example after we had created a demo item in each folder.

Filling the snippet

You can add anything you want to the snippet. Just drag & drop all the desired renderings on it, enter content, add more datasources and so on. Just as if the snippet wasn't there. 

For my test, I just added a column splitter with a RichText component in the first column and a reusable RichText component in the second.

If you check the created items in the content editor you will notice that all was done as expected - the local datasources are underneath the snippet item in a Data folder and the reusable RT component datasource item was also on it's normal place in the global texts folder.

Reusing the snippet

Datasource Configuration

Before you start reusing the snippet, take a look at this section in the snippet item:

This is the datasource configuration which by default (standard values) will be set on "Do not copy".
You need to think about this one. I didn't find a way to set this in the experience editor by the way - if someone knows how to do that please share.. and if not possible the SXA team may put this on their backlog ;)

What are the options here:
  • Do not copy - use global datasource : if this is set when the snippet is reused, the datasources are not created locally but instead the snippet will refer to the original datasources. Meaning that if you change anything, it will also change on all other locations where the snippet is used. The snippets remain coupled.
  • Copy global data source to local context upon slection: if this is set when the snippet is reused, the snippet item and all the related datasources (it's children) are copied to the current item's local Data folder. Meaning that you can change all data within the snippet without affecting anything else. The snippets are not coupled anymore.
  • Ask user whether the copy of global data source to local context is required upon selection: seems obvious.. ask the user which of the above he wants.
I tried all three of them and will focus further on the last one. 

If this is set, and the user added his snippet a screen is presented to him/her as seen in the screenshot on the right. For me this was crystal clear, but I did get the comment that for a (simple) end user that might not be the case. Haven't been able to test that yet, so future will tell. Anyway, I added the snippet I created twice on a new page - once with "yes" and once with "no".

If I looked in the content editor all looked fine (at first sight). The items were created locally one - for the snippet where I selected "yes" as expected. Also, as expected, the reusable RichText component datasources were not copied - the remained in the global data folder.  In the experience editor all looked fine as well. 

Small issue
Too bad I did find a little issue though.. apparently in case of "yes" the datasource items were created fine, but were not set in the snippet rendering - so the snippet was still using the global ones. It does work when you use the "copy to local" settings, so the issue is only when letting the user choose. I created a ticket with Sitecore Support and will edit here when I get a fix. 


As I was really looking forward to it, it's a shame I found an issue - but as I'm sure the sxa team will fix this (and I will keep you posted). The feature has everthing I think I wanted so actually: Hooray for Boo.. no, snippets :)

Wednesday, January 10, 2018

Upgrading to Sitecore 9 update-1 (rev. 171219)

Notes from upgrading from a 9.0 initial release

I just upgraded a stadalone (vanilla) install of Sitecore 9.0 initial release (rev. 171002) -installed with SIF- to update-1 (rev. 171219). All went fine, no real issues but a few small tips might help...
Note that we had installed our initial version with SIF
If you have used SIF-less (a UI Wrapper for Sitecore Installation Framework) remember that your result will be the same as it uses SIF underneath.

Sitecore configuration files

When installing Sitecore with SIF, some of your parameters are set inside the Sitecore configuration files. This will give conflicts when upgrading to update-1. Luckily the Sitecore upgrade wizard is now smart enough to tell us this. I had conflicts in:
  •  App_Config\Sitecore.config: The 'dataFolder' variable has been manually modified (weird?)
  • App_Config\Sitecore\ContentSearch\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config:  the url to our Solr instance was in here
  • App_Config\Sitecore\ContentSearch\Sitecore.ContentSearch.Solr.Index.Master.config
    App_Config\Sitecore\ContentTesting\Sitecore.ContentTesting.Solr.IndexConfiguration.config: <param desc="core">...</param> is in there and changed to $(id) instead of the name of the core - so if that doesn't match the name of the index it won't work anymore. Note that this is only changed for some indexes like master and core, not for others like web.
  • Web.config: the search provider was reset to Lucene - had to switch back to Solr

An mentioned Sitecore will warn you about these changes so no worries - just make sure that you do review the configs after the upgrade or you will get errors.

EXM install

EXM is "new" in this release (not really new, but it wasn't available in the initial release) and comes packed within the Sitecore box now. During the upgrade process you will need to deploy the EXM databases. It won't tell you how to set your SQL user though (or it assumes that we use the dbo - do not!). I tried to figure it out (see StackExchange).
My conclusion for the user in the connection string -at the moment- is: 
  • read/write rights on both databases
  • execute rights on the exm.master database

Tuesday, January 9, 2018

Custom Sitecore DocumentOptions with Solr

Almost 2 years ago I wrote a post about using custom indexes in a Helix environment. That post is still accurate, but the code was based on Lucene. As we are now all moving towards using Solr with our (non-PAAS) Sitecore setups, I though it might be a good idea to bring this topic back on the table with a Solr example this time.

(custom) indexes

I am assuming that you know about Helix, and about custom indexes. If you ever created a custom index you probably have used the documentOptions configuration section - maybe without noticing. It is used to include and/or exclude fields and templates and define computed fields. So you probably used it :)

And it wouldn't be Sitecore if we couldn't customize this...

Our own documentOptions

Why? Because we can. No..  we might have a good reason, like making our custom index definitions (more) Helix compliant. Normally your feature will not have a clue about "page" templates. But what if you want to define the include templates in your index? Those could be page templates.. or at least templates that inherit from your feature template. That is why I build my own documentOptions - to include a way to include templates derived from template-X.


So the idea now is to create a custom document options class by inheriting from the SolrDocumentBuilderOptions. We add a new method to allow adding templates in a new section with included base templates. This will not break any other existing configuration sections.

An example config looks like:
<documentOptions type="YourNamespace.TestOptions, YourAssembly">
    <include hint="list:AddIncludedBaseTemplate">
This looks very familiar - as intended. We create a new include section with the hint "list:AddIncludedBaseTemplate". The name 'AddIncludedBaseTemplate' will come back later in our code.



public virtual void AddIncludedBaseTemplate(string templateId)
  Assert.ArgumentNotNull(templateId, "templateId");
  ID id;
  Assert.IsTrue(ID.TryParse(templateId, out id), "Configuration: AddIncludedBaseTemplate entry is not a valid GUID. Template ID Value: " + templateId);
  foreach (var linkedId in GetLinkedTemplates(id))
    AddTemplateFilter(linkedId, true);
To see the rest of the code, I refer to the original post as nothing has to be changed to that in order to make it work on Solr (instead of Lucene).


To change the code from the Lucene example to a Solr one, we just had to change the base class to SolrDocumentBuilderOptions. 
We are now again able to configure our index to only use templates that inherit from our base templates. Still cool. And remember you can easily re-use this logic to create other document options to tweak your index behavior. 

Friday, January 5, 2018

A Sitecore 9 upgrade story

Sitecore 9 upgrade

I was asked to do an upgrade of a Sitecore 8.2 site to the new Sitecore 9. The site includes

Sitecore9 - © @jammykam
Sitecore 9 - © @jammykam
  • xDB with custom facets, outcomes, ...
  • WFFM with custom save actions (e.g. write data to the custom facets)
  • custom indexes (on Lucene)
  • extra publishing target
  • Sitecore Powershell Extensions
  • ...
For the upgrade we decided to use the default upgrade tools as explained in the Sitecore Upgrade guide which can be found on the developer network site.

We did bump into a few issues - some of them our fault, others maybe not.. :)
But I though it might be a good idea to share the experience - the upgrade is not fully finished yet so I might be adding stuff later.
First tip of the day:  stay awake and follow the install guide thoroughly. Apparently during the process we sometimes forgot a (small) step which included in errors later on - easily fixed by doing that step but taking you much longer because you need to figure out what's wrong..

The upgrade package

The analysis of the upgrade package gave us a lot of warnings. Most of them could be ignored, some of them addressed code: our code and the Powershell Extensions. We decided to ignore this as it was a test and we wanted to know if the upgrade would succeed without removing that code. Of course, we had removed all configs that might break stuff (like the ones regarding custom WFFM stuff - as you do need to disable the standard WFFM configs as well). We had no complaints about configs - those were clean :)

The installation of the package went well.. almost. At the very end it gave an error:
"An error occured while copying files. Some files might not have been updated."

Inspection of the logfiles showed us that a file could not be deleted and we had to finish manually - luckily this is was in the post-installation steps so we can assume that the installation/upgrade was done:
ERROR:System.Exception: Could not find a part of the path '...\sitecore\shell\Applications\Social\Wizards'.
ERROR:An error occured while copying files. Some files might not have been updated.
You must manually run the following batch file to complete the installation ...\temp\__Upgrade\Upgrade_20171121T163720573\process.bat
Details: [s]Sitecore.Update.Installer.Exceptions.PostStepInstallerException: Error has occurred during file installation. 
at Sitecore.Update.Installer.Items.PostStepInstaller.Process(IProcessingContext entry, IProcessingContext context)

We examined the other logs and it appears that this path in the Social folder was already deleted during the installation so why delete it again? Anyway, we ran the batch file and proceeded.


Our site includes WFFM so we have to update this as well. This is done after the installation of the upgrade package and before the installation of xConnect. We did make a small mistake though.. The upgrade guide does mention updating your Solr setup prior to updating any modules. But as we didn't have any solr to start with, an assumption was made (or we just weren"t thinking) that we could upgrade wffm and still keep our search provider to Lucene - this is still supported on a standalone instance without xDB so that should work...

But we got: 
System.AggregateException: One or more errors occurred. Sitecore.Update.Installer.Exceptions.CriticalInstallationException: Critical installation exception occurred. System.AggregateException: One or more exceptions occurred while processing the subscribers to the 'packageinstall:items:ended' event

We do have a custom index though.. and the syntax for Lucene indexes changed slightly (so we learned here). This caused our upgrade process to fail miserably. After changing the search provider to Solr (including our custom index configuration) the upgrade worked.
Tip: fix your (custom) indexes when the upgrade guide tells you to "update" Solr


When switching to Solr, do test your queries. This seems obvious of course. But we noticed that Solr does behave slightly different in some cases compared to Lucene. This is not really an upgrade issue but as more people will be switching to Solr when upgrading to Sitecore 9 this might be worth mentioning. One issue we had was a query that retrieves lots of entries.. worked fine with Lucene and terribly slow with Solr - getting them in batches resolved this.

Installing xConnect

xConnect is new, so it has to be installed rather than upgraded. We had done this before so that would be a piece of cake. The prerequisites on the server(s) were ok and all was set to run SIF (the Sitecore Install Framework). All went fine and the webdeploy was running and.. boom.  The database deployment went wrong because the SQL user we had defined in the json configuration already existed.

Hmz.. indeed. One of the steps during the upgrade is deploying new databases (ExperienceForms and Processing.tasks). And I assumed (yes, assumptions.. it was not in the document) I had to add the SQL user to those databases. Which probably is indeed the case. I first thought that would be the issue, but it's not. That still was a good idea. But apparently we also had to remove the user from the Marketingautomation, ReferenceData and Processing.pools databases.

Tip: make sure your SQL user is not attached to the 3 mentioned databases before installing xConnect

Yes, we are good to go! The webdeploy managed to finish this time.

Well, almost..  this time because I did something really silly (just before holidays people do silly things) trying to install xConnect in my "current" folder. This will cause the installer to fail as it can't access it's own log files anymore as they are "in use". So:
Tip: do not install xConnect in your "current" folder

Starting windows services

We also had an issue starting the windows services installed with xConnect. This means we had to re-run the installation with a custom json, just to perform the tasks after starting the services. The reason the services didn't start are probably related to our environment but I'll mention them in case it helps someone: the user "local service" which is running the services had no access rights to the folders where the jobs are located (..\App_data\jobs\continuous\..). Just add those rights and test by starting the services manually.

After care

We have a running site now. Time to take a look at the logs ;)

Issue 1 : Sql Exceptions

Exception: System.Data.SqlClient.SqlException
Message: Could not obtain information about Windows NT group/user 'DOMAIN\...', error code 0x5.
Source: .Net SqlClient Data Provider
We restored our databases on a SQL 2016 instance before the upgrade so we already matched the required version. After asking Sitecore, this is not necessary - you can upgrade and switch afterwards as well. The issue is not that we switched before, but we restored the databases on the sql server with our windows accounts and that was not ok. We had to change the db owner to a dedicated sql user to fix this.

Issue 2 : Path analyzer errors

Path analyzer errors. Quite a few of those in the logs...  We checked the /sitecore/admin/PathAnalyzer.aspx page and noticed indeed that we had issues - some maps did not get deployed. We checked those in Sitecore, and noticed that our marketer friends for some reason deleted some standard goals. One of them was "Login" and this was now required. Packaging the missing goals from another instance, installing them and deploying the maps made the issues disappear. 

Remember that the upgrade process starts with running scripts that restore deleted Marketing Taxonomies and Marketing Definitions but that does not include any missing goals..

Publishing target

For our extra publishing target we had to add some (new) configuration as well.

xDB Data Migration

This one is still under investigation... we succesfully ran the migration tool but are facing some aggregation errors in the logs now. And seem to be missing some data...  Will update if applicable ;)

Next steps

  • Rewriting our custom facet code, ...  might be another post.
  • Try this all over again when update-1 is released :)
  • Try this all over again on another project with all our lessons learned...

Wednesday, October 18, 2017

Sitecore 9 Forms : translating error messages

Sitecore 9 Experience Forms

Sitecore released a successor for Web Forms for Marketers (WFFM) with the new Sitecore 9 version, called Sitecore Forms.

In a multilingual environment you want to translate your forms, which is easy as the form editor itself has a language switch.  But what about the error messages - those things a visitor gets when (s)he fills in the form not as expected...

Translating error messages

Find the validators in the Sitecore Content Editor: /sitecore/system/Settings/Forms/Validations

Each validator has a Message field. Create versions of the validator item in all the languages needed and fill (or update) the message field as desired. Just check that the message is valid: the parameters in the text need to match the original value. As in the example (image) for email there is "{0}". A developer will recognize this syntax - all translations do need this token as it will be replaced by the field name when displaying the error.

Client-side messages

The messages entered here will be used for client- and server-side error messages. Which is nice. Just be aware that somewhere also jquery validation gets used and that can lead to strange situations. I noticed that the email validation can give two different error messages depending on what the error is.. and one of them is not translated as it comes from jquery. 

Translating the jquery validation is out of scope here as that is documented elsewhere and is not related to the Sitecore solution. 

Required field

You might notice that the most used validation - required field - is not in the list. That is because this is a special case.. also on a Forms field, you would not select a required validator, but just tick the required checkbox. 

The code for the required field validation works in a different way and the message will come from the Sitecore dictionary. If you want to update or translate it, you need to add a key to that Dictionary. This can be done in the master database - just add an item in the /system/Dictionary - the name doesn't really matter. The key however needs to be exactly "{0} is required.". The phrase can be anything you want, just remember to add the {0} token. 

Create versions and translate in all languages needed - and don't forget to publish.

Sitecore 9 Forms: custom submit action

Sitecore 9 Experience Forms

Sitecore released a successor for Web Forms for Marketers (WFFM) with the new Sitecore 9 version, called Sitecore Forms.

Submit actions - save actions

Sitecore Forms comes with a few submit actions (or save actions as we used to call them, or how they are still called on the submit button) out of the box like Trigger Goal, Redirect to Page, Save Data ... but is also missing a few and developers will probably -just as with wffm- need to make custom ones.

Custom redirect

I wrote a post on Sitecore 8 about customizing the redirect url. In Sitecore 9 with Forms, that article is no longer valid as we should use a different approach and use a custom submit action. So lets try to do that: create a submit action that redirects to a page and adds the value of a field (e.g. email) to the querystring to be processed on the results page.
Could become useful as there is no ootb option for a success message anymore.

The code

Let's go straight to the good stuff. We'll create a class derived from SubmitActionBase<>  and as we want to create redirect functionality we can reuse the ActionData class from the existing redirect so it becomes derived from SubmitActionBase<RedirectActionData>.

public class CustomRedirectAction : SubmitActionBase<RedirectActionData>
  public CustomRedirectAction(ISubmitActionData submitActionData) : base(submitActionData)

  protected override bool Execute(RedirectActionData data, FormSubmitContext formSubmitContext)
    Assert.ArgumentNotNull(formSubmitContext, "formSubmitContext");
    if (data == null || !(data.ReferenceId != Guid.Empty))
        return false;
    var item = Sitecore.Context.Database.GetItem(new ID(data.ReferenceId));
    if (item == null)
        return false;

    var email = string.Empty;
    var postedFormData = formSubmitContext.PostedFormData;
    var field = postedFormData.Fields.FirstOrDefault(f => f.Name.Equals("Email"));
    if (field != null)
        var property = field.GetType().GetProperty("Value");
        var postedEmail = property.GetValue(field);
        email= postedEmail.ToStringOrEmpty();
    var defaultUrlOptions = LinkManager.GetDefaultUrlOptions();
    defaultUrlOptions.SiteResolving = Settings.Rendering.SiteResolving;
    formSubmitContext.RedirectUrl = new UrlString(LinkManager.GetItemUrl(item, defaultUrlOptions)) + "?email=" + email;
    formSubmitContext.RedirectOnSuccess = true;
    return true;

We start by fetching the item to redirect to - which is found in the reference id. If this is not available, we get out.

Fetching data from the submit context

Next thing to do is get the value of the email field. That was a bit tricky and I'm not sure whether I used the best approach here - I did get this idea from the Save Data code so it won't be all bad :).

The good news is that we can use the item names (of the "field" item of the form) so we shouldn't have issues with translations. And it will work for other forms if they keep the name identical (the name is not the label shown on the form by the way). Finding the field is easy as they all are a property of the posted form data available in the form submit context. But getting the data from that field was not so trivial - but the "property" stuff works.


As we have all our data now, we van trigger the redirect. Based on the redirect item we can construct the base url - and then add the email to it in the querystring. 


Once the final url is known, we can set it in the form submit context. We also set the redirect property to true and abort the context. Aborting the context will prevent other actions to be executed. Our redirect should be the last.

Configuring Sitecore Forms

Once we have our code, we need to configure Sitecore Forms to use it. The configuration is done in Sitecore.

Open a content editor and go to /sitecore/system/Settings/Forms/Submit Actions. Create a new item of type Submit Action. Fill in the Model Type (e.g. Forms.CustomRedirectAction) and an error message. 


The editor field will define how your editor can configure your action. In this case we want them to be able to select an item from the content tree. We could use the existing one from the ootb redirect action, but lets create a new one..

Move to the core database and locate /sitecore/client/Applications/FormsBuilder/Components/Layouts/Actions. You will see the existing editor options there (including the RedirectToPage which has a wrong and confusing display name -Trigger Goal- at the time of writing).  As these are all editor actions that will open a content tree at a certain location, the easiest way to accomplish our needs is copy an existing tree and update the items needed - which is actually just the ItemTreeView. The StaticData field defines the root item available for your editors. You might have an issue getting the correct data in here, as the droptree on this field will show you the core database. And you want data from the master database.. switching to raw values and entering the desired guid will do the trick!

Once this is set, we can go back to the master database and finalize our submit action by selecting our newly created editor option.

That's all, your submit action should be available on your forms in the "add" list of any submit button. 

And don't forget to publish ;) 

Sitecore 9 Forms : custom validation

Sitecore 9 Experience Forms

Sitecore released a successor for Web Forms for Marketers (WFFM) with the new Sitecore 9 version, called Sitecore Forms.


Sitecore Forms comes with a few validators out of the box like email, string length, ... and of course the required field but that one is kinda special.

What if you want to validate fields with your own custom business rules?  
Lets see how we can create a validator that checks the length of the string input (based on parameters) and combines that with a regular expression validation that all characters are numeric.

Custom validator

We start by creating a new class and derive from RegularExpressionValidation. This will give us some functionality concerning the regular expression validation.

We define our class properties that we'll need to setup the validation:
public override string RegularExpression => "^[0-9]+$";
private int MinLength { get; set; }
private int MaxLength { get; set; }
Three functions need to be overridden:
  • Initialize: initializes the validator based on the validationModel object
  • Validate : server side validation based on the value
  • GetClientValidationRules : registers the client side validation rules


The resulting code will look like this:
public class CustomValidation : RegularExpressionValidation
  public override string RegularExpression => "^[0-9]+$";

  private int MinLength { get; set; }
  private int MaxLength { get; set; }

  public CustomValidation(ValidationDataModel validationItem) : base(validationItem)

  public override void Initialize(object validationModel)
    var stringInputViewModel = validationModel as StringInputViewModel;
    if (stringInputViewModel == null) return;
    MinLength = stringInputViewModel.MinLength;
    MaxLength = stringInputViewModel.MaxLength;
    Title = stringInputViewModel.Title;

  public override ValidationResult Validate(object value)
    if (value == null) return ValidationResult.Success;
    var num = value.ToString().Length;
    if (num > 0 && (num < MinLength || num > MaxLength))
      return new ValidationResult(FormatMessage(Title));

    var regex = new Regex(RegularExpression, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture | RegexOptions.Compiled);
    var input = (string)value;
    if (string.IsNullOrEmpty(input) || regex.IsMatch(input)) return ValidationResult.Success;
    return new ValidationResult(FormatMessage(Title));

  public override IEnumerable<ModelClientValidationRule> GetClientValidationRules()
    if (MaxLength > 0)
        yield return new ModelClientValidationStringLengthRule(FormatMessage(Title), MinLength, MaxLength);

    yield return new ModelClientValidationRegexRule(FormatMessage(Title), new Regex(RegularExpression, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture | RegexOptions.Compiled).ToString());

We compile the code and publish the dll to the website, where we will add the validator to the configuration in Sitecore.

Sitecore Forms configuration

Open the Content Editor and go to the Forms section in Settings: /sitecore/system/Settings/Forms/Validations. Create a new item of type Validation (or copy an existing and update the fields).

You need to fill in the type (e.g. "Forms.CustomValidation,Forms") and a message. The message has to contain "{0}" as this will be replaced with the title of the field in the code - so e.g. {0} is not a valid entry for this field.
Also, do not forget to create a version in every language of your site and translate the message.

Now we need to attach the validator to the field types in order to make it visible when selecting a validator. Go to the Field Types section and let's take the single line text field: /sitecore/system/Settings/Forms/Field Types/Basic/Single-Line Text. Find the Allowed Validations field and your custom validator to the selected ones.

That's it..  if you now go to a form and select a single-line text field, you will be able to select your newly created validator.

ps: don't forget to publish ;)