Wednesday, February 7, 2018

Displaying extra data on a Sitecore 9 Goal

Goals in Sitecore 9

In our project we had goals set when a whitepaper was downloaded. This works fine, but it would be much nicer if we could show which whitepaper was actually downloaded. All the whitepapers have an identifier code, so the challenge was to add this identifier to the goal data.

Capturing custom goal data

Capturing the data is very well explained in the official documentation of Sitecore. We used the "Data" and "DataKey" properties of the goal:
  • DataKey: a key that identifies the contents of  "Data" - e.g. 'Whitepaper code'
  • Data : any data collected as part of triggering the event - in our case the whitepaper identifier
This all works fine and we can see the data in the database (xDB shards) as part of the events.

Displaying custom goal data

Data is nice, but you need a way to show it. It wasn't immediatly clear how to do this, so I asked it on Sitecore Stack Exchange.
Jarmo Jarvi pointed me in the good direction by mentioning a tremendous blog post by Jonathan Robbins. The blog post was based on Sitecore 8 - the idea behind my post here is to show the differences when doing this in Sitecore 9 and xConnect. The result will be the same:
an extra column in the goals section of the activity tab of a contact in the Experience Profile

ExperienceProfileContactViews

The ExperienceProfileContactViews pipeline is where all the magic happens. Adding the extra column to the results table is identical to the description for Sitecore 8. Fetching the data (in GetGoals) however is quite different as we have to use xConnect now:
public class AddGoalDataColumn : ReportProcessorBase
{
  public override void Process(ReportProcessorArgs args)
  {
    args.ResultTableForView?.Columns.Add(Schema.GoalData.ToColumn());
  }
}

public static class Schema
{
  public static ViewField GoalData = new ViewField("GoalData");
}

public class FillGoalData : ReportProcessorBase
{
  public override void Process(ReportProcessorArgs args)
  {
    var resultTableForView = args.ResultTableForView;
    Assert.IsNotNull(resultTableForView, "Result table for {0} could not be found.", args.ReportParameters.ViewName);
    var i = 0;
    foreach (var row in resultTableForView.AsEnumerable())
    {
      var goalData = args.QueryResult.Rows[i].ItemArray[4];
      if (goalData != null)
      {
        row[Schema.GoalData.Name] = goalData;
      }

      i++;
    }
  }
}
public class GetGoals : ReportProcessorBase
{
  public override void Process(ReportProcessorArgs args)
  {
    var goalsDataXconnect = GetGoalsDataXconnect(args.ReportParameters.ContactId);
    args.QueryResult = goalsDataXconnect;
  }

  private static DataTable GetGoalsDataXconnect(Guid contactId)
  {
    var goalsTableWithSchema = CreateGoalsTableWithSchema();
    var contactExpandOptions = new ContactExpandOptions(Array.Empty<string>())
    {
      Interactions = new RelatedInteractionsExpandOptions("WebVisit")
      {
        StartDateTime = DateTime.MinValue
      }
    };

    FillRawTable(GetContactByOptions(contactId, contactExpandOptions).Interactions.Where(p => p.Events.OfType<Goal>().Any()), goalsTableWithSchema);
    return goalsTableWithSchema;
  }

  private static DataTable CreateGoalsTableWithSchema()
  {
    var dataTable = new DataTable();
    dataTable.Columns.AddRange(new[]
    {
      new DataColumn("_id", typeof(Guid)),
      new DataColumn("ContactId", typeof(Guid)),
      new DataColumn("Pages_PageEvents_PageEventDefinitionId", typeof(Guid)),
      new DataColumn("Pages_PageEvents_DateTime", typeof(DateTime)),
      new DataColumn("Pages_PageEvents_Data", typeof(string)),
      new DataColumn("Pages_Url_Path", typeof(string)),
      new DataColumn("Pages_Url_QueryString", typeof(string)),
      new DataColumn("Pages_PageEvents_Value", typeof(int)),
      new DataColumn("Pages_Item__id", typeof(Guid)),
      new DataColumn("SiteName", typeof(string))
    });

    return dataTable;
  }

  private static void FillRawTable(IEnumerable<Interaction> goalsInteractions, DataTable rawTable)
  {
    foreach (var goalsInteraction in goalsInteractions)
    {
      foreach (var goal in goalsInteraction.Events.OfType<Goal>())
      {
        var currentEvent = goal;
        var row = rawTable.NewRow();
        row["_id"] = goalsInteraction.Id;
        row["ContactId"] = goalsInteraction.Contact.Id;
        row["Pages_PageEvents_PageEventDefinitionId"] = currentEvent.DefinitionId;
        row["Pages_PageEvents_DateTime"] = currentEvent.Timestamp;
        row["Pages_PageEvents_Data"] = currentEvent.Data;
        row["Pages_PageEvents_Value"] = currentEvent.EngagementValue;
        var dataRow = row;
        const string index = "SiteName";
        var webVisit = goalsInteraction.WebVisit();
        var str = webVisit?.SiteName ?? string.Empty;
        dataRow[index] = str;
        if (currentEvent.ParentEventId.HasValue)
        {
          if (goalsInteraction.Events.FirstOrDefault(p => p.Id == currentEvent.ParentEventId.Value) is PageViewEvent pageViewEvent)
          {
            row["Pages_Item__id"] = pageViewEvent.ItemId;
            var urlString = new UrlString(pageViewEvent.Url);
            row["Pages_Url_Path"] = urlString.Path;
            row["Pages_Url_QueryString"] = urlString.Query;
          }
        }

        rawTable.Rows.Add(row);
      }
    }
  }

  private static Contact GetContactByOptions(Guid contactId, ExpandOptions options = null)
  {
    using (var client = SitecoreXConnectClientConfiguration.GetClient())
    {
      if (options == null)
      {
        options = new ContactExpandOptions(Array.Empty<string>())
        {
          Interactions = new RelatedInteractionsExpandOptions("IpInfo", "WebVisit")
        };
      }

      var contactReference = new ContactReference(contactId);
      var contact = client.Get(contactReference, options);
      if (contact == null)
      {
        throw new ContactNotFoundException($"No Contact with id [{contactId}] found");
      }

      return contact;
    }
  }
}

One more change is the configuration. It is almost the same as in Sitecore 8, but we don't need to change the goals-query anymore - instead we have to patch the GetGoals in the same ExperienceProfileContactViews pipeline.
<sitecore>
  <pipelines>
    <group groupName="ExperienceProfileContactViews">
      <pipelines>
        <goals>
          <processor patch:after="*[@type='Sitecore.Cintel.Reporting.Contact.Goal.Processors.ConstructGoalsDataTable, Sitecore.Cintel']"
             type="MyNamespace.AddGoalDataColumn, MyProject" />
          <processor patch:after="*[@type='Sitecore.Cintel.Reporting.Contact.Goal.Processors.PopulateGoalsWithXdbData, Sitecore.Cintel']"
             type="MyNamespace.FillGoalData, MyProject" />
          <processor patch:instead="*[@type='Sitecore.Cintel.Reporting.ReportingServerDatasource.Goals.GetGoals, Sitecore.Cintel']"
             type="MyNamespace.GetGoals, MyProject" />
        </goals>
      </pipelines>
    </group>
  </pipelines>
</sitecore>

Final step - core database

As Jonathan mentioned, we do need to add a new item in the core database to display our new column in the list of Goals. Create this item under the path /sitecore/client/Applications/ExperienceProfile/Contact/PageSettings/Tabs/Activity/Activity Subtabs/Goals/GoalsPanel/Goals and set a descriptive HeaderText and a DataField value that matches the schema name.

Your result should look like this:



I woud like to thank Jonathan Robbins for his original blog post and Jarmo Jarvi for his answer on SSE.

Monday, February 5, 2018

Display custom contact facets in Sitecore 9 Experience Profile

Custom contact facets

Creating custom facets is something that pops up quite often when customers start using Sitecore xDB and contact data. Sitecore 9 changed lots of things for developers working with xDB. The new xConnect layer is a wonderfull thing but when upgrading to Sitecore 9, the odds that you will need to rewrite some/all your code regarding xDB are not looking good.

Although.. you might/should see this as an opportunity to re-think what has been implemented before.

I want to focus on one particular part in this post. We had some custom facets implemented. Doing this with xConnect in Sitecore 9 is well documented on the official Sitecore documentation site.

Experience Profile

But when you have custom facets you usually also want to display this information in the Experience Profile. We used to do this based on some blog post from Adam Conn and/or Jonathan Robbins.
When upgrading to Sitecore 9, we had to made some small changes to this code. Especially (and obviously) the part where we actually do a query to fetch the data from xDB.

Contact View pipeline

As you can read in the previously mentioned blog posts the contact view pipeline is where alle the magic happens in 3 steps:
  1. Creating a data structure to hold the results of the query
  2. Executing the query
  3. Populating the data structure with the results of the query
The part where our changes are located can be found in the execution phase. Our processor will be in a pipeline referred to by a build-in Sitecore processor:
<group groupName="ExperienceProfileContactDataSourceQueries">
  <pipelines>
    <my-custom-query>
      <processor type="MyNamespace.MyDataProcessor, MyNamespace" />
    </my-custom-query>
  </pipelines>
</group>

<group groupName="ExperienceProfileContactViews">
<pipelines>
  <demo>
    ...
    <processor type="Sitecore.Cintel.Reporting.Processors.ExecuteReportingServerDatasourceQuery, Sitecore.Cintel">
      <param desc="queryName">my-custom-query</param>
    </processor>
    ...
  </demo>
</pipelines>
</group>
Let's focus on getting the data.


Query Pipeline Processors

The query pipeline processor is the part that executes the query - gets the data. It is (still) a processor based on ReportProcessorBase.

We will be using the model that was created with the custom facet. For the example we have:
  • custom facet: "CustomFacet"
  • property "Company" in this facet (type string)
The code for MyNamespace.MyDataProcessor:

public override void Process(ReportProcessorArgs args)
{
  var table = CreateTableWithSchema();
  GetTableFromContact(table, args.ReportParameters.ContactId);
  args.QueryResult = table;
}

private static DataTable CreateTableWithSchema()
{
  var dataTable = new DataTable() { Locale = CultureInfo.InvariantCulture };
  dataTable.Columns.AddRange(new[]
  {
    new DataColumn(XConnectFields.Contact.Id, typeof(Guid)),
    new DataColumn("Custom_Company", typeof(string))
  });

  return dataTable;
}

private static void GetTableFromContact(DataTable rawTable, Guid contactId)
{
  string[] facets = { CustomFacet.DefaultFacetKey };
  var contact = GetContact(contactId, facets);
  var row = rawTable.NewRow();
  row[XConnectFields.Contact.Id] = contactId;

  if (contact.Facets.TryGetValue(CustomFacet.DefaultFacetKey, out var customFacet))
  {
    row["Custom_Company"] = ((CustomFacet)customFacet)?.Company;
  }

  rawTable.Rows.Add(row);
}

private static Contact GetContact(Guid contactId, string[] facets)
{
  using (var client = SitecoreXConnectClientConfiguration.GetClient())
  {
    var contactReference = new ContactReference(contactId);
    var contact = facets == null || facets.Length == 0 ? client.Get(contactReference, new ContactExpandOptions(Array.Empty<string>())) : client.Get(contactReference, new ContactExpandOptions(facets));
    if (contact == null)
    {
      throw new ContactNotFoundException(FormattableString.Invariant($"No Contact with id [{contactId}] found"));
    }
   
  return contact;
  }
}

When comparing this code to what we had in Sitecore 8, we have a bit more code but it all seems to make sense. Especially when  you get familiar with coding with xConnect.
What happens here in short:
  1. Create a dataTable
  2. Fetch the contact from xConnect with the necessary facet(s)
  3. Read the data from the contact and add it to the dataTable
  4. Set the dataTable in the pipeline arguments

Special thanks to Nico Geeroms who did the actual coding and testing on this one.