In my previous article, I shared a comprehensive troubleshooting guide for Sitecore XM Cloud tracking and personalisation. The guide addresses common issues, explains investigation steps and provides solutions for these issues.

However, understanding how the personalisation engine works in depth can further help in diagnosing persistent issues and developing personalised websites. This article visualises what happens behind the scenes when you enable personalisation and tracking in your Sitecore XM Cloud applications.

Overview of the personalisation workflow

Before we start, let's familiarise ourselves with key elements of the diagram we are going to look at.

1. On the left hand site we can see the Browser, it is responsible for sending requests to our application and displaying the result to end users.

2. The JSS app sits at the top centre of the diagram and it represents the Rendering Host role in Sitecore Headless topology. It is the application that processes incoming requests and handles the presentation layer. In this case it is a Next.js application based on the XM Cloud foundation template.

3. Edge / XM API is shown on the right hand side, it is a GraphQL endpoint that returns layout definition and content for requested pages, including personalised variants. It can be Experience Edge API for cloud-based setups or a local CM container API endpoint for development purposes.

4. Finally, at the bottom we can see Tracking & Interactive API powered by the embedded instance of Sitecore CDP & Personalize. This is where audiences are stored, conditions are executed and analytics is collected.

When we talk about Next.js applications, there are two main rendering methods: Server-Side Rendering (SSR) and Static Site Generation (SSG). Sitecore XM Cloud personalisation engine works slightly differently with these two approaches so we will cover both of them to understand their specifics.

Server-Side Rendering

This is what happens when a website visitor opens a page that has personalised variants.

Step 1. When a user loads the website or navigates to a new page, the browser sends an HTTP request to the Next.js application, including cookies and HTTP headers that can be used in personalisation conditions.

Step 2. The Next.js application runs all registered middleware modules, with the Personalize middleware being of particular interest. This middleware sends an API request to the GraphQL endpoint to fetch all personalised variants for the current page configured in the CMS.

If there are no personalised variants configured for this page or the page is not found, this middleware will exit and page generation will continue as usual.

Step 3. If a page has more than one variant, the middleware sends another API request to the Personalize API to detect if the current visitor matches any of the audiences configured for this page. This is where cookies and HTTP headers received from the browser will help as they will be passed to the Personalize API to identify the visitor.

Step 4. By combining responses from these two API requests, the middleware determines which personalised page variant is suitable for the current visitor (cat-themed page in the diagram below 😺).

If the visitor matches an audience configured for the page, the middleware will rewrite the page path to a special personalised variant path (for example, from /_site_Test/Pets to /_variantId_0dd7b00680be49c6815ca4d0793a36da/_site_Test/Pets) and this will instruct the Next.js application to use the specific page variant when rendering the page. So the personalised version of the page will be rendered on the server and returned to the browser.

If the visitor does not match any audiences, then the default page variant will be rendered and returned to the user.

Step 5. Once the page is rendered in the browser, a special React component responsible for tracking will send an API request to the CDP Stream API to register the page view, including which personalised variant was shown. This data is later will be aggregated and shown in analytics reports.

Static Site Generation (SSG)

The SSG process flow is similar to SSR but has some specifics related to this rendering method. Now, let's see what are these differences.

Step 1. This step is exactly the same as for SSR - the browser sends an HTTP request to the Next.js application with cookies and HTTP headers.

Step 2. This is where things get different from the SSR process. When the Personalize middleware kicks in, it checks if there are any pre-rendered page variants for this page (the default, cat-themed 😺 and dog-themed 🐶 variants in the diagram). If yes, it skips the API request to the Edge / XM API, otherwise it will fall back to the standard SSR process and fetch personalised variants for the current page.

Step 3. This step is the same as in the SSR flow - if a page has personalised variants, the middleware sends an API request to the Personalize API to identify visitor's audience.

Step 4. If there is a match and personalised page variants are pre-generated, the middleware will rewrite the page path and then the appropriate personalised page variant will be chosen and returned to the browser (looks like it's the cat-themed variant again! 😺).

If there are no pre-generated personalised variants, but they exist in the CMS and the visitor matches one of the audiences, then the middleware will rewrite the page path, the Next.js app will generate the page variant and save the static output for future requests. This is the default process, see notes at the end of the article to learn more about static generation of personalised page variants.

If the visitor does not match any audiences, then the default page variant will be returned to the user using the statically generated HTML if it exists.

Step 5. As with SSR, once the page is returned to the browser the CdpPageView React component will send an API request to track the page view event for reporting.

As you can see, the flow is very similar for SSR and SSG. The SSG method with static HTML generation and skipping some API requests can give us a performance boost, especially for websites with high traffic and personalisation enabled on frequently visited pages.

Notes

Personalize middleware

The middleware is provided by Sitecore as a part of the JSS XM Cloud add-on for Next.js. Please note that this add-on is compatible with JSS version 21.6 and later. For earlier versions the Next.js Personalize add-on is used that is now obsolete.

This add-on is only compatible with Sitecore XM Cloud due to specific naming conventions and pre-configured settings required for the embedded CDP and Personalize instance.

Build-time static generation of personalised page variants

To expand on the step 4 of SSG process, let's see when exactly personalised page variants are generated. As you may know, hosting providers often limit the time available for SSG builds. By default, pre-generation of personalised page variants during build is disabled in to avoid long build times.

However, if sufficient build time is available (for example, your website does not have too many pages) or you have critical personalisation rules on key pages (for instance, you only have a small number of personalised variants on the homepage or an important campaign page), then SSG for personalised variants can be explicitly enabled.

This can be done by modifying the file src/lib/sitemap-fetcher/plugins/graphql-sitemap-service.ts and setting the includePersonalizedRoutes parameter to true in the sitemap service constructor:

this._graphqlSitemapService = new MultisiteGraphQLSitemapService({
    clientFactory,
    sites: [...new Set(siteResolver.sites.map((site: SiteInfo) => site.name))],
    includePersonalizedRoutes: true,
});

Just make sure to watch your build time after enabling this setting to avoid build failing or incurring unnecessary hosting costs.

Sitecore XM Cloud provides robust support for personalisation out-of-the-box for both SSR and SSG rendering methods. The add-on with Personalize middleware and CDP tracking component streamlines the process of fetching, matching, and delivering personalised content to website visitors while tracking interactions for reporting.

Hope this article helps to understand the entire process of personalisation and tracking in XM Cloud and allows you to build well-performing and personalised applications. Feel free to share your thoughts and questions in the comments!