Wednesday, September 9, 2020

Deployment SCWDP Packages Using Octopus

    There are few ways, how Sitecore is deployed to your hosting. If you are using Azure PaaS then for deployment you use Azure Pipelines. But, when you use on-premise hosting then there is very high probability that you use TeamCity + Octopus.

     Sitecore had introduced SCWDP packages with with Sitecore Azure Toolkit. Indeed, SCWDP packages are well-known WDP packages. It is standard format of files that is used by web deploy to publish your code to IIS. I like SCWDP tools in Sitecore Azure Toolkit. It is step forward to standardization. You are able pack your code and deploy to website. Also, you have backward compatibility. It is possible to convert regular Sitecore packages to .scwdp files with one command line. And you can easily deploy these packages to Azure PaaS. Many thank to Bart Verdonck for list of articles, how to create and use .scwdp packages in Azure.

    But when you will try to use these .scwdp packages with Octopus, you will be unpleasantly surprised. Builtin Octopus step templates don't allow you to deploy these files. Of course you can deploy .scwdp file using powershell+msdeploy, but you need to repeat it for each your project and each package. That is why I decided to wrap this powershell script into step template that could be reused in many projects. 

    Octopus has quite friendly way to contribute step templates. You need to fork OctopusDeploy/Library, create step template by exporting your step from Octopus and send pull request. After review and approve of pull request, it becomes available in list of community templates. Now it can be easily reused in different Sitecore deployment on different Octopus servers.


P.S. I know that Docker is future, and Sitecore is moving in that direction to provide unified deployments to different cloud platforms and on-premise hosting. But, I hope that for companies, who are on 9 version, this Octopus step template will save their time.

Saturday, May 2, 2020

Execute GDPR "Right to be forgotten" on your custom data in Sitecore 9.3

This article is inspired by Rob Habraken presentation during Global Virtual SUGCON 2020. Rob described how to automate anonymizing based on Sitecore Marketing Automation actions. Also, he described how it could be applied to data in Sitecore Forms.

From my side, I recently worked on cleaning up custom data in Sitecore to execute "right to be forgotten" and be fully GDPR compliant. I want to present general way, how to implement it in Sitecore for any custom data.

Sitecore 9.3 is GDPR compliant. And you should not worry, how it saves contact data. But you should know that all contact data could be anonymized by next action:

  1. Open Experience Profile
  2. Find contact that you are interested in
  3. Click on actions menu
  4. Select Anonymize action

This sequence of action, will execute right to be forgotten.

It works until you need to save some additional data in some separate locations(CRM, data storage, etc.). Once you need to have right to be forgotten for your custom data, you need to implement it by yourself. You are not able to rely on Sitecore, because Sitecore has no idea how you implemented custom saving of data. That is why you need to subscribe to Sitecore xConnect events to clean up your data in a same time, when it is done by Sitecore.

Sitecore xConnect has plugin architecture and you are able to register your plugin. It is very convenient, your configuration and your code lives separately from xConnect, but interacts via events. It makes your code very maintainable and you can easily upgrade it to next version of Sitecore.

To join your custom data with Sitecore - you have one requirement, you need to save your custom data with contact identifier. Having contact identifier provides you ability to find records related to contact, for which you need to execute right to be forgotten.

To run your code during right to be forgotten, you need to register your plugin in Sitecore xConnect:
<Settings>
  <Sitecore>
    <XConnect>
      <Collection>
        <Services>
          <EraseFormSubmissionWhenExecutingRightToBeForgotten>
            <Type>Sitecore.Weareyou.Feature.FormsExtension.XConnectServicePlugin.EraseFormSubmissionWhenExecutingRightToBeForgotten, Sitecore.Weareyou.Feature.FormsExtension</Type>
            <As>Sitecore.XConnect.Service.Plugins.IXConnectServicePlugin, Sitecore.XConnect.Service.Plugins</As>
            <LifeTime>Singleton</LifeTime>
          </EraseFormSubmissionWhenExecutingRightToBeForgotten>
        </Services>
      </Collection>
    </XConnect>
  </Sitecore>
</Settings>

Place this file under <xConnect root folder>\\App_Data\Config\Sitecore\Collection\.

Then you need to implement your plugin:


using System;
using Microsoft.Extensions.Logging;
using Sitecore.Framework.Conditions;
using Sitecore.XConnect;
using Sitecore.XConnect.Operations;
using Sitecore.XConnect.Service.Plugins;

namespace Sitecore.Weareyou.Feature.FormsExtension.XConnectServicePlugin
{
    /// <summary>
    /// Implementation was inspired by Sitecore.EmailCampaign.XConnect.Operations.ClearSupressionListWhenExecutingRightToBeForgotten
    /// XConnect service plugin that clears forms data on executing right to be forgotten
    /// Makes Sitecore Forms to be GDPR compliant
    /// e.g. Sitecore > xProfile > Any contact > Action > Anonymize
    ///
    /// To enable it you need to copy XML configuration to sc93xconnect.dev.local\App_Data\Config\Sitecore\Collection
    /// Configuration is present in this repository Project\xConnect\code\App_data\config\sitecore\Collection\sc.XConnect.Service.Plugins.FormsAnonymize.xml
    /// </summary>
    public class EraseFormSubmissionWhenExecutingRightToBeForgotten : IXConnectServicePlugin, IDisposable
    {
        private const string PluginName = "EraseFormSubmissionWhenExecutingRightToBeForgotten";
        private XdbContextConfiguration _configuration;
        private readonly ILogger _logger;
        
        public EraseFormSubmissionWhenExecutingRightToBeForgotten(ILogger<EraseFormSubmissionWhenExecutingRightToBeForgotten> logger)
        {
        _logger = logger;
        }
        public void Dispose()
        {
            GC.SuppressFinalize(this);
        }

        /// <summary>
        /// Subscribes  to events
        /// </summary>
        /// <param name="config">XdbContextConfiguration</param>
        public void Register(XdbContextConfiguration config)
        {
            _configuration = Condition.Requires(config, "config").IsNotNull().Value;
            _configuration.OperationAdded += OnOperationAdded;
        }

        private void OnOperationAdded(object sender, XdbOperationEventArgs xdbOperationEventArgs)
        {
            Condition.Requires(xdbOperationEventArgs, "xdbOperationEventArgs").IsNotNull();
            RightToBeForgottenOperation rightToBeForgottenOperation;
            if ((rightToBeForgottenOperation = (xdbOperationEventArgs.Operation as RightToBeForgottenOperation)) != null)
            {
                if (rightToBeForgottenOperation.Status == XdbOperationStatus.Canceled)
                {
                    _logger.LogDebug($"[{PluginName}] Skipping operation as it has been cancelled");
                }
                else
                {
                    rightToBeForgottenOperation.DependencyAdded += OnDependencyAddedToRightToBeForgottenOperation;
                }
            }
        }

        private void OnDependencyAddedToRightToBeForgottenOperation(object sender, DependencyAddedEventArgs args)
        {
            Condition.Requires(args, "args").IsNotNull();
            GetEntityOperation<Contact> getEntityOperation;
            if ((getEntityOperation = (args.Predecessor as GetEntityOperation<Contact>)) != null)
            {
                getEntityOperation.StatusChanged += OnGetEntityOperationStatusChanged;
            }
        }

        private void OnGetEntityOperationStatusChanged(object sender, StatusChangedEventArgs args)
        {
            Condition.Requires(sender, "sender").IsNotNull();
            Condition.Requires(args, "args").IsNotNull();
            GetEntityOperation<Contact> getEntityOperation;
            if ((getEntityOperation = (sender as GetEntityOperation<Contact>)) != null && args.NewStatus == XdbOperationStatus.Succeeded)
            {
                var entity = getEntityOperation.Entity;
                var contactId = entity.Id;
                if (contactId.HasValue)
                {
                    var identifier = entity.Identifiers
                        .FirstOrDefault(i => i.Source.Equals("xDB.Tracker"));
                    if(identifier != null) { 
                        if (Guid.TryParse(identifier?.Identifier, out var id))
                        {
                           //-------------------
                           //HERE SHOULD BE CODE THAT CLEAN UP CONTACT DATA IN YOUR CUSTOM LOCATION
                           //-------------------
                        }
                    }
                }
            }
        }

        public void Unregister()
        {
            _configuration.OperationAdded -= OnOperationAdded;
        }
    }
}

What this plugin does: it subscribes on xConnect events and run your code once right to be forgotten event was triggered.


Let's sum up, how to continue to be GDPR complaint if you need to save some custom user data:
  1. If it is possible, you need to save user contact data in Sitecore to have anything in one place (e.g. contact facets)
  2. If for some reason, you need separate storage for contacts data then you need to save these data with contact identifier. It allows you to find these data later.
  3. You need to implement and register xConnect plugin. It should clear your data, when right to be forgotten was triggered in Sitecore

You can see this approach in an action in my GitHub repository forked from Rob Haraken repository that he used for demo on Sitecore Virtual SUGCON 2020.

Tuesday, February 25, 2020

Solr Proxy Console: Another Tool for Your Sitecore Tool Belt

Have you ever been faced with a challenge when Sitecore search doesn’t work on your server? If the answer to this is yes then you’re in the right place… the tool I’m exploring below will be sure to make your development life easier!

Let’s imagine that something related to Solr search doesn’t work on your server... To find out what is wrong, you would need to use Solr console, your next steps would then be:


  1. Connect to remote desktop on the server, where Sitecore site is hosted
  2. Find ContentSearch.Solr.ServiceBaseAddress Sitecore settings value
  3. Open a browser on a remote desktop session and navigate to Solr URL


This process has some inconveniences: 

There is a possibility that you will not have access to the remote server, depending on your company or client policy
If it is your internal company test server, where are hosted many sites, you and your colleagues will “fight” for sessions that are allowed on the server, kicking off each other
Working on a remote desktop has some lag which means it's not as convenient as using your local machine

Last year I probably opened Solr console through a remote server more than 100 times; that’s why I decided to improve this process! Sitecore websites can play the role of the proxy server and transfer requests to real Solr admin panel. So, I have created a Sitecore module.

Here’s how to use it:


  1. Download Sitecore update package from AppVeyor
  2. Install update package using update installation wizard /sitecore/admin/UpdateInstallationWizard.aspx
  3. Open URL of Solr proxy https://yourwebsite/solr

Optional steps, if you want to use HTTP handler instead of injecting into HttpRequestBegin pipeline:


  1. Open Web.config file and add handler configuration>system.webServer>handlers:
    <add verb="*" path="solr/*" type="Foundation.SorlProxy.SolrHandler, Foundation.SorlProxy" name ="SolrHandler" />
  2. Add "/solr" to IgnoreUrlPrefixes Sitecore setting (Sitecore.config Sitecore>Settings>Setting[name="IgnoreUrlPrefixes"])
  3. Disable Foundation.Solr.Proxy.config configuration file

If you want to know more about how this module works or have any issues, look on GitHub repository, where all sources are located.

Saturday, February 22, 2020

Sitecore and Google Lighthouse Integration

Google Lighthouse became almost industry standard tool for measuring performance, accessibility, progressive web apps and SEO. There are few reasons, why it had happened. First of all, it is integrated with most popular browser: Google Chrome. Everyone is able to press F12(open DevTools) click “Generate report” button and get results. You should not be an expert, Lighthouse will tell you if your images are not optimized, if your scripts running too long, if you have wrong page structure that have bad influence on SEO. Second reason of Lighthouse popularity is that Google is number one web search engine. And as it is major source of traffic for many websites, it has sense to optimize your pages based on Google recommendations, which are can easily get from Lighthouse.

Pages performance, accessibility and SEO are very important for any website built on Sitecore. That is why I decided to integrate Lighthouse with Sitecore. It will allow you to run and access reports directly from Sitecore interface, have historical information and historical charts. As for me, it could be good checker for your day to day work with Sitecore. You are doing some improvement, you can easily understand how this improvement influence on particular pages and whole website. Or you are doing some change that influences on many pages, you can easily find out if it doesn’t break anything related to performance or accessibility.

I am glad to introduce: Sitecore.Lighthouse. Sitecore module that provides ability you to get everything from Lighthouse directly from Sitecore interface. On 22 February of 2020 it was tested only on Habitat Home and other few sites. That is why any contribution or bug reports are welcome. 



Saturday, August 10, 2019

Local Docker Registry for Sitecore Images

    There are a lot of articles how to run Sitecore on Docker. But no one explain, how it is possible to do without paid account on Docker hub(or alternative).
    If for some reasons, you don't want to use Docker Hub(or alternative) for hosting your Sitecore images and want to host them locally, this article is for you. Here are steps that you need to achieve it:
  1. Run registry-windows container. It will be your local "Docker Hub".
    docker run -d -p 5000:5000 --restart=always --name registry -v D:\registry:C:\registry stefanscherer/registry-windows:2.6.2 

    Where:
    5000:5000  is mapping your local port to the port inside container

    --name registry is setting name to "registry" of your container.

    stefanscherer/registry-windows:2.6.2 is image name(version tag is optional). As you will run Sitecore containers in Windows mode then it makes sense to run registry which is Windows container.

     D:\registry:C:\registry  is mapping your local "D:\registry" folder to folder "C:\registry" inside container. It is required to save all your work if something go wrong. All your images in your local registry will be saved on your drive, not inside the registry container. 
  2. Verify that your registry container is up and running. You can check it by opening URL: http://localhost:5000/v2/_catalog. Expected result: Empty JSON response. 
  3. Modify your build.ps1 that will be used to build Sitecore images. You need to add -Registry "localhost:5000" parameter to make sure that your images will be pushed to your local repository.
  4. Run building and pushing Sitecore images (build.ps1).  It will take sometime.
  5. Verify that you get images in your local repository. Open URL: http://localhost:5000/v2/_catalog. It should return something like this, depending on your build.ps1:
  6. Configure docker to use local registry:
  7. Now you are ready to download Sitecore images from local registry and run them in containers.

Friday, July 19, 2019

Small Remark about Sitecore Agents

It is possible to create Sitecore agents that could be configured via parameters from configuration files. E.g.:

  <agent type="YourNamespace.SomeAgent" method="Run" interval="00:05:00">
    <param desc="database">master</param>
    <param desc="website">shell</param>
  </agent>

and you have constructor to initialize agent: SomeAgent(string database, string website).

Desc attribute in configuration is used only for description of parameter. And if you occasionally change order in configuration file, you will get different order in your constructor. Database will become website. Website will become database.

So, here is reminder: order of parameters in configuration of Sitecore agents matters. Desc attribute is used only for information and nothing else.