Extension of Sitecore Log Abilities

    Sitecore have an ample capabilities on logging. All logging system is based on log4net services. And if you need to change destination of your logs all that you need – configure right log appender. There is wide range of log4net appenders available out of the box. And also big amount of open source modules that you are able to fit for your needs. For example:

1. You can write your logs in MongoDB. 
2. You can create Slack bot that will put logs in Slack channel. 
3. Use ElasticSearch+LogStash+Kibana(or RabbitMQ) 

     

    But there are few peculiar properties that doesn’t allow to use major appenders out of the box. Log4net web.config configuration section in Sitecore is initialized by Sitecore.Logging (not by log4net.dll) assembly and it requres additional steps. Sitecore.Logging is extended copy of log4net assembly. Even namespaces are same! While fitting new appender to Sitecore your appender should be inherited from log4net.Appender.AppenderSkeleton. Take into account that class with same namespace is present in both log4net.dll and Sitecore.Logging.dll. You need to use class from Sitecore.Logging.dll !!!

To summarize, list of steps of fitting new appender to Sitecore will be next:

1. Get sources of appender 
2. Add reference to Sitecore.Logging assembly with alias Sitecore for ability to use log4net and Sitecore.Logging in same project 
3. Inherit your appender from Sitecore.Logging AppenderSkeleton class implementation MyAppender : Sitecore::log4net.Appender.AppenderSkeleton 
4. Implement missed class members: you need to reimplement Append method. It has different signature comparing to default one Append(Sitecore::log4net.spi.LoggingEvent loggingEvent) 

Sitecore ExM in Depth: How Sitecore Email Experience Manager Determine If Message Was Opened

When you look at source of html email message that was delivered by Sitecore Email Experience Manager you see strange image tag that is located before closing BODY and HTML tags:

<img height="1" width="1" border="0" style="border-color: transparent;" src="http://your.host/sitecore/RegisterEmailOpened.aspx?ec_contact_id=3D0A4C096C4287C4C50A33E8BB5DB87E04&ec_message_id=3D16BE08364227=4836B57FB8CDCEB15C85" />

It is service ExM tag that is inserted to all email. This image tag has source property link to service ExM page: RegisterEmailOpened.aspx. And as parameters are transferred IDs of contact and email. When your mail agent(or browser) tries to load this image it sends request to Sitecore server and triggers event about email opening. Of course it will not work for all mail clients and web mail portals, but it is still good way to get more information about behavior of email recipients. (One of reasons could be not downloading images by default. It is frequent setting for mobile mail agents.)

Sitecore ExM in Depth: Hard and Soft Bounce Detection Facts

There are few facts about soft and hard bounce detection that are not obvious and need your attention,when you try to get messages appeared in "Hard Bounce" or "Soft Bounce" engagement analytics states:

• ExM version is up to 3.2.1 bounce detection works only when messages delivery is performed via Sitecore AppCenter. Setting UseLocalMTA determine if Sitecore MTA(mail transfer agent) should be used. When setting UseLocalMTA is equal true and your  then you will not be able to get hard and soft bounce messages states.
• Starting from Sitecore Email Experience Manager 3.3 you are able to detect bounces with your local MTA. 
• Bounce detection is performed by scheduled task: /sitecore/system/Modules/E-mail Campaign Manager/Instance Tasks/Content Management Primary/Check Bounced Messages. Default interval between running of task is 12 hours 30 minutes. Be patient: contacts will not appear immediately in  hard and soft bounce states.
• Easiest way to check if latest bounce detection was success is to check Items field of Check Bounced Messages scheduled task. It should contain time stamp of latest success bounce detection. If  latest bounce detection was failed this field will contain "1/1/0001 12:00:00 AM" or empty string.
• Bounce detection is performed via request to Sitecore App Center service.

Sitecore "Schedule" Field in Depth

All Sitecore Schedule tasks have Schedule field that correspond for setting up interval of running tasks.

It is quite intuitive, especially when you will read description. It has format {0}|{1}|{2}|{3}.
Where:

1. Date stamp when task running should be started. 
2. Date stamp when task running should be ended.
3. Days of week when task should be run.
4. Time span interval how often task should be run.

All of components of this string have clear format, except third. Let's look  how it works. Under hood third component is cast to DayOfWeek enum.

public enum DaysOfWeek
{
 None = 0,
 Sunday = 1,
 Monday = 2,
 Tuesday = 4,
 Wednesday = 8,
 Thursday = 16,
 Friday = 32,
 Saturday = 64
}

And value that you should indicate days of week when schedule should be run is sum of this components. E.g.: task that should be run on Monday and Thursday. 2 +16 = 18. (indeed there is binary operation OR of 100 and 100000 values, but you can use decimal system for simplifying ).

It could take some time to get this value in head when you don't work with schedule field day to day. That is why I prepared script that will calculate this value for you:


P.S. Another good option is Sitecore Shell Wax shared source module. If Content Managers do changing of scheduled tasks day to day - it is better option to use this module.

Sitecore EXM: The remote name could not be resolved: 'default-cd-cluster'

While configuring sending EXM messages on Sitecore I faced with problem. Some emails were delivered, others not. Sitecore Email Experience Manager log reported next error:

ERROR Failed to enroll a contact in the engagement plan.
Exception: System.Net.WebException
Message: The remote name could not be resolved: 'default-cd-cluster'
Source: System
   at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
   at System.Net.WebClient.DownloadString(Uri address)
   at Sitecore.Modules.EmailCampaign.Core.Gateways.DefaultAnalyticsGateway.EnrollOrUpdateContact(Guid contactId, Guid planId, Guid automationStateId, EcmCustomValues customValues, String[] validStates)
   at Sitecore.Modules.EmailCampaign.Core.Analytics.AutomationStatesManager.EnrollOrUpdateContact(Guid contactId, Guid planId, String stateName, EcmCustomValues customValues, String[] validStates)
   at Sitecore.Modules.EmailCampaign.Core.Dispatch.DispatchManager.EnrollOrUpdateContact(Guid contactId, DispatchQueueItem dispatchQueueItem, Guid planId, String stateName, EcmCustomValues customValues)
   at Sitecore.Modules.EmailCampaign.Core.Dispatch.DispatchTask.OnSendToNextRecipient()

After search this error message I found two articles (first, second) that say the same: you should change Analytics.ClusterName setting. And this solution really helps. This setting is located in Sitecore.Analytics.Tracking.config file and EXM uses it to access to /sitecore/AutomationStates.ashx handler to change contact state in engagement plan.

But, doesn't sound strange? That you should to configure something to start work with Sitecore single instance. How about installation CM/CD/Analytics on one server? Or how about development and testing environments? I know that there are a lot of places where Sitecore as web application requires host name, but everywhere it works automatically. You shouldn't configure something additionally. That's why I decided to customize Sitecore to use 'default-cd-cluster' as key that will be replaced in EXM with current host name. And it was quite easy:

public class CustomAnalyticsGateway : DefaultAnalyticsGateway
{
    public override ContactLockingStatus TryGetContactForUpdate(ID contactId, 
            LeaseOwner leaseOwner, 
            System.TimeSpan leaseDuration, 
            System.TimeSpan timeout, 
            out Sitecore.Analytics.Tracking.Contact contact, 
            out string webClusterName)
    {
       var r = base.TryGetContactForUpdate(contactId, leaseOwner, leaseDuration, timeout, out contact,
           out webClusterName);
       if (webClusterName!= null && webClusterName.Contains("default-cd-cluster"))
       {
           webClusterName = System.Web.Hosting.HostingEnvironment.ApplicationHost.GetSiteName();
       }
       return r;
    }
}

If webClsterName contains 'default-cd-cluster' we change it with current sitename(it also will work if we will replace it with null due to Sitecore EXM later fill it with another value if it is null). And you should let EXM know to use this gateway in your configuration:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <gatewayFactory>
      <analyticsGateway type="YourSolution.CustomAnalyticsGateway, YourSolution"/>
    </gatewayFactory>
  </sitecore>
</configuration>

With this custom class and configuration Email Experience Manager started to work on all machines. And you should not change 'default-cd-cluster' with host name in all your solutions. But don't forget to change this setting when going to live environment.

Commerce Server Desktop Business Tools: Unsuccessful Attempt to Run on Mac OSX

I use .Net framework more than 10 years and long time ago wanted to try Mono. But, for web development it wasn't useful. Users see only frontend and does care what is under hood. Finally I have found task how I can try Mono: it is Commerce Server Desktop Business Tools.

Sitecore Commerce 8.1 powered by Commerce Server is shipped with two option of store management: Desktop Business Tools and Merchandising Manager. Merchandising Manager is SPEAK web application that allows to manage store from web application. Desktop Business Tools is set of Windows applications that also allows to manage store. Desktop Business Tools has much more features comparing Merchandising Manager. And if you have an option what to use I prefer Desktop Business Tools. Mac OSX users don't have such option(same for Linux users if someone uses Linux on desktop). But Desktop Business Tools are written majorly using .Net framework. What if we try to run them using Mono?

First of all you should forget about installer. You should install Desktop Business Tools on Windows machine and then copy all from C:\Program Files (x86)\Commerce Server 11 to your Mac. After copying and attempt to run 

sudo mono -v CatalogManager.exe

you will see that it requires additional assemblies, e.g. CommerceServer.Core.Catalog.dll. They could be added to Mogo GAC using gacutil (very similar as on Windows) or you should copy them from Commerce Server 11\Assemblies to Commerce Server 11\Business User Applications\bin. Trying to run again CatalogManager.exe via Mono again unsuccessful.  Commerce Server Business tools contains references to Windows unmanaged assemblies: msi.dll, kernel32.dll, user32.dll. You couldn't have them on Mac OSX.  But you could remove this references from code by changing Commerce Server Desktop Business Tools assemblies. Fortunately these reference are not related with business logic and you can replace with something that have similar logic or make stubs that return constants for some methods. Now we can see empty Catalog Manager window, but still get error: 

"System.ArgumentException: A null reference or invalid value was found [GDI+ status: InvalidParameter]

  at System.Drawing.GDIPlus.CheckStatus (Status status) <0x2e486d8 + 0x0016b> in <filename unknown>:0 "

It is error in Mono itself so I stopped here. Google says that it is probably Mono bug. But Mono is cross-platform. Let's check if we can run Commerce Server Desktop Business Tools on Windows and Linux using Mono.

On Windows it works badly: there are a lot of appearance bugs and wizards don't work at all.

However you don't need to run it on Windows by Mono, here you have .Net framework.

On Ubuntu it works in a same way as on Windows, but probably looks a little bit better:

Conclusion: I managed to run Commerce Server Desktop Business Tools with Mono on Windows and Ubuntu. But as it was not designed to work on this platform it is very buggy. You can use it for management already existed content(everything that doesn't require wizards). For Mac OS I faced with Mono error, but I think that it is possible to run tools on Mac, probably you should try different Mono versions. And you always are able to setup Desktop Business Tools on Windows server and use RDP to it to manage anything in your store. It is much more better way!

P.S. I skipped few actions in description to make it faster:

• Changing location of configuration files (there is no AppData folder on Mac or Linux)
• Disabling tracing (it doesn't work)

And I don't recommend to use Commerce Server Desktop Business Tools anywhere except Windows platform. One thing that addition could be checked: how it is running under Wine? But it task for next time.

Sitecore and IIS URL Rewrite module

    URL Rewrite is quite popular IIS module. It is SEO tool that allows you to improve site crawling by search engines. You can do a lot of things using it:

• Add or remove trailing slash
• Enforce lower case URLs
• Use canonical hostnames
• Redirect to HTTPS
• etc.

    But it should be used very carefully with big systems, where you are unable to control everything. As example I'll show which problems could cause thoughtless URL rewrite configuration with Sitecore. The example that is given below correspond only to configuration CM and CD on one machine, otherwise you are free to configure URL rewriting on CD, but don't do anything on CM.

    Simplest URL rewriting rule is "Remove Trailing Slash". You can add this rule by using predefined templates inside IIS embedded wizard.

  

    This rule redirect all requests from "http://yourawesomewebsite.com/page1/" to "http://yourawesomewebsite.com/page1". It helps to avoid of indexing one your page twice, wrong resolving it by analytics, etc. But URL Rewrite rules has one bad feature, while redirecting they transfer POST request to GET. Let see how it can breaks Sitecore functionality.

Sitecore 8 has List Manager Lanchpad  application. It uses MVC approach for communication client with server. And as good MVC practice controller action are marked with method attributes. Delete list, delete folder, move list and others are marked only with [HttpPost] attribute. They do not allow [HttpGet]. And unfortunately client side JavaScript code call this method with trailing slash. E.g.:

http://yourawesomewebsite.com/sitecore/api/ssc/ListManagement/Actions/%7B0264829E-6680-49EB-85B8-703EF9414BA5%7D/DeleteListById/

URL Rewrite module will remove trailing slash and translate this request from POST to GET. As result we will get error:

How this issue could be solved:

1. Do not use URL Rewrite module with complex website which work you could not fully control. Or use only with a part of site that you fully control. It is CD part for Sitecore.
2. Rewrite URL Rewrite rule. In our example it could be "^(((?!ListManagement).*)/$" instead of standard "(.*)/$"  
3. Use URL Rewrite module alternatives, as example ISAPI_Rewrite

Sitecore PageExtenders: non greedy approach

   There is great Sitecore feature: Page Extenders. It allows you to insert any HTML code to your pages. However inaccurate usage of it can harm your site.
   

    Once I installed Federated Experience Manager on Sitecore 7.2 I was unable to get it worked. For some reasons JavaScript on its pages failed. After deeper investigation I found that on any FxM request to server I got incorrect JSON. It was well formed, but at the end string

<input name="SC_ANALYTICS_PAGE" type="hidden" id="SC_ANALYTICS_PAGE" value="ecbb31a2-4c43-494d-88a1-811994213a89" /> 

was added. I figured out that this string was added by SBOS Accelerator module that was installed few days before. Reason of adding this string was greedy PageExеnder. It added this string everywhere, except few sites(shell, modules_shell). Such behavior also break some admin pages, e.g.: ShowConfig.aspx.

   After fixing this problem I can make conclusion: that in general it is much more better to specify where PageExtender should insert controls rather than specify where it should not insert controls. Even if you add a lot of exceptions, you could not cover by your exceptions everything that could appear in custom solutions. 

Sitecore Social Connected 2.1: Second Life

     After installation of Sitecore Social Connected 2.1 on Sitecore 7.2 following this reference (with few unusual steps for modules) I found out that I am not able to add Facebook and Twitter accounts. It was not a problem of this module. It was caused by changes in these social networks APIs. Trying to google solution I found it for Twitter. It was pretty easy, just replace one assembly with another.

   
    Error, that is appeared in log:
ERROR Value was either too large or too small for an Int32.
Exception: System.OverflowException
Message: Value was either too large or too small for an Int32.
Source: mscorlib
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at System.Convert.ToInt32(String value)
   at TweetSharp.TwitterService.GetAccessToken(OAuthRequestToken requestToken, String verifier)
   at Sitecore.Social.Twitter.Networks.Providers.TwitterProvider.AuthGetAccessToken(AuthArgs args)
   at Sitecore.Social.Client.Connector.SocialLogin.ProcessRequest(HttpContext httpContext)

   Could be fixed in 2 steps:

1. Downloading newer Social Connected 3.0 module
2. Unzipping and replacing TweetSharp.dll in your Sitecore

   I tried to find some similar solution for error with Facebook API:
"Invalid Scopes: offline_access, publish_stream, read_stream. This message is only shown to developers. Users of your app will ignore these permissions if present. Please read documentation for valid permissions at https://developers.facebook.com/docs/facebook-login/permissions".
But there was no ready to use solution. Problem was inside:

private void AddPermissionsToRequest(GetAccountCredentialsRequest accountCredentialsRequest, string networkName)
{
 string a;
 if ((a = networkName.ToLowerInvariant()) != null)
 {
  if (a == "facebook")
  {
   accountCredentialsRequest.Permissions.Add("offline_access", string.Empty);
   accountCredentialsRequest.Permissions.Add("publish_stream", string.Empty);
   accountCredentialsRequest.Permissions.Add("manage_pages", string.Empty);
   accountCredentialsRequest.Permissions.Add("read_stream", string.Empty);
   return;
  }
...

Ok, it is only constants, lets change them:

1. Open ILDASM
2. Dump Sitecore.Social.Client, Version=2.1.0.0 to some.il file
3. Replace "offline_access", "publish_stream", "read_stream" with "publish_pages", "user_posts", "public_profile". It is not strict correspondence, it should be deeply investigated, but these permissions will be enough to create and post social messages.
4. Using ILASM, assemble some.il into Sitecore.Social.Client and copy it to our Sitecore bin folder

But after trying to add Facebook account we get new error message:

ERROR The given key was not present in the dictionary.

Exception: System.Collections.Generic.KeyNotFoundException

Message: The given key was not present in the dictionary.

Source: mscorlib

   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)

   at Sitecore.Social.Facebook.Networks.Providers.FacebookProvider.GetDisplayName(Account account)

   at Sitecore.Social.Facebook.Client.Wizards.AddNetworkAccount.PageControls.FacebookWaitForAuthPageControl.GetFacebookAccount(Application application, String accessTokenSecret)

   at Sitecore.Social.Facebook.Client.Wizards.AddNetworkAccount.PageControls.FacebookWaitForAuthPageControl.GetAccountDisplayName()

   at Sitecore.Social.Client.Wizards.AddNetworkAccount.PageControls.WaitForAuthPageControl.CheckAuthStatus()

   After investigation I figured out that problem is inside:

public string GetDisplayName(Account account)
{
 System.Collections.Generic.IDictionary<string, object> accountData = this.GetAccountData(account, "/me");
 if (accountData == null)
 {
  return null;
 }
 return string.Format("{0} {1}", accountData["first_name"], accountData["last_name"]);
}

   Now Facebook returns only "name" instead of "first_name" and "last_name" separately. Guess, how we will solve this problem? Following similar steps that we have for previous fix we get new version of Sitecore.Social.Facebook.dll assembly. After copying it to Sitecore bin folder we are able to add Facebook account, create and publish posts with it.

Yahoo! We gave Sitecore Social Connected 2.1 second life.

P.S. Probably it is not all problems that could appear with changes in Facebook and Twitter APIs, but described above method fixes at least part of them. Use this solution on your own risk!