Showing posts with label GitHub. Show all posts
Showing posts with label GitHub. Show all posts

Saturday, November 13, 2021

Sitecore Solr Proxy Console: Update to Sitecore 10 Version

It is a copy of my article, initially placed here to keep all things in one place.

 2 years ago I had problems with troubleshooting issues with search based on Sitecore Solr. I solved that problem by writing a small tool that makes the content management server acts as a reverse proxy. It receives a request and depending on the requested URL, it either handles the request by Sitecore or redirects it to the Solr server. It made things simple, I didn't need any additional access. All that I needed was the ability to log in on Sitecore CM.

Time passed, and the problem with access to the Solr admin console didn't disappear. It became even more actual with Azure, Docker, and Kubernetes hosting. Of course, when you are an admin on your Azure subscription, you can easily give yourself permission. But sometimes, especially when you work with big companies, giving permission is not a quick process. But you need to solve the issue with search right now. And one of my colleagues recently faced the same problem. It means that issue is still actual.

The version that was created for Sitecore 8, didn't work and required a few updates. The first was the way how Sitecore Solr connection is configured. The second was a change in arguments for HttpRequestProcessor implementation. The third was changes in the authentication process. After these small changes, now module is ready for Sitecore 8-10 versions.

How to use module

  1. Download update package from GitHub releases page. You need to use 1.+ version for Sitecore 8.+ and 2.+ version for Sitecore 9.+ and 10.+
  2. Install update package using Sitecore update installation wizard
  3. You will get a link on Sitecore desktop to run Solr proxy
  4. You will be able to access Sitecore Solr Proxy using https://yourwebsite/solr for Sitecore 8.+ and https://yourwebsite/sitecore/shell/solr/ for Sitecore 9.+ and Sitecore 10.+

I hope, this module will save your time during troubleshooting search issues in different environments.

Saturday, February 22, 2020

Sitecore and Google Lighthouse Integration

Google Lighthouse became almost industry standard tool for measuring performance, accessibility, progressive web apps and SEO. There are few reasons, why it had happened. First of all, it is integrated with most popular browser: Google Chrome. Everyone is able to press F12(open DevTools) click “Generate report” button and get results. You should not be an expert, Lighthouse will tell you if your images are not optimized, if your scripts running too long, if you have wrong page structure that have bad influence on SEO. Second reason of Lighthouse popularity is that Google is number one web search engine. And as it is major source of traffic for many websites, it has sense to optimize your pages based on Google recommendations, which are can easily get from Lighthouse.

Pages performance, accessibility and SEO are very important for any website built on Sitecore. That is why I decided to integrate Lighthouse with Sitecore. It will allow you to run and access reports directly from Sitecore interface, have historical information and historical charts. As for me, it could be good checker for your day to day work with Sitecore. You are doing some improvement, you can easily understand how this improvement influence on particular pages and whole website. Or you are doing some change that influences on many pages, you can easily find out if it doesn’t break anything related to performance or accessibility.

I am glad to introduce: Sitecore.Lighthouse. Sitecore module that provides ability you to get everything from Lighthouse directly from Sitecore interface. On 22 February of 2020 it was tested only on Habitat Home and other few sites. That is why any contribution or bug reports are welcome. 



Sunday, July 8, 2018

Sitecore and Akamai Hidden Gems, Part 1: GeoIp Detection

    Akamai is well known CDN(content delivery network) provider. It provides content delivery features that are very widely used  across popular websites. Besides CDN, Akamai provides additional features, but not everyone is aware of them. One of them is providing GeoIp data. In brief, how it works:
  1. Akamai maintains database of IP addresses
  2. When user requests your website under Akamai, Akamai is able to parse IP address of request on the fly and add information about user’s geographic location, network, connection speed, etc. to request header. 
  3. You get X-Akamai-Edgescape request header on your server that contains a lot of information that could be used. E.g.: X-Akamai-Edgescape: georegion=263,country_code=US,region_code=MA,city=CAMBRIDGE,dma=50 6,pmsa=1120,areacode=617,county=MIDDLESEX,fips=25017,lat=42.3933,l ong=-71.1333,timezone=EST,zip=02138-02142+02238-02239,continent=NA ,throughput=vhigh,asnum=21399
  4. You are able parse this header and get more information about your visitor to show him relevant information on website
      Out of the box Sitecore also provides provides GeoIp personalization. Both Akamai and Sitecore GeoIp detection have it’s benefits. Akamai GeoIp is free(if you are already customer of Akamai) and could be quicker, because you don’t need to spent server time to get information about IP address. Sitecore GeoIp is ready to use out of the box and is better integrated with Sitecore Experience Platform. Depending on project you can prefer different GeoIp provider.
    If you choose Akamai then you are able to find article that describes how to do it. That solution works properly for Sitecore 6.6 - 7.2, but doesn’t work on Sitecore 8+. Since some time Sitecore moved parsing GeoIp headers to separate thread:

if (orCreate.GeoIpResolveState == GeoIpResolveState.Unresolved)
{
 GeoIpManager.StartResolvingThread(orCreate);
}

That thread doesn’t know nothing about your context, that is why call of HttpContext.Current.Request.Headers["X-Akamai-Edgescape"] causes “Object reference not set to an instance of an object” exception. It means that only overriding of LookupManager will not work. I have managed to transfer data from Akamai header to Sitecore analytics context in next way:
  1. Disable Sitecore.Analytics.Pipelines.CommitSession.UpdateGeoIpData, Sitecore.Analytics.Pipelines.CreateVisits.UpdateGeoIpData and Sitecore.Analytics.Pipelines.EnsureClassification.UpdateGeoIpData processors
  2. Override Sitecore.Analytics.Pipelines.StartTracking.UpdateGeoIpData processor

using Foundation.Akamai.GeoIp;

namespace Foundation.Akamai.Pipelines
{
    public class UpdateGeoIpData 
    {
        public void Process(object args)
        {
            var whois = new LookupProvider().GetInformationByIp("");
            if(whois!=null)
            {
                Sitecore.Analytics.Tracker.Current.Interaction.SetGeoData(whois);
            }
        }
    }
}

    This approach works, because parsing header is quick operation that doesn’t need running separate thread. As I understand four UpdateGeoIpData processors in different pipelines were required to sync thread that get GeoIp information with main HttpRequest thread.
    To get more details you can review my repository on GitHub or download package and try it by yourself.

Wednesday, May 30, 2018

Adding HtmlCache Viewing Feature to Sitecore.Rocks Visual Studio Extension

It is copy of my article placed in Sagittarius blog

After writing an article about viewing Sitecore HTML cache, I thought that it is not convenient to move custom code from one project to another. When, instead, there is ability to extend widely used existing Sitecore development tool Sitecore.Rocks then it should be done. I decided to contribute into it.

First of all, you need to fork Sitecore.Rocks project on GitHub.

    Sitecore.Rocks works like server-client application. You have server side (Sitecore.Rocks.Server, connector in other words) that is copied to all Sitecore websites and client side: interface in Visual Studio to display different things from Sitecore. Fortunately, there is already “Data” column for cache viewer to display cache details and we don’t need to modify interface to add something new. But this column shows only path to item, when cache key is Sitecore identifier. In all other cases Data column will be empty. Let’s extend amount of data that could be shown in this column.

     Logic that returns data to be displayed in this table located in /src/Sitecore.Rocks.Server/Requests/Caches/GetCacheKeys.cs file. We can modify it and add ability to return cache value when cache type is string or id:


    After building a solution, we need copy Sitecore.Rocks.Server.dll to bin folder of our website. Now, you are able to see what it inside cache. If you are interested in viewing other types of cache(not only string and id) then you are able to extend code above with your needs.




If my pull request is accepted, then this feature will be available in one of next Sitecore.Rocks version.

Friday, May 4, 2018

How to Check What is Inside Sitecore HTML Cache?

It is a copy of this blog post.

Sitecore has a lot of caches: AccessResultCache, DataCache, DeviceItemsCache, HTML cache, ItemCache, ItemPathsCache, PathCache, RegistryCache, RuleCache, StandardValuesCache, ViewStateCache, XslCache, FieldReaderCache and others. When you facing with caching issue on your website(something is either not cached, or cached but should not), it is important to have the ability to see what is present in cache.

All Sitecore caches consist of cache key and cache value. Cache key is always string. Cache value could be any type of object depending on cache. We can take as example HtmlCache cache. When you see what cache key you have and what data is there, you can better tune caching of controls, especially when you have custom(not out of the box) cache criterias (vary by IP, vary by country, vary by role, etc.) and a lot of controls on a page. Here is code snippet to get everything inside HtmlCache:
It requires using reflection due to private methods in HTML cache. The result of execution GetCaches method will be List<Tuple<string, string>>, that you can filter, make search or display on the page. It can save you a lot of time when debugging Sitecore caching issues. Here is an example of service page that displays cache keys and cache values:



If you can’t add this code to some secure page in your solution(you don’t want to do it or need some out of the box solution) then there are also other useful tools for troubleshooting issues with Sitecore cache. But they will not show you the cached value of HtmlCache entry:


Wednesday, April 11, 2018

Using WebP Format in Sitecore

It is a copy of this blog post to have all my articles in one place.
WebP is promising format that allows good optimization comparing to JPEG and PNG. According to tests it provides lossless images that are 26% smaller compared to PNGs and lossy images that are 25-34% smaller than comparable JPEG images at equivalent SSIM quality index.
This format becomes more and more popular over the web nowadays. Google promotes WebP format, it released WebP as open source to allow anyone works with it and suggest improvements. But, is it supported by major browsers? Going to CanIUse gives answer that not:


(support details are on 2018/01/21, it could be changed over time)

It is supported by Chrome, Opera, Chrome for Android. Safari and Firefox are experimenting(not supported yet) with supporting WebP images. IE and Edge doesn’t support it.
But how it can be used now, when not all browsers support it? Each browser(web client) provides Accept header when getting resources from server. If this header contains image/webp, web server know that it can returns WebP format. As example Akamai CDN use this behavior. It can return optimized WebP images for web clients who support it and JPEG and PNG for those who doesn’t support.
Let’s consider how it could be used in Sitecore. It has no sense to add support of WebP format to media library for now, we can’t return this file format to all web clients. But it makes sense to return WebP format to web clients that can use it. Saving about 25% of time on loading images can make big difference for user experience, especially on mobile. I decided don’t write new image optimizer from the scratch and add support of WebP to well known tool for images optimization for the Sitecore: Dianoga.
To make it works, we should get understanding how Sitecore Media Library and Sitecore Media Library cache work. Sitecore media library creates files on disk after each request, or uses previously created files. By default these files are located under \Website\App_Data\MediaCache\website\{some folders}. There are few files in each folder. First type of files is cache for image. It is image itself, Sitecore don’t need to preprocess width, height and other parameters for each media request. It does processing only once. Second type of files is .ini file, it is metadata for cached object. Metadata contains next values:
Key. e.g. ?as=False&bc=0&h=0&iar=False&mh=0&mw=0&sc=0&thn=False&w=0. Key contains all media query parameters.
Extension - extension of file.
Headers - cached headers that should be returned to web client
DataFile - file name of object that should be returned to web client.

We need to extend forming of key, to add one more parameter that will indicate support of WebP format. It will require extending of MediaRequestHandler:

public class MediaRequestHandler : Sitecore.Resources.Media.MediaRequestHandler
{
 protected override bool DoProcessRequest(HttpContext context, MediaRequest request, Media media)
 {
  if (context?.Request.AcceptTypes != null && (context.Request.AcceptTypes).Contains("image/webp"))
  {
   request.Options.CustomOptions["extension"] = "webp";
  }

  return base.DoProcessRequest(context, request, media);
 }

 private static bool AcceptWebP(HttpContext context)
 {
  return context?.Request.AcceptTypes != null && (context.Request.AcceptTypes).Contains("image/webp");
 }
}

Our key will contain one more additional parameter: extension. E.g.: ?as=False&bc=0&h=0&iar=False&mh=0&mw=0&sc=0&thn=False&w=0&extension=webp
Let’s add handler that will process WebP compatible requests based on
CommandLineToolOptimizer:

public class WebPOptimizer : CommandLineToolOptimizer
{
 public override void Process(OptimizerArgs args)
 {
  //If WebP optimization was executed then abort running other optimizers
  //because they don't accept webp input file format
  if (args.AcceptWebP)
  {
   base.Process(args);
   args.AbortPipeline();
  }
 }

 protected override string CreateToolArguments(string tempFilePath, string tempOutputPath)
 {
  return $"\"{tempFilePath}\" -o \"{tempOutputPath}\" ";
 }
}

It is very easy, it runs cwebp.exe tool that converts JPEG or PNG to WebP. It doesn’t utilize all available command line options, and could be tuned depending on requirements. All others code changes are more about Dianoga configuration and unit tests, I will not stop on them in this article. If you want more details, you can review all changes in GitHub repository.

How to enable Dianoga WebP support for your project:
  1. Clone GitHub repository and build project
  2. Enable Dianoga.WebP.config.disabled config
  3. Open web.config and change line <add verb="*" path="sitecore_media.ashx" type="Sitecore.Resources.Media.MediaRequestHandler, Sitecore.Kernel" name="Sitecore.MediaRequestHandler" /> to <add verb="*" path="sitecore_media.ashx" type="Dianoga.MediaRequestHandler, Dianoga" name="Sitecore.MediaRequestHandler" />
  4. If you have custom MediaRequestHandler (e.g. Habitat is used) then skip step 3 and override DoProcessRequest method with detection of support of WebP format. See MediaRequestHandler code listening above.

P.S. It is experimental feature, use it on your own risk. :-)

Wednesday, January 4, 2017

Sitecore Log4Net Appender for Slack


    Slack became essential part of almost all projects(or teams). It is very powerful communication tool that speed up sharing information. As we use Slack for our Sitecore-based project I decided to create prototype of slack bot that will share information about Sitecore production site health to Slack channel. 


    After quick search I found that something similar was already created on Sitecore Hackathon, named Slack for Sitecore. There is also video that describe how it works.

    But as for me, it is too complex to use it in real projects. You don't need to have a lot of information in Slack channel, otherwise you will skip all messages from Slack bot. And also you don't need to spend a time on configuration of module. Sitecore already has mechanism how to show that something went wrong - logging errors to log file. Why don't use it for our Slack bot?

    First of all, we should take Slack log4net appender and change it to be suitable for Sitecore. It could be done quite easily by changing dependency from log4net assembly to Sitecore.Logging assembly.

    Now, we are able to add log4slack appender to our Sitecore configuration:

<appender name="SlackAppender"
    type="Log4Slack.SlackAppender, Log4Slack">
  <WebhookUrl value="https://hooks.slack.com/**********************" />
  <Channel value="#testing" />
  <Username value="*********" />
  <IconUrl value="" />
  <IconEmoji value=":ghost:" />
  <AddAttachment value="true" />
  <AddExceptionTraceField value="true" />
  <UsernameAppendLoggerName value="true"/>
  <threshold value="Error" />
</appender>
....
<root>
  <priority value="INFO"/>
  <appender-ref ref="LogFileAppender"/>
  <appender-ref ref="SlackAppender"/>
</root>

   And we got next messages in our Slack channel:

    Of course, you don't need having all logs to be transferred to your Slack channel, because no one will read them. That is why you either should set up separate logger for critical messages or set logging level for this appender to error: <threshold value="Error" />. And you will not miss any critical exception on your site.