Monday, September 12, 2022

Validate Your Sitecore Serialization Using Pre-commit Git Hook

 This is a copy of my blog post originally posted here to keep all things in one place.

How often do you push broken Sitecore serialization with your commits? If the answer is "never" then I envy you, you are a very neat developer! Then how often does someone in your team push broken serialization? I don’t believe that on second question answer will be "never". Depending on the size of the project and the intensity of work, it will definitely happen from time to time. Let’s figure out how to prevent it.

Git has the ability to add hooks. Hooks are programs you can place in a hooks directory to trigger actions at certain points in git’s execution. There are many of points, where we are able to include our programs(scripts): applypatch-msgpre-applypatchpost-applypatchpre-commitpre-merge-commitprepare-commit-msgcommit-msgpost-commitpre-rebasepost-checkoutpost-mergepre-pushpre-receiveupdateproc-receivepost-receivepost-updatereference-transactionpush-to-checkoutpre-auto-gcpost-rewritesendemail-validatefsmonitor-watchmanp4-changelistp4-prepare-changelistp4-post-changelistp4-pre-submitpost-index-change. Seeing this big list of hooks you may understand, that it is possible to add your custom logic only everywhere in the Git execution process. It is like events or pipelines in Sitecore: places, where you can extend existing logic according to your needs. Despite of big list of hooks, they are not too popular. The major part of projects will have either no Git hooks, or you may have hooks that run linters or force you to follow a certain commit message format.

Recent Sitecore versions provide powerful Sitecore CLI. This CLI includes work with serialization. And we are interested in the validation of serialization. Now, in order to check if Sitecore serialization saved on disk is valid, you don’t even need Sitecore runtime. You can do it by running one command dotnet sitecore ser validate. Let’s add it to Git pre-commit hook to avoid our team members pushing broken serialization to git.

  1. You need to download this Gist.
#!/bin/sh
# Pre-commit Git hook
# It runs Sitecore CLI Serialization check
# And doesn't allow to perform commit if serialization is broken
validate=$(dotnet sitecore ser validate)
regex_errors="Errors were detected"
count=$(echo "${validate}" | grep -c "${regex_errors}")
if test $count -gt 0
then
	echo "Sitecore serialization errors were detected"
	echo "Please run 'dotnet sitecore ser validate'"
	echo "And fix all errors before commit"
	exit 1
else	
	exit 0
fi
  1. Then you need to put it into .git\hooks directory with name pre-commit
  2. Now, before making a new commit, serialization validation will be executed. And if something is wrong with serialization, you will not be able to make a commit. You will get an error message:

Git Hook in Action

After fixing serialization issues, you should be able to move forward. And now you may be confident that no one in your team will be able to push broken serialization and block someone else’s work.

Git hook will be a good option until you don’t have a huge amount of serialized items. It will work fine until running dotnet sitecore ser validate is fast. But if it will start to take more than a few seconds it may start to annoy developers. What you can do in that case? You will need to incorporate validation into your CI pipelines. You may run it on each pull request to make sure that no one will be able to merge broken serialization to the main branch.

Tuesday, September 6, 2022

Covering Sitecore Dianoga Module with Integration Tests

 This is a copy of my blog post originally posted here to keep all things in one place.

 Dianoga grew over time. There were added different image formats, different strategies, logging, additional settings like path exclusion, etc. And all these things are configurable. You can turn them on and off and achieve the behavior that you want. In 2022, there are 15 configuration files. I decided to add new file format support, but the number of different configurations scared me. How to test all of them and be sure that everything works? How quickly reproduce the reported issue?

The answer is automated testing. One of the most valuable things in covering your code with tests is that you are not afraid to change it.Dianoga has already some unit test coverage.

But as Dianoga modifies Sitecore behavior, even 100% unit tests code coverage doesn’t guarantee that everything every time will work. It could be checked with integration tests. Integration tests require running Sitecore. We don’t want to have an infrastructure for it. Fortunately, we can easily run Sitecore in Docker containers. The only one thing is required: quick and easy ability to add Dianoga module to your Sitecore container. And that was the one of reasons for the creation of Dianoga Asset Image.

How does it work:

  1. Integration tests are located in the integration-tests folder of Dianoga sources.
  2. We have different docker-compose configurations: for sync mode, for async mode, with enabled SVG, with enabled SVG and WebP, with enabled SVG/WebP/avif/jpegxl.

visual studio code: docker-compose.yml

  1. You can easily add a new configuration if you need to run a specific test to reproduce some issue
  2. We have a few test pages with a number of images on them.
  3. We have test container. It runs .net core tests project. This project makes requests to Sitecore containers and verifies that Dianoga module squeezes images as expected.

visual studio: integration tests project

  1. We have Powershell script run-tests.ps1 that runs Docker configurations one by one and gather test results.
  2. All test results are saved to integration-tests\docker\data\tests folder

Sample results output

powershell: tests output

Conclusion

Running all these tests takes some time. But it saves a huge amount of time when you made some changes in your code and you want to make sure that all configurations still work as expected! Also, it become much easier to troubleshoot reported issues. You are able to set up a new Docker configuration and run a test to reproduce the issue. And as a bonus, you get examples, of how to configure Dianoga with different image formats and different strategies for your containerized environment.