Showing posts with label CI/CD. Show all posts
Showing posts with label CI/CD. Show all posts

Thursday, June 10, 2021

Sitecore SXA CI/CD Improvements

It is a copy of my article initially placed here to keep all things in one place.

Saving compiled files in the source code is a bad practice. No one will argue that having either exe or dll or minified css or minified js is not good for your source control. But I continue to see serialized Sitecore SXA theme files added to source control. Many projects have pre-optimized-min.yml (or separate main.yml, styles.yml, scripts.yml, etc.) files saved in source control. And the common explanation, why it is there, is that it is hard to do on CI because it requires Sitecore to get these files. Let's figure out is it really complex and how we can avoid it.

The Problem

Firstly, let's talk about why having exe, dll, minified css, minified js, and pre-optimized-min.yml saved in your repository is not good for your project. All companies use version control systems for codebase management. Majorly it is Git. It is an awesome tool that allows the simultaneous work of many developers on the same project. But it works properly only when you don't save compiled (dll, exe) or transpiled (minified css, minified js) files. Once you start to save these files in the source control, you start to get conflicts when developers need to merge their changes. It happens because developers can work on different .cs, .js, .css, etc. files, but these files will be compiled or transpiled into the single file and you will get a conflict there. And pre-optimized-min.yml is the only serialization of transpiled minified js or css. It means that it will cause the same issue. Your developers will struggle with constant source conflicts and will spend time on resolving them. It means that it is better don't have them in source code. And there are 2 solutions, how we can get rid of them.

Solution 1

The easiest solution is to move execution of SXA gulp tasks(buildSass, buildCss, buildJs, cssOptimization, jsOptimization, uploadCss, uploadJs) to your CI\CD pipeline. You will need:

  1. Remove all files produced by SXA gulp tasks from source control, including serialized theme files.
  2. Add execution of SXA gulp tasks to your deployment step

Solution 2

Having pre-optimized-min.yml means that you already have either Unicorn or TDS in place. And you already delivering content to your environments using serialization. Let's look at content of this file:

---
ID: "bb97e335-fb50-4c14-b0e8-270f0f246f0a"
Parent: "e9eacb5b-ba97-46fb-af75-184cba337e1a"
Template: "962b53c4-f93b-4df9-9821-415c867b8903"
Path: "/sitecore/media library/Themes/Experiment/styles/pre-optimized-min"
DB: master
SharedFields:
- ID: "06d5295c-ed2f-4a54-9bf2-26228d113318"
  Hint: __Icon
  Value: "-/media/9be3803bcbfd44afaf16a8d2c94f15a2.ashx?h=16&thn=1&w=16"
- ID: "40e50ed9-ba07-4702-992e-a912738d32dc"
  Hint: Blob
  BlobID: "2afa94b5-0b9a-4319-82a1-8b17a411e275"
  Value: 6bGVmdDtjbGVhcjpi..................=
- ID: "6954b7c7-2487-423f-8600-436cb3b6dc0e"
  Hint: Size
  Value: 72312410
- ID: "6f47a0a5-9c94-4b48-abeb-42d38def6054"
  Hint: Mime Type
  Value: text/css
- ID: "c06867fe-9a43-4c7d-b739-48780492d06f"
  Hint: Extension
  Value: css
Languages:
- Language: en
  Versions:
  - Version: 1
    Fields:
    - ID: "25bed78c-4957-4165-998a-ca1b52f67497"
      Hint: __Created
      Value: 20210526T095146Z
    - ID: "5dd74568-4d4b-44c1-b513-0af5f4cda34f"
      Hint: __Created by
      Value: |
        sitecore\anton

Almost everything that is present in this file is text. There are only 2 fields that cause merge conflicts: Blob and Size. Are we able to fill these 2 fields without Sitecore? Yes, we are! Blob is base64 encoded content of the file: pre-optimized-min.css(or pre-optimized-min.js). And Size is the size of this file. We can use this PowerShell script to get both these values from pre-optimized-min.css:

<#
Get source file, convert to base64 string, put into the template, and put into destination
#>
param (
    [string]$source,
    [string]$destination,
    [string]$template
)

$content = [IO.File]::ReadAllText($source)
$bytes = [System.Text.Encoding]::UTF8.GetBytes($content)
$encodedText =[Convert]::ToBase64String($bytes)

$size = (Get-Item $source).length

$templateContent = [IO.File]::ReadAllText($template)

$destinationContent = $templateContent -replace '\${{size}}', $size
$destinationContent = $destinationContent -replace '\${{blob}}', $encodedText

[IO.File]::WriteAllText($destination, $destinationContent)

All additional steps that you need is to create a template, with Blob and Size values replaced with ${{blob}} and ${{size}} and run this script before Unicorn sync in your CI/CD pipeline.

Conclusion

Adding binary files to the source control is a bad practice. And your serialized SXA theme files are binary files (because of the presence of base64 string). Removing these files from source control and improvement of your CI/CD pipeline will save a lot of time and the mental health of your developers. And they will be grateful for this improvement.

Tuesday, August 7, 2018

CI/CD for Your Sitecore Pet Project Using AppVeyor

It is copy of my article placed in Sagittarius blog
There is lots of information about how to build continuous integration, continuous delivery and continuous deployment for enterprise Sitecore projects. But, none of these include information about Sitecore projects on GitHub with CI/CD (at least I haven’t faced with them). That is why I have decided to write a step-by-step guide on how to get CI/CD for your Sitecore pet project for free.
Let’s start:
  1. Create accounts for GitHub and AppVeyor (using authorization via GitHub)
  2. Create your Sitecore project and push it to GitHub
  3. Create and push appveyor.yml file to your repository
#Version of project could be based on build number
version: 1.0.{build}
#Path(s) to artifacts. We will create Sitecore update package and ship it as artifact
artifacts:
  - path: build\artifacts\*.update
    name: Sitecore.Akamai
before_build:
  #Configure Nuget to use public Sitecore packages feed
  - nuget sources add -Name SitecorePublicFeed -Source https://sitecore.myget.org/F/sc-packages/api/v3/index.json
  #Restore packages configured in solution
  - nuget restore Sitecore.Akamai.sln
build:
  #Solution file
  project: Sitecore.Akamai.sln
#Configuration of deployment artifacts to GitHub as release. It is not possible to save artifacts on Appveyor due to limited time of storage (30 days)
after_build:
  #Run PowerShell script that will build .update Sitecore package
  - ps: .\build\build.ps1  
deploy:
  #Release name
  release: Sitecore.Akamai-v$(appveyor_build_version)
  #Description of release
  description: 'Using Akamai features inside Sitecore'
  #Deploy to GitHub
  provider: GitHub
  #Secure token to upload files to GitHub https://www.appveyor.com/docs/deployment/github/
  auth_token:
    secure: rkLSxUbN2YMMG/r6lzLq1PN0n07dqJBtk/8ZR2c/InGy0SBOsmqGXfIQWMQOZUAs 
  draft: false
  prerelease: false
  on:
    branch: master                # release from master branch only
    appveyor_repo_tag: true       # deploy on tag push only
install:
  #Intallation of Sitecore.Courier Powershell module to have ability to build Sitecore package
  - ps: Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
  #Intallation of Sitecore.Courier to have ability to build Sitecore package
  - choco install sitecore-courier

    4. Commit and push powershell script that will prepare Sitecore update package with files and items using Sitecore.Courier
#Get package version from Appveyor
$version = $env:APPVEYOR_BUILD_VERSION
if ($version -eq $null) {
    $version = "1.0.0"
}
"Package version: " + $version
 
#Clear build directories
Remove-Item build\package -Recurse -ErrorAction Ignore
Remove-Item build\artifacts -Recurse -ErrorAction Ignore
#Create directories structure for package
New-Item -Name build\package -ItemType directory
New-Item -Name build\artifacts -ItemType directory
New-Item -Name build\package\Data -ItemType directory
New-Item -Name build\package\bin -ItemType directory
New-Item -Name build\package\App_Config\Include\Foundation -ItemType directory
 
#Copy Sitecore items and assemblies to proper location
Copy-Item .\src\Foundation\Akamai\code\bin\Foundation.Akamai* .\build\package\bin
Copy-Item .\src\Foundation\Akamai\code\App_Config\Include\Foundation\Foundation.Sitecore.Akamai.config .\build\package\App_Config\Include\Foundation
Copy-Item .\src\Foundation\Akamai\serialization\* .\build\package\Data -recurse
 
#Run Sitecore Courier to prepare .update package
$packageCmd = "Sitecore.Courier.Runner.exe -t build\package -o build\artifacts\sitecore.akamai." + $version + ".update -r"
iex $packageCmd

   5. Add a new project to Appveyor from GitHub
   6. Configure the publishing of artifacts. Create a new GitHub token. Encrypt it using AppVeyor and update appveyor.yml with new encrypted value.
   7. Update readme.md file with badge of build status
That is all. After next commit to your GitHub repo, new AppVeyor build will be triggered and Sitecore update package will be prepared. Sitecore update package will be available from artifacts tab on build details page. New GitHub release will be created after the creation of the new tag in the repository. The release will contain an archive of source files and Sitecore .update package. Update package will contain files and items and could be installed on any Sitecore website using update installation wizard.

As a bonus, you can configure code metrics on Sonar Cloud for free:
  1. Create a new Sonar Cloud account using GitHub authorization
  2. Generate new Sonar Cloud token and encrypt it using AppVeyor
  3. Change you appveyor.yml configuration:
before_build:
  #Start SonarQube runner
  - MSBuild.SonarQube.Runner.exe begin /k:"Sitecore.Akamai" /d:"sonar.host.url=https://sonarqube.com" /d:"sonar.login=%sonar_token%" /d:"sonar.organization=github-antonytm" /n:"Sitecore.Akamai"
after_build:
  #Stop SonarQube runner
  - MSBuild.SonarQube.Runner.exe end /d:"sonar.login=%sonar_token%"
  - ps: .\build\build.ps1  
environment:
  sonar_token:
    secure: xxx #Put your encrypted Sonar Cloud token here
install:
  #Install msbuild SonarQube runner
  - choco install "msbuild-sonarqube-runner" -y

     4. Update readme.md file with Sonar Cloud code quality badges.

It took some time to set up this configuration first time. But for the second project, it took less than half an hour. This free lightweight cloud CI/CD is a good investment in saving your time to avoid preparing new builds manually. And you should not necessarily select services described in this article. It is only one way that was shown as an example. GitHub could be replaced with Bitbucket, AppVeyor with TravisCI, etc.

You can see all this in action inside my repository on GitHub.