Setting up CI/CD
This article describes how to go from local work to CI/CD
CI - short for continuous integration - and CD - continuous delivery - is a dev/ops principle that allow teams to collaborate on projects while still maintaining a high degree of code cohesion. This is accomplished by making sure changes made by developers is integrated with all other changes as soon as possible, ensuring all changes are compatible and keeping the main branch in a releasable state at all times.
Since the main branch is always releasable, it should be released often, or continuously, hence the CD part of CI/CD. While it doesn't always make sense to release to production immediately, code should - at least - be continually released to a development test environment where all changes can be validated and tested.
Note
This article continues from the introduction to working in teams. It recommended to read it first before continuing with this article.
Prerequisites
This section covers some of the things you need to think about before setting up CI/CD:
- Release and build pipelines
- Deploy targets
- How to set up a clean host
Release and build pipelines
To get started with continuous integration and deployment, you need Build and Release pipelines - for instance:
- Locally hosted solutions like Jenkins
- Cloud-based solutions like Travis CI
- Pipelines in Azure DevOps.
The only requirement is that it's possible to build dotnet applications, and that the pipeline runner can communicate with/deploy to the deploy targets.
Deploy targets
Before starting continuous deployment, it's important to decide which deploy targets are necessary.
Consider this; if all developers are working locally, they are also working independently. That is great from the individual developer's perspective, but it's not great for validating the complete solution in relation to customer requirements.
The recommendation is to have a shared test site that is always synchronized with the main branch and an updated database. Having such a site enables testing of the whole solution to make sure that features do not clash with each other.
As such, it is recommended that you have:
- A shared test environment for internal dev testing
- A staging environment for user acceptance testing
- A production environment
With such a setup, your development environment is Local development -> Shared test -> Staging -> Production
. You can choose to host the shared test site any way you like, including in the cloud, as long as the requirements for hosting DynamicWeb 10 are met.
Clean host
Hosting the solution in the DynamicWeb Cloud enables you to take advantage of a version-less application strategy. This means that you don't need to worry about the version of the application or when to update, it's handled for you.
If, however, one or more of the target environments for the deployment requires BYO App or bring your own application - e.g. hosting environments like Azure or AWS - you need a clean host application to deploy. This is because custom code in DynamicWeb should be installed through the Files-folder - in DynamicWeb Cloud, it's required.
So in order to provide a great developer experience when running locally, but still accommodate version-less and BYO hosting environments, we need the right host projects.
Extending the solution setup from this guide, we make the following changes:
- Add project references to
MyCustomerProject.CustomCode
and any other custom code projects toMyCustomerProject.Host
- Make sure
MyCustomerProject.Host
is used as the startup project- These changes allow us to have the great development experience that developers are used to: press
F5
and run
- These changes allow us to have the great development experience that developers are used to: press
- Create a new host project called
MyCustomerProject.DeployHost
orMyCustomerProject.CleanHost
, or anything that makes sense to you
Doing this allows us to have the great developer experience as well as supporting the hosting environments that are not natively familiar with DynamicWeb.
Note
It's also a good idea to make sure all project are using the same version of DynamicWeb. To simplify this process, remove all references to DynamicWeb packages from the projects and create a new file in the root of the solution called Directory.Build.props
and add the package reference to it.
<Project>
<ItemGroup>
<PackageReference Include="Dynamicweb.Suite.Ring1" Version="*" />
</ItemGroup>
</Project>
Change the package to the specific release ring you want to use. If hosting in DynamicWeb Cloud, be sure to align the specified ring with the host.
Setting up continuous integration
The goal of the continuous integration pipeline is to build the artifacts necessary to deploy the solution - more specifically:
- The custom code
- The Files-folder
- Potentially the clean host application.
How the build pipeline specifically needs to be configured changes from build system to build system. In this example, we use Azure DevOps so you may need to adapt the setup to your specific environment.
Build the clean host
To build the clean host, we use the publish command for the dotnet cli task.
- task: DotNetCoreCLI@2
displayName: 'Build DeployHost'
inputs:
command: publish
publishWebProjects: false
projects: |
**/*.DeployHost.csproj
arguments: '-c Release --output $(Build.ArtifactStagingDirectory)'
Build custom code
Custom code does not need to be published, rather we use the build command in the dotnet cli. We configure it to build all projects in the solution but excluding the following:
- Files: Project should not be built, rather it's deployed as-is
- Host This is never used outside local development
- DeployHost / CleanHost: Handled in a previous step
- task: DotNetCoreCLI@2
displayName: 'Build custom projects'
inputs:
projects: |
**/*.csproj
!**/*.Files.csproj
!**/*.DeployHost.csproj
!**/*.Host.csproj
arguments: '-c Release --artifacts-path $(Build.BinariesDirectory)/output'
Creating the pipeline artifact
With everything built, we now need to package it up into the pipeline artifact. The artifact is a zip-file with three additional zip-files inside.
- The clean host (
DeployHost.zip
) - The custom code (
Custom.zip
) - The Files (
Files.zip
)
The clean host is already zipped due to how the task is configured, so we just need to take care of the remaining two zip-files.
Firstly, we want to archive the custom code assemblies, but since the task doesn't allow globbing paths, we need to copy the assemblies first.
- task: ArchiveFiles@2
displayName: 'Archive $(Build.BinariesDirectory)/custom'
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)/custom'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/Custom.zip'
After they've been copied, we zip them.
- task: ArchiveFiles@2
displayName: 'Archive $(Build.BinariesDirectory)/custom'
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)/custom'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/Custom.zip'
Now we repeat the process for Files
- task: CopyFiles@2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
Contents: '**\publish\*.zip'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: ArchiveFiles@2
displayName: 'Archive SnizzlePuff.Files/Files'
inputs:
rootFolderOrFile: SnizzlePuff.Files/Files
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/Files.zip'
Finally, we create the pipeline artifact with the contents we've just created.
- task: PublishBuildArtifacts@1
displayName: 'Publish Artifact: drop'
Deploying the solution
When it comes to deploying the build pipeline artifact, we must consider the deployment strategy for each of the deploy targets. Some targets need are deployed automatically while some may require a manual approval step before new releases are deployed. In most cases:
- The internal testing environment should be updated as soon as a new artifact is available
- Deployments to staging or production would need more scrutiny
First things first, we need to create a release pipeline that contains all our environments except for the local. The pipeline is connected to the build pipeline and configured to run when a new artifact becomes available.
There are many different ways to deploy a solution. In this example, we'll be using the DynamicWeb CLI as it simplifies a lot of operations. Be aware that this requires a running DynamicWeb solution in the other end, as well as an API key to interact with the running solution.
Install the DynamicWeb CLI as a global tool using NPM to make sure it's available for the subsequent tasks.
- task: Npm@1
displayName: 'npm install -g @dynamicweb/cli'
inputs:
command: custom
verbose: false
customCommand: 'install -g @dynamicweb/cli'
Extract the custom code assemblies from the artifact.
- task: ExtractFiles@1
displayName: 'Extract files'
inputs:
archiveFilePatterns: '**/Custom.zip'
destinationFolder: '$(System.DefaultWorkingDirectory)/Custom'
Install the custom assemblies into the running solution.
- powershell: |
$dlls = Get-ChildItem -Filter *.dll
foreach ($dll in $dlls) {
dw install "$dll" --host $(host) --apiKey $(apiKey)
}
workingDirectory: '$(System.DefaultWorkingDirectory)/Custom'
displayName: 'Assembly loop to install'
Extract the Files from the artifact.
- task: ExtractFiles@1
displayName: 'Extract files '
inputs:
archiveFilePatterns: '**/Files.zip'
destinationFolder: '$(System.DefaultWorkingDirectory)/Files'
Upload Files to the running solution.
- script: 'dw files -iro . / --host $(host) --apiKey $(apikey)'
workingDirectory: '$(System.DefaultWorkingDirectory)/Files'
displayName: 'dw files'
The next steps varies depending on whether the deploy target is in DynamicWeb Cloud or not.
Next steps for DynamicWeb Cloud
When the solution is hosted in DynamicWeb Cloud, you can take advantage of the version-less nature of the hosting environment.
All you need to do after the previous steps is to recycle the site. It is optional, but strongly recommended. To recycle the site, simply upload a file called changeversion.txt
to /Files/System/CloudHosting
using the CLI. The file only needs to contain an indication of which release ring to use. Pointing to the same release ring as the solution is currently using will cause a recycle rather than a version change. Read more about cloud hosting.
- powershell: |
"R1" > changeversion.txt
dw files -i changeversion.txt System/CloudHosting --host $(host) --apiKey $(apikey)
displayName: 'Recycle application'
Next steps for other hosts
In this case, the solution is hosted in an Azure Web App.
- Use the Azure Web App task to deploy
DeployHost.zip
to the site - Restart the site (recommended, but not required)
- Ping the site to bring it back up (optional)
Onboarding new developers
One of the main challenges is how to get started as a new developer on the team. Code is easily replicated by cloning the repository, but getting data is a little more tricky. Some manual steps are required, but this can most likely be scripted to simplify the process. Additionally, this is made easier by using the shared test environment as the base.
In broad terms, the process is
- Clone the solution repository
- Download/export the database
- Set up the solution locally using the steps outlined in the installing Swift guide