This example covers better practices in continuous integration, CI, for .NET. While it can be adapted for other CI-CD tech stacks, the example uses a Github Actions workflow.
Why This Example of CI in .NET?
- .NET is not easily configured to do discrete CI steps that use intermediate product from the previous steps, and documentation can be misleading or even lacking
- Developing and debugging pipeline yaml is still more of a brute-force, try, try again effort rather than a sophisticated, efficient effort
- Validating that the pipeline jobs and steps do the intended in terms of efficient use of resources is not always at the forefront of CI approaches — sometimes, it’s “It works!” meaning the intended product is generated
- Visibility into pipeline metrics such as storage consumption or duration of pipeline can be missing
Example Scope vs “Real World”
- This example’s workflow covers only a trigger of a basic push to the GitHub repository.
- For a live system, we would likely to have story, master, and release branches as well as merge request to take into account in our plan
- This approach does not emulate a complex repo of multiple subsystems but can be adapted for such
- Caching and artifacts are included as examples of better practices
- Generally, for a simple project with few dependencies, caching and artifacts may not be worth the network load or worth maintaining a more complex yaml — depending on your CI optimization goals
- For real systems, take an additional step of deploying the app to a testing environment where other apps live, to test how the app plays well with others
- If possible, automate this automated as part of the CI as systems-level testing (aka integration testing)
- This approach covers only CI, not CD; a later post will cover deployment
Overview of CI Stages
So, this approach uses
- A cache for external nuget dependencies
- Artifacts for intermediate ci products passed from one job to the next
GitHub Actions Workflow Yaml
While this example uses GitHub Actions workflow, you can adapt the approach to other CI tools.
Matching the diagram above, the yaml shows the CI approach used. Before we discuss CI stages, let’s take a look at how the yaml starts.
name: LabDemo.MinimalApi.BuildByStages
run-name: ${{ github.actor }} is experimenting with GitHub actions 🚀
on: [push]
env:
PROJECT: LabDemo.MinimalApi
TEST_PROJECT: LabDemo.MinimalApi.Tests
BUILD_ARTIFACT_NAME: build
PUBLISH_ARTIFACT_NAME: publish
BUILD_PATH: build
PUBLISH_PATH: publish
jobs:
First, on push. On line 3 the workflow runs whenever a push to the repository happens.
- This demo app’s CI has no special use cases for pull requests or branches.
Second, workflow-global variables. On line 5 we establish environment variables.
- Ensures consistency in the workflow
- Makes the yaml less error-prone
- Makes reuse of this yaml for another project much easier.
Important Caveat About Caching
The following applies to GitHub Actions. For other tools, the caching may work differently. Think it through.
You must use the cache action in your workflow before you need to use the files that might be restored from the cache. If the provided key matches an existing cache, a new cache is not created and if the provided key doesn’t match an existing cache, a new cache is automatically created provided the job completes successfully. — GitHub Actions Cache README
CI Validation
Overall, we want to ensure
- The workflow produces the expected output
- Job and step behaviors are meeting expectations and/or performance goals
For output, success of the workflow is usually enough, with a little manual inspection sprinkled in.
For behavior, manual inspection has its limits — at least through most CI tools.
- In this case, GitHub Actions have minimal metrics, like the time a workflow took to complete
- Our example will dump sizes of subdirectories so that, by visual inspection, one can see not only the cached nuget and artifacted build and publish directories but also their sizes
- If you want to go beyond this basic logging, third party tools exist (see “How to Track” in this article on optimizing CI-CD).
Local Debugging and Exploring of the Stages
The source code includes a bash script you can use to emulate the CI steps locally and examine behavior.
└─➤ ./ci -a
=============================================================
CI support script
>>>>>>>>>>>>>>>>>>>>>>>>>>> ALL
>>>>>>>>>>>>>>>>>>>>>>>>>>> CLEAN
21M ./LabDemo.MinimalApi.Tests/
3.9M ./LabDemo.MinimalApi/
108K ./img/
>>>>>>>>>>>>>>>>>>>>>>>>>>> RESTORE-BUILD
MSBuild version 17.4.0+18d5aef85 for .NET
Determining projects to restore...
Restored C:\Users\crmcg\source\repos\LabDemo.MinimalApi\LabDemo.MinimalApi\LabDemo.MinimalApi.csproj (in 1.25 sec).
LabDemo.MinimalApi -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\build\LabDemo.MinimalApi.dll
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:03.07
63M .
>>>>>>>>>>>>>>>>>>>>>>>>>>> TEST
Determining projects to restore...
Restored C:\Users\crmcg\source\repos\LabDemo.MinimalApi\LabDemo.MinimalApi.Tests\LabDemo.MinimalApi.Tests.csproj (in 28.96 sec).
1 of 2 projects are up-to-date for restore.
LabDemo.MinimalApi -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\build\LabDemo.MinimalApi.dll
LabDemo.MinimalApi.Tests -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\build\LabDemo.MinimalApi.Tests.dll
Test run for C:\Users\crmcg\source\repos\LabDemo.MinimalApi\build\LabDemo.MinimalApi.Tests.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Passed! - Failed: 0, Passed: 8, Skipped: 0, Total: 8, Duration: 99 ms - LabDemo.MinimalApi.Tests.dll (net7.0)
>>>>>>>>>>>>>>>>>>>>>>>>>>> PUBLISH
MSBuild version 17.4.0+18d5aef85 for .NET
Determining projects to restore...
All projects are up-to-date for restore.
LabDemo.MinimalApi -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\LabDemo.MinimalApi\build\LabDemo.MinimalApi.dll
LabDemo.MinimalApi -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\publish\
LabDemo.MinimalApi.Tests -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\LabDemo.MinimalApi.Tests\build\LabDemo.MinimalApi.Tests.dll
LabDemo.MinimalApi.Tests -> C:\Users\crmcg\source\repos\LabDemo.MinimalApi\publish\
22M ./LabDemo.MinimalApi.Tests/
4.6M ./LabDemo.MinimalApi/
21M ./build/
108K ./img/
722M ./nuget/
45M ./publish/
>>>>>>>>>>>>>>>>>>>>>>>>>>> METRICS
Original Size: 26533 . bytes
Ending Size: 834168 . bytes
Size increase: 807635 bytes
Time elapsed: 02m:09s
What’s Next
Further articles on CI stages to come — the build stage is next.