PipelineLib/Guides/How to define a node test pipeline
Welcome to How to define a Node test pipeline, a simple PipelineLib guide to get your NodeJS project's test/lint/other scripts running for each patchset submitted to Gerrit.
Prerequisites
- A basic understanding of the Blubber image build tool. The Blubber tutorials can help you with that.
- An understanding of
.pipeline/config.yaml
basics, namely how to define a simple pipeline that will build a test image variant and run it. Some of the PipelineLib tutorials should help you learn these basics.
And of course, don't hesitate to get in touch with the WMF Release Engineering Team for help with following this guide.
Guide
The basic steps to getting your NPM scripts running in CI using Blubber and PipelineLib are:
- Define a generic image variant in
.pipeline/blubber.yaml
that invokesnpm run-script
. - Define a new pipeline that includes stages for building the image variant and running it for all relevant test/lint scripts.
- Tell our CI system about your new pipeline by following the How to configure CI for your project supplemental guide.
Define a generic npm run-script
image variant
dev@laptop:~$ cd src/project/foo
dev@laptop:foo$ vim .pipeline/blubber.yaml
version: v4
variants:
all-dependencies:
base: docker-registry.wikimedia.org/nodejs-devel
copies: [local]
node:
requirements: [package.json, package-lock.json]
npm-run:
includes: [all-dependencies]
entrypoint: [npm, run-script]
This simple .pipeline/blubber.yaml
gives you an image variant including all your production and development dependencies that can execute any script—given as a runtime argument—defined in your project's package.json
.
.pipeline/blubber.yaml
file, or your project might require different/additional configuration. What's critical is that you have some sort of image variant defined that can run your project's test/lint entry points.Define a test pipeline
Now that you have a generic image variant configured for running any given npm run-script
, define a test
pipeline that builds and runs it for all relevant project test/lint scripts.
dev@laptop:foo$ vim .pipeline/config.yaml
Let's say your project has two scripts that should run for every patchset submitted to your project's repo, npm run-script test
and npm run-script lint
. You'd define a .pipeline/config.yaml
that looks something like this.
pipelines:
test:
blubberfile: blubber.yaml
stages:
- name: built
build: npm-run
- name: unit-tested
run:
image: '${built.imageID}'
arguments: [test]
- name: linted
run:
image: '${built.imageID}'
arguments: [lint]
(Optionally) Inject environment variables into the container
Depending on your testing requirements, you may wish to inject environment variables or jenkins credentials into the container when it is run. You can define them like so.
pipelines:
test:
blubberfile: blubber.yaml
stages:
- name: built
build: npm-run
- name: unit-tested
run:
image: '${built.imageID}'
arguments: [test]
env:
VAR1_NAME: ${VALUE}
VAR2_NAME: 'some other value'
credentials:
- id: SONAR_API_KEY
name: SONAR_API_KEY
(Optionally) Define an execution graph that runs your scripts in parallel
Depending on your testing requirements, you may wish to run these two (or more) test suites in parallel. Configuration of execution
is how you accomplish that.
dev@laptop:foo$ vim .pipeline/config.yaml
pipelines:
test:
blubberfile: blubber.yaml
stages:
- name: built
build: npm-run
- name: unit-tested
run:
image: '${built.imageID}'
arguments: [test]
- name: linted
run:
image: '${built.imageID}'
arguments: [lint]
execution:
- [built, unit-tested]
- [built, linted]
execution
graph is defined by reading the configuration reference.This configuration essentially means: There are three distinct stages in two branches of execution. Execute built
first, then branch out and execute unit-tested
and linted
independently in parallel. As a visual graph, it ends up looking something like:
[set-up] ↓ built ⇙ ⇘ linted unit-tested ⇘ ⇙ [tear-down]
Next up
Now that you have a test pipeline defined for your project, head on over to How to configure CI for your project to tell our CI how to run it for each patchset submitted to Gerrit.