Maps/Services deployment

Deploying maps services

Before you actually start deploying check out some basic tips for deployment.

Developer setup for updating deployment repo

The deployment repo is used to compose the kartotherian and tilerator code and it's dependencies. Before a new versions can be deployed there must be an update to the deployment repo.

We build the deploy repo in Docker in order to ensure that the Node modules have the correct binaries. We build the deploy repo using Docker for Mac or a Linux machine. Before you start the first build of the deploy repo run through the setup instructions.

Kartotherian is developed on Gerrit, and the old Github repository is abandoned and is no longer used by WMF infrastructure.

Publish library dependencies to NPM before building for deployment

In order to be able to package kartotherian and tilerator, the process to bump the version and publish to npm is crucial, therefore you should have permission to publish packages in the @wikimedia npm organization. The version bump occurs using manually package by package or using the lerna dependency manager.

Building for deployment

Kartotherian and Tilerator are deployed according to the standard process for deploying Wikimedia Node.js services.

Update and build the deployment repo

Sync the code and deploy repos with current master.

cd ~/path/to/kartotherian/deploy
git status
git reset --hard origin/master
git checkout .
git clean -fd
git checkout master
git pull
git --no-pager log --decorate -n 1
git submodule update --init
cd ./src
git checkout master
git pull
cd ..
git add src
git branch
git status

At this point, commit any updates to the src submodule.

If using Docker for Mac start the Docker daemon by clicking on the Whale icon in the menu bar. (Should work automatically on Linux.) Run the tests in Docker and build the new commit for the deploy repo:

./server.js build --deploy-repo --force

And push to Gerrit:

git review

Deploy to Beta Cluster

The steps are similar to #Deploy to Production but using different machines, of course.

To deploy:

ssh deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud

scap deploy --environment beta "`git log --pretty=format:'%s' -n 1`"

To verify service deployment you can ssh into deployment-maps-master01.deployment-prep.eqiad1.wikimedia.cloud.

Use #wikimedia-releng (instead of #wikimedia-operations) to see if there are issues. You may want to log manually in this channel using !log until phab:T156079 is resolved. This logs to the Releng team's Server admin log

Deploy to Production

Maps/v2/Common tasks#Deploy new code

Deploying services

Refer to Services/Deployment for general instructions.

In most cases, Kartotherian and Tilerator should be deployed together, to ensure that all dependencies (in particular, styles) are in sync.

Monitoring

To watch logs on the maps server, a custom script has been included:

tail-kartotherian -f