Performance/Synthetic testing/Result storage

Synthetic test result storage

The synthetic test produces test results in the form of HTML, videos, screenshots and browser tracelogs from the tests. These results are useful when you need investigate a performance regression. We used to use S3 on AWS to upload the data, but we pushed a lot and compared to running our own server, S3 was much more expensive. Today we use server at Hetzner with 4 disks (two raids) where we store the data. The data is then served from a nginx instance running on that server.

Server setup

You can login to the server using: ssh USERNAME@116.202.221.47

The data is stored in /data/ and /home/data/ (it's two different disks). The synthetic test server SCP the content to this server.

The data is served using nginx and the setup for the instance it located in /etc/nginx/sites-available/wikimedia.sitespeed.io

Old test results are removed after 35 days using the crontab for root (yep we should change that). It looks like this:

0 10 * * * find /data/* -mtime +35 -exec rm -rf {} \; >/dev/null 2>&1

0 20 * * * find /home/data/* -mtime +35 -exec rm -rf {} \; >/dev/null 2>&1

When other server uploads the data, it uses the user uploader that has access to the both directories that is used for uploading files.