Maps/Tile storage

Setting up Cassandra tile storage

• On first start of a Cassandra node, it is necessary to clean the data directory rm -rf /srv/cassandra/*. See Cassandra documentation for more details.
• Also remember to touch /etc/cassandra/service-enabled before starting Cassandra on each node. See https://phabricator.wikimedia.org/T214166

• A new Maps cluster should have only one node in its Cassandra seeds configuration and so will not connect to other hosts to bootstrap when the service starts. See this example hiera configuration for a new single-node cluster.

• change the cassandra super user password to match the one configured in private repo using cqlsh:

 cqlsh `hostname -f` -u cassandra
 Password: cassandra
 ALTER USER cassandra WITH PASSWORD '<password>';

• Setup of user access / rights for cassandra

 cat /usr/local/bin/maps-grants.cql | cqlsh --cqlshrc=/etc/cassandra/cqlshrc `hostname -f` -

• Setup replication of Cassandra system_auth according to documentation. We simply set a replication factor of 2 in maps at present: ALTER KEYSPACE system_auth WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '2'};
• Very important point to note is there's a catch when setting up replicator factor for system_auth keyspace on cassandra. We recently discovered that increasing replication factor according to docs causes outage on cassandra. See T214434 and T157354. Also this: Incident documentation/20190122-maps.

• Initial creation of cassandra keyspace: To prevent accidental modification of schema, Tilerator source configuration does not allow to create schema by default. The sources file used by tilerator / kartotherian is configured in /etc/(kartotherian|tilerator|tileratorui)/config.yaml, look for the sources: key. This is a reference to a sources file in the kartotherian / tilerator source directory. For example /srv/deployment/tilerator/deploy/src/sources.prod2.yaml.

The easiest way to create a new keyspace is to run Tilerator with a custom sources file, which instructs tilerator to create the missing keyspace. For example, create a temporary file, e.g. /home/yurik/my-source-file with the following configuration (change v4 with the keyspace declared in the sources configuration file):

v4:
  uri: cassandra://
  params:
    keyspace: v4
    cp: {var: cassandra-servers}
    username: {var: cassandra-user}
    password: {var: cassandra-pswd}
    # These parameters are only used if keyspace needs to be created:
    repfactor: 4
    durablewrite: 0
    createIfMissing: true
    copyInfoFrom: {ref: gen}

And run this bash script:

sudo -u tilerator node /srv/deployment/tilerator/deploy/node_modules/@wikimedia/tilerator/scripts/tileshell.js \
  # Use TileratorUI configuration (including password variables)
  --config /etc/tileratorui/config.yaml \
  # Use this sources file instead of the one in the config file
  --source ~/my-source-file.yaml

Bulk Copying

The fastest way to copy a large number of tiles from one source to another is to use a large number of parts and specify saveSolid=true (skips solid tile detection). E.g. to copy all z16 tiles from v3 to v4, use src=v3 dst=v4 zoom=16 parts=60 saveSolid=true