Jump to content

Obsolete:Tools/Overview

From Wikitech

This page provides an overview on available Tool Labs resources. For detailed information see: Help:Tool Labs

This page contains historical information. It may be outdated or unreliable.

System resources

Bastion hosts

Hostname IP pub IP priv Type SSH Fingerprint Purpose Comment
tools-login.wmflabs.org 208.80.155.130 10.68.16.7 m1.medium ssh fingerprint tools users login only
tools-dev.wmflabs.org 208.80.155.132 10.68.16.8 m1.medium ssh fingerprint tools users compilation / interactive tasks
bastion-eqiad.wmflabs.org 208.80.155.129 10.68.16.5 m1.small ssh fingerprint labs/instance users
bastion2-eqiad.wmflabs.org 208.80.155.153 10.68.16.66 m1.small ssh fingerprint labs/instance users
bastion3-eqiad.wmflabs.org 208.80.155.154 10.68.16.67 m1.small ssh fingerprint labs/instance users
bastion-restricted-eqiad.wmflabs.org 208.80.155.155 10.68.16.68 m1.small ssh fingerprint opsens & celebs

Proxies

Hostname IP pub IP priv Type SSH Fingerprint Purpose Comment
tools-webproxy 208.80.155.131 10.68.16.4 m1.medium Nginx Web Proxy for Tools Nginx 1.5.0 – tools.wmflabs.org/*
dynamicproxy-gateway 208.80.155.156 10.68.16.65 m1.small Nginx Web Proxy for Instances Nginx 1.5.0 – *.tools.wmflabs.org

Sun Grid Engine (SGE)

Hostname IP pub IP priv Type SSH Fingerprint Purpose Comment
tools-master – 10.68.16.9 m1.small Grid master node
tools-shadow – 10.68.16.10 m1.small Grid master backup node
tools-submit – 10.68.17.1 m1.small Cron submit node
tools-exec-01 208.80.155.140 10.68.16.30 m1.large task exec node
tools-exec-02 208.80.155.141 10.68.16.31 m1.large task exec node
tools-exec-03 208.80.155.142 10.68.16.32 m1.large task exec node
tools-exec-04 208.80.155.143 10.68.16.33 m1.large task exec node
tools-exec-05 208.80.155.144 10.68.16.34 m1.large task exec node
tools-exec-06 208.80.155.145 10.68.16.35 m1.large task exec node
tools-exec-07 208.80.155.146 10.68.16.36 m1.medium task exec node
tools-exec-08 208.80.155.147 10.68.16.37 m1.large task exec node
tools-exec-09 208.80.155.152 10.68.17.64 m1.large task exec node
tools-exec-10 208.80.155.186 10.68.17.65 m1.large task exec node
tools-exec-11 208.80.155.187 10.68.17.144 m1.xlarge task exec node
tools-exec-12 208.80.155.188 10.68.17.166 m1.xlarge test node ubuntu-14.04-trusty
tools-exec-13 208.80.155.189 10.68.17.184 m1.xlarge task exec node
tools-exec-cyberbot 208.80.155.255 10.68.16.39 m1.medium task exec node Cyberpower678
tools-exec-gift 208.80.155.255 10.68.16.40 m1.medium task exec node Giftpflanze
tools-exec-wmt 208.80.155.255 10.68.16.41 m1.medium task exec node Wikimedia tasks
tools-webgrid-01 208.80.155.255 10.68.16.28 m1.xlarge Web Server node
tools-webgrid-02 208.80.155.255 10.68.17.9 m1.xlarge Web Server node
tools-webgrid-03 208.80.155.255 10.68.17.123 m1.xlarge Web Server node
tools-webgrid-04 208.80.155.255 10.68.17.174 m1.xlarge Web Server node
tools-webgrid-tomcat 208.80.155.255 10.68.16.29 m1.xlarge Java Servlet and JavaServer node

Volumes

Volume Size Purpose Comment Browse
/home 30 TB   users home directories Files
/data/project tools directories Files
/shared/ directory for shared resources Files
/data/scratch 7.3 TB   temporary user files no redundancy Files
/public/backups 20 TB   backups scheduled by sysops Files
/public/dumps 9.1 TB   public Wikimedia datasets Files
/public/keys    930 MB   ssh keys Files

Database Servers

Hostname Type Purpose List Comment Physical host Storage Memory Monitor
tools-redis Redis 2.6 Fast key-value store for caching/queueing – Port: 6379 VM 7 GB 7 GB
tools-db MySQL 5.5 Tools data list databases Binary Log enabled labsdb1005 8 TB 32 GB
PostgreSQL 9.1 OSM data list databases labsdb1004
s1.labsdb MariaDB 10.0 Labs replica slice list databases enwiki labsdb1001 7 TB 180 GB Ganglia
s2.labsdb MariaDB 10.0 Labs replica slice list databases nl-, pl-, it-, zhwiki labsdb1002 7 TB 180 GB Ganglia
s4.labsdb commonswiki
s5.labsdb dewiki, wikidatawiki
s3.labsdb MySQL 5.5 Labs replica slice list databases 816 other wikis labsdb1003 7 TB 180 GB Ganglia
s6.labsdb MySQL 5.5 Labs replica slice list databases fr-, ja-, ruwiki
s7.labsdb MySQL 5.5 Labs replica slice list databases centralauth, metawiki

Tools shared resources

Resources provided by users/tools/projects for developers

Databases

Resource Purpose Maintainer Docs
meta_p Metadata for all wikis hosted in labs (on every slice) Coren List content
s51892_toolserverdb_p Metadata like toolserver.namespace (on every slice) Nosy bug 48625
CatGraph In-memory graph database for Wikipedia category structure Johannes_Kroll_(WMDE) CatGraph

Applications / Frameworks

Resource Purpose Maintainer Docs
Intuition Framework to provide internationalization ( i18n ) for tools Krinkle documentation
Pywikibot This framework is a collection of tools made to fit the maintenance need on Wikipedia Legoktm et al. mw:Manual:Pywikipediabot
Parsoid A bidirectional wikitext parser and runtime. Converts between wikitext and HTML/XML DOM with RDFa Various mw:Parsoid
MWParserFromHell Python package that provides an easy-to-use and outrageously powerful parser for MediaWiki wikicode Earwig et al. Documentation

API's

Resource Purpose Maintainer Docs
API Sandbox Page to experiment with the MediaWiki web service API Various mw:API:Main_page
Supercount Supercount (X!'s Edit Counter) API, user & article analysis Cyberpower678 Documentation

Scripts / Guides / Tools

Resource Purpose Maintainer Docs
MySQL queries MySQL sample queries to pull data from replicated databases Various –
OAuth-hello-world Simple "Hello world" program to show how to use OAuth Anomie mw:Extension:OAuth
SQL Optimizer Tool to check custom queries with MySQL optimizer Hedonil bug 48875
Quarry Tool to run SQL queries from web frontend YuviPanda

Misc

Resource Purpose Maintainer Docs
Paste Wikimedia pastebin Petrb –
Etherpad Wikimedia etherpad Various Etherpad
UTF-8 encoding table Comprehensive UTF-8 encoding table with Unicode codepoints and HTML encoding web –
Static This project hosts several heavily used static files Ireas –

Virtualization

Virtualization Environment

OpenStack Compute (Nova)

Virtualization hosts

Hostname CPU RAM Type Monitor Comment
virt1001 24 170 GB Cisco UCS C250 M1 / 2x Intel Xeon CPU X5650 @ 2.67GHz Ganglia
virt1002 24 170 GB Cisco UCS C250 M1 / 2x Intel Xeon CPU X5650 @ 2.67GHz Ganglia
virt1003 24 170 GB Cisco UCS C250 M1 / 2x Intel Xeon CPU X5650 @ 2.67GHz Ganglia
virt1004 24 170 GB Cisco UCS C250 M1 / 2x Intel Xeon CPU X5650 @ 2.67GHz Ganglia

Virtual box types

(also referred to as "instances" or "boxes")

Type vCPU RAM Hypervisor OS default Comment
m1.small 1 x 2.6 GHz 2 GB KVM Ubuntu 12.04-precise (3.2-virtual x86_64)
m1.medium 2 x 2.6 GHz 4 GB KVM Ubuntu 12.04-precise (3.2-virtual x86_64)
m1.large 4 x 2.6 GHz 8 GB KVM Ubuntu 12.04-precise (3.2-virtual x86_64)
m1.xlarge 8 x 2.6 GHz 16 GB KVM Ubuntu 12.04-precise (3.2-virtual x86_64)

Contact & Communication

Contact

Name IRC Default location TZ Role Comment
Marc-André Pelletier Coren Quebec, Canada UTC-5 root admin Tool Labs ringmaster
Andrew Bogott andrewbogott Minneapolis, US UTC-6 root admin Labs OpenStack, instances & stuff
Petr Bena petan Czech Republic UTC+1 admin MediaWiki, huggle, bots & stuff
Tim Landscheidt scfc_de Hamburg, Germany UTC+1 admin Shell accounts & stuff
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Obsolete:Tools/Overview&oldid=1986596"