Jump to content

Performance/Runbooks/Analyze memcached

From Wikitech

memkeys

This is like "top" but for memcached traffic. Real-world examples:

# from an appserver, see local requests to the "on-host tier" instance
mw0011$ sudo memkeys -i lo -p 11210

# from an appserver, see outgoing requests to the memcached cluster
mw0011$ sudo memkeys -i lo -p 11213

# from a memcached server, see all incoming requests to that server
mc0032$ sudo memkeys -i eth0
mc0046$ sudo memkeys -i ens2f0np0

Example output:

# memcached key                                                     calls   objsize req/sec
# WANCache:global:rdbms-server-readonly:deployment-db07:enwiki:ǀ#ǀv 3581    53      12.88
# WANCache:global:centralauth-user:44df0d9a65e7d3fad8664f4b2dc2ǀ#ǀv 2973    1178    10.40
# WANCache:enwiki:messages:en:hash:v1ǀ#ǀv                           2954    124     10.19

Upstream:

cachedump

Inspied by @elukey via T203786#4999052.

krinkle@mc1048:~$
echo "stats cachedump 1 0" | nc -q 1 localhost 11211 > dump-01.txt
echo "stats cachedump 16 0" | nc -q 1 localhost 11211 > dump-16.txt
echo "stats cachedump 26 0" | nc -q 1 localhost 11211 > dump-25.txt
echo "stats cachedump 32 0" | nc -q 1 localhost 11211 > dump-16.txt
$ head cache-01.txt

ITEM WANCache:global:SqlBlobStore-blob:commonswiki:tt%3A429993772|#|v [756 b; 1660458645 s]
ITEM WANCache:global:revision-row-1.29:frwiki:6035826:144772360|#|v [790 b; 1660264028 s]
ITEM WANCache:commonswiki:MessageBlobStore:ext.visualEditor.language:nl:2b6785d…|#|v [724 b; 1660719066 s]
$ cat dump-*.txt | fgrep WANCache | cut -d " " -f 2| cut -d ':' -f3 | sort | uniq -c | sort -rn

  18669 SqlBlobStore-blob
   8855 filerepo-file-foreign-description
   1735 preprocess-hash
    596 getExtendedMetadata
    254 MessageBlobStore

Upstream:

See also