Swift/Preliminary test plan
< Swift
The "Swift" project is Current as of 2012-04-01. Owner: Bhartshorne. See also RT:1384
This page documents a preliminary test plan to get a basic handle on how swift performs, as well as a battery of functional tests to verify swift is working correctly.
Background information
Some statistics from ms5, to be used as a starting point for the tests
- How many images per second does ms5 serve? about 110/s at peak (from ganglia
- How many requests per second does ms5 send back to the image scalers? about 50/s at peak (from ganglia
- How much storage does ms5 have? 10T, 8T used. (ms7: 27T, 18T used)
- What is the rate of disk consumption on ms5? Historic: 0.3T/mo. Current: 1.2T/mo (ms7: 1T/mo)
- How long does it take to request 1000 images on ms5 (sequentially)?
- note - this test is performed during normal load, so it's actually 1000 images plus regular load
- How long does it take to request 1000 images on ms5 (with 20 clients)?
- note - this test is performed during normal load, so it's actually 1000 images plus regular load
Test platforms
First iteration
three machines using high performance misc servers
- X CPU
- Y RAM
- 2 disks
two machines are configured as proxy machines, three as storage nodes (the two proxy servers are also storage nodes)
Second iteration (proposed)
separate proxy nodes from storage nodes - five high performance misc servers (two proxy, three storage)
production config (proposed)
- three proxy nodes - high performance misc
- 6 storage nodes - ES hardware (grow to 10 over 1yr)
Tests
speed
a list of 30,000 images (pulled from a 3min tcpdump off ms5) is at fenari:~ben/swift/list-of-images. This can serve as input to performance tests.
(30,000 chosen because peak load on ms5 is ~100qps == 6000qpm. 5 minutes =~ 30k.)
- How many images per second can swift serve?
- How long does it take to request 30000 images on swift (sequentially)?
- How long does it take to request 30000 images on swift (with 20 clients)?
- How does swift maintenance (rebalancing, etc) affect performance? (fail out a disk and retest 1000 image retrieval)
functional against swift directly
- upload file
- retrieve file
- rename file
- delete file
functional through mediawiki - thumbnails
- request a thumb that exists on swift
- request a thumb that doesn't exist on swift (but whose original does exist), watch it get created and stored
- re-request, verify it comes from local disk
- request a thumb for an original image that doesn't exist, verify correct error handling
- purge the image cache, verify the image gets deleted from swift
functional through mediawiki - originals
- upload file
- upload new version of file
- revert to previous version of file
- rename file
- delete file