Disk IO: EC2 vs Mosso vs Linode

Recently I read an interesting idea on amazon EC2 forum about Raid0 strip on EBS to improve disk access performance. So I am very curious to know whether this idea actually works. Technically it is also possible to setup a raid system on Linode(referral link) as well, but it will be backed by same hardware (so I didn’t test this idea).

In this test I used bonnie++ 1.03e with direct IO support. These 3 VPS have slightly different configure. Mosso server has 256MB ram with 2.6.24 kernel and 4 AMD virtual cores. Lindoe vps has 360MB ram with custom built 2.6.29 kernel and 4 intel virtual cores. EC2 high-cpu medium instance has 1.7GB ram with 2.6.21 kernel and 2 intel virtual cores.

Here is the raw test result. On each VPS I run bonnie++ 3 times, then use median of 3 tests as the final result. The summary result is unweighted average value of different columns. Due to the memory size difference, I used different test file size. The EBS I used here is 4x10GB raid0.

In this table, -D means that test run with Direct IO option. The best results are highlighted. Direct IO test on EBS taking forever, so I didn’t finish that test.

.

Write (MB/s)

Read (MB/s)

Seek (#/s)

.

Mosso -D

32.4

52.9

219

.

Mosso

56.9

52.6

225

.

Linode -D

37.7

76

187

.

Lindoe

41.5

76.1

201

.

EC2 -D

32.4

50.7

220

.

EC2

18.9

39.2

210

.

EBS Raid0

52.4

23.1

1076

In this chart, I used logarithm scales and shifted origin in order to show the relative difference between them. So the column value does not reflect the real test results. Higher value is better.

Disk IO Chart

Conclusions: There is no clear winner in this test. Each VPS has the their high score in different category. Only one thing is clear, O_Direct does not work very well on EBS. Due to the nature of VPS, the Disk IO test is very unreliable. The performance I show here is not repeatable and may not reflect the true disk performance.

  • I’d be interested to see how this compares to performance on non-virtual server hardware.

    • Yejun

      @Jon Topper
      These numbers are very close to performance of a single commodity hard driver. EBS’s IOps is close to a small size highy quality disk array. With a real high quality array, these number can be easily doubled or trippled.

  • Peter

    What were the numbers for a regular EBS (non-raided)? Or is that EC2 result on EBS?

    • Yejun

      @Peter
      I didn’t test regular EBS.

  • Pingback: time to blog » Blog Archive » Unravelling the Cloud()

  • X

    I think linode is the best option for VPSs, but the second best is Mosso. Disk I/O are OK on both. EC2 is good, but too expensive, only for companies, start-ups, etc. By the way, I played around with your yyang nginx AMI for a few minutes the other day, works OK but I prefer to install everything by myself on a naked default Ubuntu install.

  • james

    Unfortunately these numbers don’t really mean much. Aside from what else is happening on the machines on website hosting, in some cases note that writing was showing up faster than reading, which really means that it’s just writing things to disk buffers. For this test to have been meaningful there should have been an fsync at the end of the run. The direct-i/o measurements should be closer to apples-to-apples.

    When I’m doing I/O benchmarking I start every run with a clean reboot and follow an exact set of steps from bootup to start of the benchmark. It’s almost impossible to simulate that on a VPS.

  • Andy

    thanks a lot for posting the chart here.

  • What about diminishing disk I/O as the VMs are more utilised? Are these tests not only approriate to that moment in time? Apples to apples, I agree with James and we are going to see a few virtual infrastructure providers caught out by disk trashing slowing up their client appliances.

    I wonder what the EC2 VM to real ratio?

  • john

    That is very helpful for every one who use non-virtual server hardware.

    cisco course

  • sana

    Corporate Identity Logo Design

    Professionally
    logos and corporate identities can play critical role to a business success.
    That’s why ADAM IT Solutions  provide
    unique logo and corporate identities to our customers, which can uniquely
    represent them all over the world.

    Your logo… your Specific
    and memorable ID

    Your logo is the first
    image with your customers.

     An icon or colors that represent everything behind
    each company and the message that it transmits. ADAM IT Solutions creative
    design team will be able to provide you with the most unique, sophisticated and
    modern logo that will portray your company’s history, services and  Impression.

    Sana

    0561422123