Another look at amazon EBS storage performance

Lately I've gave up bonnie++ in favor of iozone. I've done lot of performance checks on hardware I've got in my hands. That also means that I have to check iozone performance on my amazon instances.

Since tested instance was t1.micro, 1Gb test file was enough to avoid false cache results.

So command run was:

./iozone -a -i0 -i1 -i2 -r4k -O -n1g -g1g

which run:
-a automode
-i0 -i1 -i2 runs selected tests, which are write, rewrite, read, reread, random write and random read,
-r4k is operation size (I/O size) which was selected to fit ext4 block size,
-O gives results in operations per second (not MB/s),
-n -g sets minimum and maximum file size (which is 1GB in this example).

And results were:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
1048576 4 4175 7813 21560 26221 1908 3162

For comparison results for 6*SATA 7200 rpm soft raid10 with ext4:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
1048576 4 53206 52767 53417 54941 162 1437

and SSD OCZ Vertex3 with ext4:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
10485760 4 164282 139274 97570 97725 5985 24186

So write and rewrite speeds are very low for that block device (with ext4 filesystem). 4k-8k iops for sequential operations is very low considering 6*SATA disk software raid gets 53k for both, and vertex3 ssd gets 164k for write and 139k for rewrite.
Read and reread between 21k-26k, again 6*SATA soft raid10 gets 53k-54k for those and vertex3 gets 97k for both.
But last measured value is random read and write, which for EBS instance storage is here 1908 iops (read) and 3162 iops (write).
6*SATA software raid10 gets 162!! random reads and 1437 random writes.
SSD Vertex3 gets 6k for random read and 24k for random write.
So for random operations aws EBS storage is much better than 6 disk raid10 but still slower than single ssd budget drive.
But EBS instance storage sucks in sequential operations.

Another test I've run was the same, except that all read and write operations was direct ones (O_DIRECT in this case). I've simply added -I switch for iozone run:

./iozone -a -i0 -i1 -i2 -r4k -O -n1g -g1g -I

And results were:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
1048576 4 302 317 1823 1869 1935 175

For comparison results for 6*SATA 7200 rpm soft raid10 with ext4:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
1048576 4 9197 9588 12182 11798 127 515

and SSD OCZ Vertex3 with ext4:

                                   random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
10485760 4 10401 11546 11980 12143 4398 4030

Those sequential values 302/317 for write and 1823/1869 for read are very low considering 6*SATA software raid10 gets 9100/9500 and 12000/11800 for those and vertex 3 gets 10500/11500 and 12k/12k for read operations.
But random iops are 1935 for read and 175 for writes. 6*SATA soft raid10 gets 127 and 515 for those, and vertex3 gets 4400 for random reads and 4000 for random writes (all operations are direct, which means that are commited to some hardware layer).

So looks like for direct I/O in sequential writes EBS storage is 30 times slower than 6*SATA software raid10 and vertex3. And for sequential reads is about 6 times slower from both. Strange but as you can see in direct operations ssd vertex3 is about equal to 6*SATA 7200rpm software raid10.
For random read operations SATA software raid10 gets bitten up by EBS storage (is like 15 times slower) and vertex3 is like 2 times faster than EBS. EBS is 3 times slower in direct random writes than SATA soft raid10 and 23 times slower than single ssd device.

So sequentials are always lame for EBS especially direct ones (but I don't care too much about sequential operations for my t1.micro instance). Randoms are quite amazing (especially writes) considering price of that storage. But random direct writes (which some databases do) are still slow for EBS instance storage.

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Main menu

Article | by Dr. Radut