Tag Archives: cloud

A look at the new rackspace cloud

I have been very lucky to be included in the first rollout of the new rackspace cloud and while it has been live (to me) for a few days, I have only just had a few minutes to have a play with it. Continue reading

Mongo Benchmarking #1

These aren’t perfect benchmarks – far from it in fact – but I just wanted to get a rough idea of the relative tradeoffs between fsync and safe over normal unsafe writes…

Time taken for 10,000 inserts – (no indexes, journalling on MongoDB 2.0.1 on Debian (Rackspace CloudServer 256Meg))

default      - time taken: 0.2401921749115 seconds
fsync        - time taken: 358.55523014069 seconds
safe=true    - time taken: 1.1818060874939 seconds

Edit: and for journalling disabled and smallfiles=true in mongo.conf

default      - time taken: 0.15036606788635 seconds 
fsync        - time taken: 34.175970077515 seconds 
safe=true    - time taken: 1.0593159198761 seconds

The results aren’t perfect, but do show how big the difference is…


$mongo = new Mongo();
$db = $mongo->selectDB('bench');
$collection = new MongoCollection($db,'bench');

$start = microtime(TRUE);
for ($i=0;$i<10000;$i++) {
  $collection->insert(array('data' => sha1(rand()) ));
$end = microtime(TRUE);
echo 'default      - time taken: '.($end-$start)." seconds \n";

$start = microtime(TRUE);
for ($i=0;$i<10000;$i++) {
  $collection->insert(array('data' => sha1(rand()) ),array('fsync' => true));
$end = microtime(TRUE);
echo 'fsync        - time taken: '.($end-$start)." seconds \n";

$start = microtime(TRUE);
for ($i=0;$i<10000;$i++) {
  $collection->insert(array('data' => sha1(rand()) ),array('safe' => true));
$end = microtime(TRUE);
echo 'safe=true    - time taken: '.($end-$start)." seconds \n";

I’m not sure that the existing number of records will make a massive amount of difference besides through the pre-allocation of files which we have little control of anyway – but it doesn’t look like there is an increase between runs even when there are a lot of entries… (perhaps we’d see more with indexes enabled)

Each run will add an extra 20,000 entries into the collection with little perceptable slowdown.

root@test:/var/www/test# php bench1.php
default      - time taken: 0.53534507751465 seconds
safe=true    - time taken: 1.2793118953705 seconds
root@test:/var/www/test# php bench1.php
default      - time taken: 0.203537940979 seconds
safe=true    - time taken: 1.2887620925903 seconds
root@test:/var/www/test# php bench1.php
default      - time taken: 0.22933197021484 seconds
safe=true    - time taken: 1.6565799713135 seconds
root@test:/var/www/test# php bench1.php
default      - time taken: 0.19606184959412 seconds
safe=true    - time taken: 1.5315411090851 seconds
root@test:/var/www/test# php bench1.php
default      - time taken: 0.2510199546814 seconds
safe=true    - time taken: 1.2419080734253 seconds

It is hard testing on a cloud server as you are at the mercy of other users impacting the available bandwidth and processor utilisation, but you can at least see trends. I hope this has been enlightening and I hope to expand on this in future…

Edit: for the one person that asked me about storage efficiency… here goes…

 > db.bench.stats()
    "ns" : "bench.bench",
    "count" : 140001,
    "size" : 10640108,
    "avgObjSize" : 76.00022856979594,
    "storageSize" : 21250048,
    "numExtents" : 7,
    "nindexes" : 1,
    "lastExtentSize" : 10067968,
    "paddingFactor" : 1,
    "flags" : 1,
    "totalIndexSize" : 4562208,
    "indexSizes" : {
        "_id_" : 4562208
    "ok" : 1

So based on this… we can work out that size/storageSize = approx 50% efficiency… so MongoDB on this dataset is using about the same again for the data storage.

If we add in indexes size/(storageSize+totalIndexSize) then the result is only about 41% efficient. I think this is a reasonable tradeoff for the raw speed it gives personally…

Case Study: Optimising a Cloud Application

I was recently brought in to examine the infrastructure of a small startup. This wasn’t anything really special, I do it quite often for various reasons. What was different was that they didn’t have issues with scaling out particularly – they had that working well with their shared nothing web application and mongodb backend. What they were having issues with was their infrastructure costs.

I normally work on through a 6 step process that has been built up over time –

  1. Monitor and gather stats/measure the problem,
  2. Standardise on a reference architecture,
  3. Add configuration management and version control,
  4. Start to define a playbook of how to do things (like up/downscale or provision new machines and clusters) and start to automate them,
  5. Bring everything to reference architecture/consolidate underutilised servers and eliminate unused infrastructure,
  6. Consider architecture changes to make it more efficient.
  7. …and repeat

I will take you through a case study showing how this process was used to lower their monthly costs. Names and details have been changed in places to protect the guilty… 😉 Continue reading

Rackspace Huddles

I’m not a Rackspace expert – far from it, however I do use the Rackspace cloud often, both as a personal customer, a business customer and for various clients. I will try to lay out how I believe it all works and how this impacts you the end user.

From time to time ‘huddles’ get mentioned. A huddle is essentially a cluster of physical servers all working together to provide cloud servers service. The Rackspace cloud consists of hundreds of separate huddles all under one administration system.

Generally this is unimportant to end users. Rackspace while not exactly secretive about it are a little cagey about them as it starts to show some of the details about what goes on behind the scenes and ruins the illusion of completely scalable infrastructure.

Each area (London, Chicago or Dallas-Fort Worth) consists one or more data centres for cloud servers, each data centre will have one or more huddle. According to The Register each huddle is

One Huddle consists of 210 servers across 14 cabinets. The basic server spec is a dual Hex core, 12GB of RAM and three 500GB hard disks on a RAID5 configuration.

Which I believe is slightly inaccurate. The cloud servers go up to 15872Meg of space, this obviously can’t be accommodated on 12Gig of RAM. Also, the server one of my cloud servers is sitting on according to the cpuinfo on a Chicago Cloud-Server is

model name    : Quad-Core AMD Opteron(tm) Processor 2374 HE
cpu MHz        : 2176668.674

So, a 2.2Ghz Quad core Opteron rather than the dual Hex core.

model name    : Quad-Core AMD Opteron(tm) Processor 2374 HE
cpu MHz        : 2409004.973

and 2.4Ghz Quad-core Opteron on one of my London cloud-Servers.

Not a big issue, I’m probably just on older hardware.

So what does a huddle mean to the end user? Generally nothing. It only seems to be important on two occasions. The first is when there is a fault with the control-plane of the huddle. If your servers are all on the same huddle then they may be affected at the same time. I don’t know how common this is but for redundancy, you are better spread over different huddles.

Rackspace do say to use one of the other zones (LON/ORD/DFW) for redundancy and this might be a better idea if your application is mission critical as network issues (or massive disasters) could knock out multiple huddles at once in a single datacentre. The problem with this is that different datacentres don’t share a common servicenet, so for some applications spreading your servers across different huddles in the same zone is the best you can get…

The second time it is important is when you need to use a shared IP. Shared IPs can only be shared with other servers in the same huddle. This isn’t as much of a problem as it sounds as if you use the API to launch the servers then you need to launch (except the first server) into a shared IP group, which will be on the same huddle automagically. However, via the web interface there doesn’t seem to be any way to specify it besides opening a chat with support and letting them toggle your default huddle and adding shared IPs for you.

Much of this is guess-work based on the available documentation and conversations with support from time to time. If you have any additional information or corrections, please get in touch mike -at- technomonk . com