Site Reliability Team Lead at News UK

View My GitHub Profile

23 November 2011

Rackspace Huddles

by Mike

I’m not a Rackspace expert - far from it, however I do use the Rackspace cloud often, both as a personal customer, a business customer and for various clients. I will try to lay out how I believe it all works and how this impacts you the end user.

From time to time ‘huddles’ get mentioned. A huddle is essentially a cluster of physical servers all working together to provide cloud servers service. The Rackspace cloud consists of hundreds of separate huddles all under one administration system.

Generally this is unimportant to end users. Rackspace while not exactly secretive about it are a little cagey about them as it starts to show some of the details about what goes on behind the scenes and ruins the illusion of completely scalable infrastructure.

Each area (London, Chicago or Dallas-Fort Worth) consists one or more data centres for cloud servers, each data centre will have one or more huddle. According to The Register each huddle is

One Huddle consists of 210 servers across 14 cabinets. The basic server spec is a dual Hex core, 12GB of RAM and three 500GB hard disks on a RAID5 configuration.

Which I believe is slightly inaccurate. The cloud servers go up to 15872Meg of space, this obviously can’t be accommodated on 12Gig of RAM. Also, the server one of my cloud servers is sitting on according to the cpuinfo on a Chicago Cloud-Server is

model name    : Quad-Core AMD Opteron(tm) Processor 2374 HE
cpu MHz        : 2176668.674

So, a 2.2Ghz Quad core Opteron rather than the dual Hex core.

model name    : Quad-Core AMD Opteron(tm) Processor 2374 HE
cpu MHz        : 2409004.973

and 2.4Ghz Quad-core Opteron on one of my London cloud-Servers.

Not a big issue, I’m probably just on older hardware.

So what does a huddle mean to the end user? Generally nothing. It only seems to be important on two occasions. The first is when there is a fault with the control-plane of the huddle. If your servers are all on the same huddle then they may be affected at the same time. I don’t know how common this is but for redundancy, you are better spread over different huddles.

Rackspace do say to use one of the other zones (LON/ORD/DFW) for redundancy and this might be a better idea if your application is mission critical as network issues (or massive disasters) could knock out multiple huddles at once in a single datacentre. The problem with this is that different datacentres don’t share a common servicenet, so for some applications spreading your servers across different huddles in the same zone is the best you can get…

The second time it is important is when you need to use a shared IP. Shared IPs can only be shared with other servers in the same huddle. This isn’t as much of a problem as it sounds as if you use the API to launch the servers then you need to launch (except the first server) into a shared IP group, which will be on the same huddle automagically. However, via the web interface there doesn’t seem to be any way to specify it besides opening a chat with support and letting them toggle your default huddle and adding shared IPs for you.

Much of this is guess-work based on the available documentation and conversations with support from time to time. If you have any additional information or corrections, please get in touch mike -at- technomonk . com