Tag Archives: optimising

MongoDB Journalling

A quick one if you are wondering about MongoDB and Journalling (especially on Debian/Ubuntu from the MongoDB repo) .

By default mongodb will generate 3gig of journal files in /var/lib/mongodb/journal this is far from ideal if you are running on a Rackspace CloudServer or smaller VPS.

There are two easy solutions:

  • If you are running as part of a ReplicaSet then you can choose to go without journalling altogether with the

    option in mongodb.conf.

  • Alternatively you can request that MongoDB uses smaller preallocated files for journalling use the

    option in mongodb.conf.

To also reduce some usage in your datafiles there is the also the

noprealloc = true

option, however I don’t see this as particularly useful considering the preallocation starts pretty small and only grows as your data does.

Case Study: Optimising a Cloud Application

I was recently brought in to examine the infrastructure of a small startup. This wasn’t anything really special, I do it quite often for various reasons. What was different was that they didn’t have issues with scaling out particularly – they had that working well with their shared nothing web application and mongodb backend. What they were having issues with was their infrastructure costs.

I normally work on through a 6 step process that has been built up over time –

  1. Monitor and gather stats/measure the problem,
  2. Standardise on a reference architecture,
  3. Add configuration management and version control,
  4. Start to define a playbook of how to do things (like up/downscale or provision new machines and clusters) and start to automate them,
  5. Bring everything to reference architecture/consolidate underutilised servers and eliminate unused infrastructure,
  6. Consider architecture changes to make it more efficient.
  7. …and repeat

I will take you through a case study showing how this process was used to lower their monthly costs. Names and details have been changed in places to protect the guilty… 😉 Continue reading