Tag Archives: optimising

MongoDB Journalling

A quick one if you are wondering about MongoDB and Journalling (especially on Debian/Ubuntu from the MongoDB repo) .

By default mongodb will generate 3gig of journal files in /var/lib/mongodb/journal this is far from ideal if you are running on a Rackspace CloudServer or smaller VPS.

There are two easy solutions:

  • If you are running as part of a ReplicaSet then you can choose to go without journalling altogether with the
    nojournal=true

    option in mongodb.conf.

  • Alternatively you can request that MongoDB uses smaller preallocated files for journalling use the
    smallfiles=true

    option in mongodb.conf.

To also reduce some usage in your datafiles there is the also the

noprealloc = true

option, however I don’t see this as particularly useful considering the preallocation starts pretty small and only grows as your data does.

Case Study: Optimising a Cloud Application

I was recently brought in to examine the infrastructure of a small startup. This wasn’t anything really special, I do it quite often for various reasons. What was different was that they didn’t have issues with scaling out particularly – they had that working well with their shared nothing web application and mongodb backend. What they were having issues with was their infrastructure costs.

I normally work on through a 6 step process that has been built up over time –

  1. Monitor and gather stats/measure the problem,
  2. Standardise on a reference architecture,
  3. Add configuration management and version control,
  4. Start to define a playbook of how to do things (like up/downscale or provision new machines and clusters) and start to automate them,
  5. Bring everything to reference architecture/consolidate underutilised servers and eliminate unused infrastructure,
  6. Consider architecture changes to make it more efficient.
  7. …and repeat

I will take you through a case study showing how this process was used to lower their monthly costs. Names and details have been changed in places to protect the guilty… 😉 Continue reading