DevOps and the future

DevOpsWe have come to a turning point in the evolution of the DevOps movement.

The big consulting companies have ran with the concept and built big expensive packages aimed at corporations; there are SaaS companies trying to carve out territory based on the tool chain and there are small DevOps consultancies that will go into a company and try to help them adapt their processes to be more DevOpsy.

We have the tooling side pretty much sewn up. This isn’t a bad thing. The problem we now have is the culture side; the people problem. We still are thinking in the concept of traditional organizations; of rigid hierarchies and top-down management and people mostly just being inter-changeable cogs in a giant machine.

We need to address this.

Organizations are getting bigger and more distributed. As this happens the communication overhead increases. Traditional solutions of adding a manager or team leader to a team to act as a single point of contact is starting to become inefficient when it comes to larger organizations. Each level of abstraction adds overhead, reduces transparency and reduces efficiency. We need a different way of working.

I don’t have the answers yet, but I am trying to understand the problem.

10 New year’s resolutions for IT pros

1st of January

Self Improvement is a hard thing to do at the best of times. Putting pressure on yourself to change at the start of a new year is probably not the best of ideas, but we do it anyway (much like choosing to run Windows 8).

So without too much additional preamble here is my list.

  1. Backups – We are all guilty of not taking adequate backups… or often in my case taking adequate backups but not automating them or keeping them organised. This doesn’t need to be a big complex system, it could just be a cron job tar’ing up the important data, encrypting it and uploading it to the cloud.
  2. Improve security – There are still a lot of us choosing to not embrace TLS for their services and with StartSSL and LetsEncrypt offering free certs and PositiveSSL (I get mine via Namecheap) being pretty cheap, there is no excuse. In addition to this why not use this time to start using keypairs for your SSH (Microsoft announced SSH support back in June)  for secure access to your systems.
  3. Stop procrastinating – We all have these little things we have put off; fixing that computer in the corner, cleaning the office, learning a new language or framework or perhaps asking that girl at the coffee shop out. The first step in getting stuff done is to just fscking start it… and then continue doing it. I just wish I could take my own advice…
  4. See the world and meet new people – I know many of us are most comfortable when it is just ourselves and our computers, but if we are to grow as people we need to experience new things and be exposed to new ideas. Perhaps goto a usergroup (I sometimes goto some of the London DevOpsy ones and PHP:East Midlands) or perhaps a conference or two.
  5. Be more polite – We all get stressed from time to time and in our quest for efficiency we can forget how we are coming across. It probably isn’t helped by the fact that many of us communicate better with a machine than face to face. Like any skill we can hone it and become more adept at it.
  6. Be more organised – We often complain that there are not enough hours i the day but conversely know people that seem to manage to accomplish so much. It generally comes down to two things: being more organised and prioritising what is most important.
  7. Spend time with people that matter (and less time with the people that don’t) – As you get older you come to understand some of the decisions you have made in life didn’t always align with your current values. I doubt that when I am old and grey and laying on my deathbed that I will say ‘I wish I had worked more’. More likely I will lay there wishing that I had spent more time with the people that were important to me. Why wait for this time when you can do something about it right now?
  8. Get more sleep – Lack of sleep can affect your health, your stress levels, your effectiveness at work and indirectly the people you live and work with. Do the world a favour and don’t shortchange yourself on your rest.
  9. Eat healthier – Your body works better if it has the right fuel. Over the past couple of years I have come to understand I can’t just keep abusing my body and I am trying to turn my health around a little… unfortunately, I haven’t found this trivial with spending 13 hours a day out of the house and commuting 250miles a day…
  10. Exercise more – Spending hours a day sitting on your arse isn’t ideal from a health perspective. I try to get up at least once an hour and stretch my legs and get out for a walk at lunch – I say try because sometimes this is easier said than done…
  11. (Bonus) Learn to be content with your life – We all have things we regret doing; choices we made; opportunities that we passed up; girl friends we haven’t got over etc. We might also be looking forward to certain events in the future…
    We need to stop living in the past and live in the now. The future hasn’t happened yet and the past is gone and can’t be changed. We need to live in the moment; embrace it and let the past go.

Under the hood of my Hack24 entry

I competed by myself at hack24. I don’t see competing alone as a massive issue personally, as it cuts down communication overhead by a massive amount. 😉

For those that haven’t heard of hackathons before, the concept is fairly simple:

  • Teams of 1-4 people.
  • Multiple challenges (at least in this event)
  • 24 hours to build a solution.
  • Only code you have written during the event (or Open Source libraries/apps) can be used.

It is quite gruelling to compete in these types of events; you have limited time and you need to prioritise your time across features and components carefully so you don’t end up with a complete backend but no frontend or similar.

Generally you omit anything that is not-essential such as login systems, logging or some error checks etc.

I used the event to test some of the ideas and applications (Microservices, RabbitMQ, Riak, Nginx, php-fpm etc) I have been using on larger projects in a smaller time constrained project – and I think the outcome proves that it works.

I am a big fan of microservices; breaking your app into small testable components with fixed interfaces is awesome for quick building and easy debugging – want to test if something is working, inject a message into the input and see what the outputs are. Once working you can mostly treat it as a single component and move onto the next piece. It keeps your mind focused and, at least for me, reduces stress.

I am a long-time PHP coder and so it made sense, when it came to quick prototyping, to use this and the lightweight Slim framework as my language of choice – both for the frontend and all the workers on the backend all running on top of a single Vultr VPS [disclosure: url contains affiliate link].

Screenshot

SMS proxy – my hack24 entry

I broke the project into 4 main parts:

  • Frontend (website, sms callback hook for receiving messages from Esendex (the challenge sponsor))
  • Receiving worker (take messages from callback and lookup entries in Riak and route to next processing step either filter or directly to sending worker)
  • Filter (responsible for transforming the message before routing the message to the sending worker)
  • Sending worker (responsible for sending the message out via Esendex)

I am open-sourcing the code as a learning tool for other people (and perhaps so I can abuse it in later events…)

You are welcome to ask any questions and I will do my best to answer them.

If you want to contact me, please use twitter or email mike @ technomonk . com

Microservices payload design

There are a lot of people talking about microservices at present. I understand it is fashionable and a lot of people are trying to get rich from consulting in the domain, but a lot of the things I hear are just plain wrong or bad practice.

People are just throwing microservices out there with little consideration of any of the basic requirements of operations. No idea of how they will monitor them; no concept of of how they can debug them and no regard as to how they upgrade them.

I came from a dev background and over time migrated into operations. I dislike fragile, hard to debug components.

I have learnt the hard way just how difficult a distributed microservice architecture can be – but it is still (when designed correctly) a better choice than a hulking monolith of code.

I like my payloads to be debuggable. For this reason I like JSON as a interchange format. It isn’t the most efficient representation format but can be easily human-parsed and is supported by most languages.

I like to know how long a request has taken through the system so I put a uuid and microsecond timestamp when every request is first seen. I also like to know how long each step takes so I also append a timestamp and additional id for every process the request passes through.

Doing these in a uniform way allows you to take a payload from anywhere in the system and be able to use the same tools to perform basic analysis.

It can bloat the payload a little, but being able to see at a glance how quickly each worker is dealing with requests, how long each request takes end to end and what bottlenecks you are seeing allows you to define and keep contracts with your service consumers and detect potential failures before they happen and automatically trip circuit breakers to mitigate some of these issues.

I don’t have all the answers, but this is starting to work well for me.

Just say no

Some time back I became aware that I said “Yes” to too many things. I liked to please people and taking on additional burdens didn’t seem like it was much of a problem.

If I was great in managing my time and nothing outside of my control came to upset the balance I would have been fine, but I wasn’t. I was in some ways as far from great time management as the North pole is from the South – literally poles apart.

Like any habit, it only becomes so if you actually do it. I have made it my goal to say no and mean it at least once a day when people ask me for stuff.

So far it is definitely helping. I have less things on my task-list and I seem to be making a dent in some things I haven’t even looked at for months. Once you start seeing progress you start to feel happier about it and it starts to become a positive feedback loop – Bonus!

I just hope I can keep it up for the next 40 days or so, so it becomes an automatic habit.

What are you waiting for?

Are you waiting on a lightening strike?
Are you waiting for the perfect night?
Are you waiting till the time is right?
What are you waiting for?

— Nickleback, “What are you waiting for”

It seems that a lot of people are waiting for the perfect moment, and up until recently I too used it as an excuse.

That is all it is; an excuse. The time will never be perfect in your mind. There will always be something that is sub-optimal, so get off your arse and just do it. Use what resources you have right now to do the best you can and improve on it later, but just get something out there.

I have literally wasted years by not following through on ideas. Many of which I have seen other people succeed with just a year or two after I didn’t even bother to try. If I had done something rather than clicking through to the next funny cat picture, like a rat pushing a bar for the next food pellet, I would certainly be in a better position than today. I might not have succeeded, but I would have at least tried and whether I failed or succeeded I would have got the experience that only trying provides.

Quick, good or cheap – choose two

I often go to start-up events and I am still suprised how few people even bother to do even basic sanity checking about their business idea.

Either something like:

“I want to build the next Facebook”
“How much budget do you have?”
“About £500 and that includes marketing”

Where they have no idea about manpower or infrastructure costs, how they will monetise it or anything else…
Or:

“I want to build a site where people can get little fixed-price jobs done”
“You mean like fiverr.com?”
“You mean there is already a site like that?”

Never even bothering to do basic research…
Or:

“I want to build a site where people can buy our [dog] products, but I need it completely writing from scratch”
“Why?”
“Our software should be what makes us unique!”
“So you want to sell this e-commerce platform to other people?”
“Nope, it will be ours and ours alone”
“Why don’t you customise an open-source platform?”
“Because other people can copy us easier”

Where they just don’t understand why they are asking for what they are asking for.

The main thing most of these people have in common is that they are not playing to their strengths. Their strength is certainly not software development or website design but they think they know enough to manage a project in these fields. Unfortunately, most of these people end up with developers that know little more than they do and a project doomed to fail.

They have failed to even do the basics of validating that the market exists and who the existing players are, what problem they are trying to solve and if they have defined requirements (like writing your code from scratch) why this requirement even exists.

For me the thing that saddens me the most is that many of these people have sunk weeks and weeks of evenings and weekends, planning what they want the site to look like without even bothering to lay the groundwork; frequently not understanding that most of what they have designed is just composed of basic design patterns.

Spending an hour with an expert (me or anyone else) could have saved them a lot of wasted time and in some cases money.

If you do feel that an hour or paid time with myself would help you get your ideas straight, then you can contact me on mike-at-technomonk-dot-com (replacing the necessary parts) or +44(0)7950892038 / skype:darkflib


 

The phrase in the title comes from an old idea in that you can only have 2 of the 3:

  • A good and cheap implementation wont be quick.
  • A cheap and quick implementation wont be good.
  • A good and quick implementation wont be cheap.

There is some truth in it, but with the advent of frameworks and open source platforms like WordPress and Drupal, it becomes less so.

Instant Messaging

I was asked after my last post what I used for instant messaging. The answer is kinda interesting in my opinion.

As some of you know I am a privacy advocate, but try to also balance that with ease of use and ease of integrating with other people.

I run my own XMPP server which I share with a number of collegues. This is running of Prosody an open source XMPP server. We use the MUC module to host a number of conference rooms where our bots and other notifications are sent.

In addition to this I also use Skype, not because I trust Microsoft to not sell my contact info out to the NSA, but because it reduces the friction of contacting me for a number of people. Forcing them to use my private XMPP server (even through federation) is too big a hurdle for them to jump.

Email overload

Ask anyone that has been online with the same email address for more than a few years and you’ll rarely hear them say that they have no issue with spam.

The truth is that as soon you start to use an email address for anything like everyday use, you will start to accumulate spam and other unwanted messages.

Email itself is kinda broken this way. The costs of sending are that small that it profits people to send out millions of untargeted emails in the hope that they get a response from that fraction of a percent that might bother to read it.

There have been a number of attempts to redress this balance but none of them have caught on outside of a tiny audience.

Personally I gave up trying to fight it head on. I use a lot of spam filtering and manual white and blacklisting which makes email usable for me, but for everyday use I use instant messaging to communicate with collegues, email only being used for big messages or forwards from other people – and I like it this way.

If something is urgent, then email is useless, the feedback loop is too open. It needs to be something more immediate in response. IM works here and for when it doesn’t work for one reason or another, I use Textsecure a secure SMS replacement app for android.

This isn’t perfect, but it is working for me so far.

Bi-phasic sleeping

Now that I don’t have to get up for work at a fixed time, my body seems to be heading towards a biphasic state; which I am told is natural for humans and what our ancestors did.

Typically this means that I go bed about 10 or 11pm and sleep for a few hours; typically 1.5-3 hours correlating to 1-2 REM cycles. Get up and work for a few hours and then head back bed for another REM cycle or sometimes 2 if I am really tired. This means I am naturally getting 4.5-6 hours sleep a night and not feeling particularly tired at the end of it.

I have to admit, this is a nice state to be in some respects, but it isn’t helpful in others as I am out of sync with my girlfriend who is on a fixed schedule with regards to uni.