A sysadmin’s reflection on several years of practicing OKR
Preface
This is the very first ever article about management ( ? ) . I even don’t know if a goal-setting framework like Objective Key Result can be classified as a managerial stuff, but as we ( our company / team ) have been following it since 2017, play an important role in our job till now, with ups and downs, pros and cons, it’s worth spreading some thoughts.
Disclaimer: I’m a Linux sysadmin , tracer and a team leader. I’m not a CEO of any startup :)
What is OKR ?
Again, according to wikipedia:
Objectives and key results (OKR) is a goal-setting framework for defining and tracking objectives and their outcomes.
In the simplest form: it’s a way for planning, implementing, tracking, assessing your tasks by:
- Focus ( to objectives )
- Alignment ( between teams based on agreement on objectives above )
- Tracking ( by key results. Measurable ones )
- Stretching ( for excellence. Aim high to be more. Though it’s OK to fail )
It was first introduced in Intel by Andy Grove, shining ( still ? ) at Google, get close to me by John Doerr’s famous book on the subject: measure what matters. If you haven’t heard about this or done it before, this blog may not for you, sorry.
Our company has been adapting this framework for several years. Though I’m not speaking for my company. This is my own opinions.
I won’t talk much about OKR itself in this blog, this is my pure reflection on how it has been so far.
First impression
I read the book twice before really doing it. ( As our ex-CEO hold a class about it and I joined him ).
The very first feeling is like: “yeah, that’s cool, innovative. It’s even OK to fail. Fantastic!”. Because i read the book, about the success stories from Google ! ( about Larry Page and something “uncomfortably exicted” ) , about inspirational startups, about pizza-making robots.
That’s nice.
Don’t worry, there’s no twist, even though Zume failed with that robot project ( for some reasons i don’t know . The link is just for reference, I haven’t read it ). There are the good, the bad, the tricky.
The good
OK to fail
Freedom. Room for errors. No pressure. That’s what we love about this new system. Don’t get me wrong, we’re still tracking things closely ( by measurable key results: still timeline ( no deadline because it’s adjustable ) , metrics, meetings to catch up / align… but we’re have more freedom than pressure. I don’t know which one is better, but I would say fortunately we have good teams with very high responsible, proactive folks. Discipline is rarely our problem.
Because.
We’re doing big things
OKR encourages us to stretch ourselves. To aim for something very ambitious. The ambition that produces a goal too big that reaching 80% of them can considered a huge successful. ( That’s why it’s OK to fail ).
We dreamed big at first, followed that spirit eagerly. At that time we’ve been struggling with maintaining old stuffs for couple years and this is a wake-up call.
We kicked-off containerization stuffs ( with docker and Mesos that time ). We upgraded the servers that there’s nobody dare to touch in years, decided to switch to ansible for a long time being fan of debian package. CI / CD is applying in a broader range, involving multiple teams. KVM was brought back to life after virtualization was buried for years since the failure of openvz in the very early day.
Big operational goal like “there’s no single point of failure” or chaos monkey practice were raised and every body is excited about it.
I poured myself into tracing area ( which is pretty new that time, in the very early day of perf and flamegraph ) and quickly felt in love. We organize our very first conference for Sysadmin.
That’s the wind of change that Scorpions should understand very well.
The bad
Implementation hell . Easy to guess. It’s easier said and done. Some reasons:
Alignment
We need alignment to really do OKR properly.
From company’s level to infrastructure team
I can’t see how we should match well with our company level objectives even till now. Company ( especially young ones like our ) should have vision on innovative products, big revenue or dominating market share, which in no way IMHO can directly involve with a team that prefer securing things, stablizing stuffs or optimizing resources.
I was feeling kind of left out at first in OKR planning sessions in the very beginning, but no matter this framework or anything else, infrastructure team’s place is in the background. It’s not bad, though. Got used to it anyway.
You won’t be jealous with a sparkling star when your job is running the whole universe, my dear sysadmin fellows :)
Between teams
Teams need to align with each other, in order to inform / adjust when there’s an unwanted mid-cycle changes, to be re-focused if needed.
And we just couldn’t do it ourselves in the early day as we were all knew to this game. We needed an OKR master to regulate all this. Human is never perfect and can easily be a single point of failure. Fortunately our C-level managers managed it in time to keep it up and things are better now.
Between team members.
We were excited at planning phase and did it quite well. But it worn out quickly over time. Due to incidents, mystical bugs ( that even made us more excited! ) or just obstacles while pursuing our ( too ) big goal.
We didn’t have internal meetings often that time and just managed tasks by task management software ( not built for OKR like weekdone, though ). Thus everything was like a plan on a paper. It’s not a living things if we don’t communicate, adjust, update with each other.
We fix ourselves quickly by doing weekly 1:1. People ( including me ) like the big ideas, agree on them while planning, but need to be refocused from being distracted by routine tasks or incidents. Quite often I lost the fire myself ( and don’t know how to pick it up , which is normal, i guess ). Discussing often helped us to some extent, even though sometimes I’m just a sounding board for my teammate. And we were optimistic, again, about our big goals.
Unfortunately, this thing is a never ending story, like a continuous improvement process. There were ups and downs, inevitably. I’m not quite confident to say that our 1:1 meetings are very effective and help us to refocus ourselves always even till now.
Thus, alignment is hard.
Big goals and OK to fail trap
The goal sometimes is too big. It’s quite difficult to judge, we ( sysadmins ) can argue that we have to much unpredictable incidents, but who doesn’t have it ? So no excuse. But the problem is, because of OK to fail, we failed our goal more often. And I was afraid that 1 day we would not feel anything about it. It’s the loser’s habit.
We have to fix it. Then we decided not to stretch more ( or we already did all what we could ) and adjust the goal, to smaller ones :)
This is a very tricky moment. Too big we’ll fail and we’ll get used to it. To small, is it OKR any more ?
Till now I can’t really know if I’m managing the balance good enough, still a big puzzle.
The tricky
The tricky balance
Of setting objectives. To big it’s unrealistic. Too small, you’re coward and not playing OKR game hard enough.
Of setting measurable key results. Too vague, it’s hard to track. Too precise, it’s just dry mechanical numbers and can be game on.
Of choosing between top-down or bottom up approach. Of letting team refocus themselves based on what they agreed on or we definitely need an OKR master’s intervention.
Do it right, you’re an expert. Otherwise, OKR can be just an mediocre planning framework.
We’re not living in our binary world of engineering anymore. This is managerial stuffs.
Routine tasks
There’s some ugly maintenance tasks ( incident responses, things that we have not yet been able to automate yet … ) that you shouldn’t put it into your OKR. But sometimes it can consume most of your time and spare no energy for OKR effort.
The solution for this can be automation ( to reducing operational toil ), improving alerting system, setting SLO for services ( to clearly decide when we should act ). But till we can finish it, save 1 objective in your OKR to deal with it explicitly.
OKR shouldn’t contain only beautifully audacious things.
Mid-cycle changes
Mostly because of agreement without enough commitment. Can be bad planning as well.
Many times we got confirmation from other teams to help us solve the dependency issue from the side which can block our objectives. But it’ll never happen because of “lacking resources”, “busy with other tasks” , thus our objective must be blocked mid-cycle. It’s inevitable, accept it.
Or sometimes it’s new idea from the top, new opportunity arises from nowhere and we have to catch it immediately. Pause everything and switch to this new project please. It’s also inevitable.
What can we do in that case ? Adapt, improve, improvise ?
I would argue first :) And set error budget for such behavior, we can’t do it all the time, otherwise this OKR framework is useless, stop it immediately.
Yeah, it’s tricky. Depend on who are you talking to.
OKR vs rewarding system
I basically agree with separating OKR with rewarding system. Because otherwise it’s some form of KPI. Or people may tend to set a more easy-to-achieve goal to game on such rewarding system. But if we have OKR and only OKR as a goal settings and tracking framework, then where else should we look for info to decide about reward for people ? Some may say that let the managers decide that. True, but people’s decision, emotion, fairness , objectivity — all those things are very tricky
Conclusion
I still love OKR so far. Especially the feeling of being like on a stage, announcing new products / achievements for the whole team in each OKR reviewing sessions. That’s very special. But I’m too far from mastering it, as it’s complicated, tricky as I explained above.
Is it uncomfortably excited still ? I don’t know.
What’s your OKR story so far ?
