Discussion about the Agile narrative

Part II: The rise of the Technologians

Putting things into perspective - update for January 2013

I wrote the original article (below) after I had read an Agile blog under which raged a debate about which was better, Kanban or Scrum. Some of the Scrum advocates claimed Kanban was too lightweight, incomplete and stepping on their toes. In response some Kanban advocates hurled back, not entirely unjustified, accusations of methodology fascism. It rather dismayed me that this was the public face of supposidly Agile champions – jostling for position and arranging methodology deckchairs on sinking projects. It highlighted what I see as a crucial problem with many who use the language of Agile to persuade but do not want to be bothered by the reality of the practical dependencies necessary for a successful deployment of Agile.

On re-reading it recently I realised it might need a bit of further clarification for those not already familiar with my work and Agile advocacy. As far as I was concerned, and I have not changed my mind since then, both Kanban and Scrum work if you get the engineering right but neither is a total solution. In fact, either can make things exponentially worse for beleaguered project managers, and everyone else working on the project, if introduced without the essential support from engineering; and the essential support for engineering.

Properly implemented, with due care and consideration for reality, both Kanban and Scrum can support good engineering practices and enhance and support the trust based management style and environment crucial to successful Agile, and, more importantly, to successful projects. They are both iterative and they both seek to focus the available resources on the highest priority work.

On the other hand, without good engineering practices, good design and sensible planning it does not matter whether you use, Kanban, Scrum or anything else, including waterfall methods for that matter. The project will implode when what it is producing no longer supports:

  • frequent requirements re-prioritisation
  • changes to time-scales and resources
  • incremental delivery of working versions of high value or critical features
  • regular and internally consistent releases and deliveries
  • rapid fault finding and fixing
  • rapid response to user-driven changes in vital aspects of functionality
Both Scrum and Kanban promise, presuppose, expect and predicate these attributes from the product under development. However some practitioners and parties marketing and implementing Scrum and Kanban are not interested in discussing the technical considerations necessary to produce this flexibility. "That is too technical" they say when the issue is raised. If your product is a technical one, this seems a tad naieve.

I am reminded of the episode from the television comedy, Fr Ted, where Father Jack responds to every theological question with the phrase "That would be an ecumenical matter". Although he has no clue about the question, its implications or how to address it, the visiting bishops think he is a gifted and wise theologian. After that I always recall this scene when I hear methodology practitioners of any stripe dismissing something crucial to their project, or product, as a technical problem which is somehow too far beneath their pay grade to concern them.

Kanban

Kanban is a technique based on Lean Manufacturing and JIT (Just in Time) Production. In lean manufacturing a Kanban card is a token placed in stock to indicate that it is time to order more when this card is exposed. Toyota borrowed this Kanban system from supermarkets. When Kanban is talked about by Agile or Lean practitioners they usually mean a method of using sticky notes and white-boards to implement a pull rather than push method of dealing with requirements. It is used to track the progress of units of work and the productivity of resource. Work is categorised as "to do", "in progress" and "completed".

Scrum

Scrum is an approach to project management that was made popular by the Agile movement. It can be useful for introducing an Agile approach into organisations in easy steps. It replaces traditional requirements capture and planning with an iterative cycle by organising the project into short intensive work periods called sprints and intense planning and progress monitoring sessions called scrums. It borrows its terminology from the game of rugby.

Software Development

Kanban and Scrum both work well in a software development context, and this, despite protestations about the universality of the tools, is where they are most successful. They are heavily dependent on a layer of engineering ability and good practice to produce results. They sometimes like to dismiss this dependency as something for the programming monkeys (akin to those monkeys that reproduce Shakespeare by accident) to sort out. Neither Scrum nor Kanban says anything about how the product is to be engineered and practitioners often do not acknowledge this requirement when selling the benefits of the Agile approach.

There is also the elephant in the room: Software development is not manufacturing; but that is a story for another day (tackled in Agile Exposed).

Successful Agile working relies on the product being amenable to this form of iterative and incremental management style. Being able to deploy either Kanban or Scrum in this context depends very heavily on the software source code being flexible enough to be rearranged liberally as the requirements, capabilities, understanding and priorities change. To respond it must adhere strongly to the ideas of modular design, low coupling and high cohesion. Agile achieves this through the disciplines of:

  • Test First Programming for dependable, maintainable and working code and for error containment.
  • Test Driven Design to maintain design focus and prevent feature creep.
  • Continuous integration for code that is architecturally sound.
  • Refactoring for modularity, coupling and cohesion as the requirements force changes on the code base.
  • Pair Programming for universal awareness of architecture and design and the removal of skills bottlenecks.
These would be ecumenical matters! according to technologians.

Technologians

My newly coined and patented term for those who believe in or proselytise divine methodology intervention and magic technology. These people can be found arguing about how many user-stories can dance on the head of a needle and whether you should be sent to project limbo for using the wrong colour sticky notes.


Original article from March 2012

The Great Kanban versus Scrum Debate

Which is better, Kanban or Scrum?

Well now, that is a divisive little question, isn't it? (Please read update above for clarification and background.)

The problem is that this sort of question closes the conversation. It leads people to argue brands rather than the need for products or even the long term health effects of using these products. It is equivalent to asking "when you are relaxing do you prefer to watch Eastenders or Coronation Street?"

The question is posed in such a way as to ignore that there are other programmes and categories of programme, such as documentaries, films, comedies and current affairs. It ignores that there are other sources of entertainment like books, radio, music, board games and conversation. It ignores that there are things to do with spare time such as arts, crafts, education, sport, cookery, woodwork or whatever can float your boat. Put in these terms that sort of closed question is not one anyone should allow to define any conversation - even if it is extended to include "X-Factor or Britain's got Talent?"

I'm all for it if you like these shows (or Kanban and Scrum) and get value from them as long as you realise that they never really amount to more than bread and circuses (Spectacle that Roman politicians used to distract the public from asking awkward questions about where all the money was really going).

Kanban or Scrum - follow the money

Who does this sort of discussion benefit? It certainly does not benefit software development or the programmers who have ultimate responsibility for doing the hard bit: developing code. Whatever methodology, tool or system is used the organisation has to have code that works and is useful. We have to consider the end users and purpose of the system. (End users are frequently not the same as customers: the purse holders who do not necessarily have to use the tools they commission).

Things like Kanban and Scrum can give the message that technical skill is secondary and really rather superfluous. I fear that many well intentioned Agile spokespeople have not considered this effect because they are coming from a strong skill base acquired before Kanban or Scrum.

As far as I can gather this limiting methodology discussion benefits a number of groups:

  • The people who sell the brands being discussed. These are people whose goal it is for you to join their cult and be unpaid representatives, increasing the market share of their brand.
  • Some unscrupulous consultancy firms who want to sell simple processes for vast fees. They tend to favour easy to deliver and misunderstood mantras. They do not want to get involved with the actual real world complexity of software development that requires understanding the medium and a level of technical skill.
  • Short term profit driven managers. These are opportunists who want to generate what looks like a profit and move on to their next career step. Should I ever have to argue that the proceeds from selling limited and necessary company assets to artificially inflate this year's profit margin is not good business? It is not good business - it is privateering (piracy on the high seas).

Does the same thing apply to Agile

Well it all depends on what you consider Agile to be. If you consider Agile to be the neatly packaged methodologies and tools that encourage the question "Waterfall or Agile"; and which couches the conversation in terms of efficiency - then yes it does apply. If you consider Agile as being an exploration of how we might deliver better engineered software; how we make the most effective use of available resources, apply accumulated wisdom and skill and how we make sure that the software we are developing stays on course to help its end users do their job, then a different paradigm applies. Agile, in that case, is a horse of a different colour.

In the Trousers of Reality I explain that anyone who presents you with an "either or" question is likely to be limiting your options and misdirecting you.  This is sometimes deliberately done to create false equivalences to promote vested interests.

In Agile Exposed I point out that Kanban is a useful tool that helps with the flow of work but that it has dependencies on the skill of the programmer, the management team and the structures it works within. It is not a stand alone or a complete solution. I also point out that Scrum is a hollow doughnut and if it results in good software it is entirely coincidental and probably the result of previously skilled professionals who succeed in by making Scrum work for them rather than working for Scrum.

What do we do about Agile then?

Whether it is called Agile or something else the philosophy behind Agile has permeated the software development industry. The genie is out of the box and we need to be careful of the wishes we make. Remember the story of the three guys stranded on a desert island who discovered a genie in a bottle and got three wishes. The first one wished he was home with his wife and children and was there in a blink. The second one wished he was in Paris drinking the finest wines money could buy and he was there in a snap. The third guy said he was lonely and wished his friends were back on the island with him.

I remember working on projects before Agile was even on the table and despairing at the gross misunderstandings that surrounded the management of software development projects. I have a fondness for Extreme Programming because it put things like customer and end user collaboration, test first and test driven programming and small continuously integrated deliveries on the radar of even the most technophobic manager. It gave us a common language and lexicon to discuss these things. Even if its high priests have long since migrated to the dark side of the profit motive the roots of Agile were planted deep in ground crying out for growth.

The principles of skilful and professional behaviour in IT are not limited to Agile but they were pushed to the forefront of the conversation by Agile. Developers started to shrug off the anti-social geek mantle and started representing their profession as one that had a place at the governance table. There is no doubt that it frightened a lot of people but many of the best professional managers listened and were continuing to listen before we started to confuse them with petty squabbles about labels, other people's marketing campaigns and which sort of whiteboard and sticky note we should use. Can we really blame them from backing off? We sabotaged our own pitch.

Silo'd development, spaghetti code, misunderstood requirements, difficult and expensive maintenance, software not fit for purpose, feature creep and all the other delights we were trying to address with Agile are still problems. The homogeneous, two-dimensional world of Kanban and Scrum does not really address them. The only thing that really addresses them is skill, experience and responsible governance. If tools like Kanban help in the right framework, I am all for them. There are even aspects of Scrum that are helpful under the right conditions if Scrum is not throwing a wobbly, dressing as Napoleon and trying to dictate terms to everyone else in the room.

The threat of outsourcing and offshoring

While we argue about Kanban, Scrum, waterfall or Agile there is a real and creeping threat that is being smuggled in by the appeasers in all camps and professional agents provocateurs. That threat is the de-skilling of software developers for short term gain. The wheeze appears to be to claim that software development can be simplified, packaged, outsourced and offshored to the cheapest bidder. Over simplification feeds this narrative.

I once worked in a company where this was happening. I thought that introducing Agile would help stakeholders appreciate the importance of holding onto the family jewels: their skilled and experienced workers. I coached the importance of well engineered code and the practices and principles that support professional relationships required to deliver solid products.

Things were going well until someone decided that Scrum was to be introduced as the sole methodology. I could only advise against it but, in the face of the speed of Scrum certification and its aggressive marketing, my advice was not taken. Scrum arrived and pushed everything else out of the way. The hard won progress in practices required to deliver quality code were considered to be too technical to be of any use. Of course they were not of any use if your goal is to get rid of your developers. Scrum was used to further the narrative that software development had nothing to do with cutting code and everything to do with packaging requirements. Developers were let go across the company while neophyte scrum masters tried to manage huge offshore teams by using burn down charts and electronic dashboards.

The profit margins looked good in the short term but the software that was delivered was less than good and it degraded faster than you can say sprint. Agile was blamed and now that same company is re-hiring and retraining developers.

Oppose the narrative, change it, or at least don't actively promote your own demise

Kanban and Scrum are easily used to further the narrative that software development does not require technical skill or architectural consistency and that the programming side can easily be automated. This is a handy narrative for people who want to make a quick buck at the expense of an entire profession. With the importance of IT to our economy and society, deskilling and offshoring our technical jobs has much more serious consequences than which way you organise your sticky notes on your whiteboard. We need to put the emphasis back on the value of technical skill and the training of our own software developers as an important and ongoing infrastructure priority before it is too late and we become a nation of experts in defunct management theories.

You may say that having a discussion about Kanban or Scrum is fine tuning the details but I would say that it is misdirection. I also understand that there are people involved in the conversation who have the best possible motives who have made these things work and want to spread the good word. If you have made these things work it is your skill that has worked not the methodology. It was how you used it that was important not whether it was the best tool. Can the makers of hammers reasonably claim that their hammer is more important than the skill of the carpenter? If the carpenter has no hammer he/she will use a screwdriver or dowels or staples. The carpenter's skill outweighs the tools. A good workman will always find a way regardless of tools and a bad workman always blames his/her tools.

A craftsperson is more limited by materials and lack of skills than tools so can we please turn the conversation from tools to materials and craftsmanship. Provide the environment and training and let the people doing the work choose the tools they need to do the job.

Comments/discussion related to this article at: Oakdust Discussions

Oakdust Home Page