Almost three years ago at the Agile Development Practices conference Mary Poppendieck took the stage and announced to the assembled agilists that Agile has become mainstream. It was met with applause.

This moment reflects very well the mood of those involved in the agile movement back then. Everyone was sure that agile approach and practices will now take the industry by storm and reshape the way we work on software projects. For some time it indeed looked like Scrum, XP and other less known practices and methodologies will replace the dreaded waterfall and the poor quality it consistently delivered in software. Alas, three years later it is clear that even though almost everyone now claims to be ‘agile’ not everything turned out so great. In fact, it turned out that implementing Agile in teams is very hard and in large companies with many teams even harder. There were many success stories – but an also a great number of (mostly untold) stories of agile failing to deliver its promises. Clearly, Agile was working as expected only in some places.

Many suspected that where Agile fails it is simply not done right. A research done in 2009 that showed for example that about half of supposedly Scrum teams didn’t work in iterations. That means their Scrum was simply faked. This, however, doesn’t explain why those teams did fake it. As a wave of (for now) slight disillusion with agile can now be felt in the industry agile leaders try to keep the momentum going and find some explanations. Some laid blame on the Scrum Alliance and its program of quickly certifying many people to be Scrum Masters when all they did was sit for two days in class without falling visibly asleep. Others pointed out that technical practices in many teams were so poor that developing a truly tested, working increment every month was not possible – this is exactly what Ken Schwaber tries to address now with his PSD training programs focused on developers.

However, while all of this may be true there is one more reason why the picture of agile in 2010 is less rosy than expected. This is – I think – primarily a social problem, much less a problem of technical practices, methodologies or certifications.

If we look at agile as a social phenomenon inside the IT industry then this approach to development evolved within a very specific group of people. Despite all of the differences between them those were the people who were (and mostly are to this day) truly passionate about their work. They do care about software, about quality, craftsmanship and how people feel at the end of their work day. Furthermore, for the most part the original signatories of the now-famous Agile Manifesto were people who at some point in their careers had a very tough project to do – and managed to do it.

Agile in its initial years – until it entered mainstream – was spreading mainly among people who were similar to its founders – really interested in working better, in delivering good products, in improving quality. For those reasons they were prone to be quite optimistic when thinking about the way others approach their work. I feel – though they never confirmed it – that Schwaber, Sutherland and others truly believed that programmers, testers, analysts and others IT wizards wanted to do things right and knew how, and if only the burden of waterfall and dysfunctions it promotes will be lifted they will rise to the occasion and, well, do things right.

The problem, however, is that most people don’t care about the work they do. They are forced to work by the economic realities – because they need money to pay their bills.

Consequently, in their work they are not moved by passion for what they do but rather follow the path of least resistance to optimize the outcome – get more money while doing less. They get through the work week doing as little as required to keep their job, then joyfully devote their weekends to things they truly care about. As a result they don’t respond to agile as expected, they don’t embrace it, because they don’t like the clarity it brings and the level of engagement it calls for. They want to be told what to do, as strange as it may appear. Hence the attempts at dogging it by faking it (like telling management they do Scrum, but not delivering working software in iterations) and other forms of passive resistance.

While some companies – especially small startups – can have the luxury of selecting only employees that do care about software as such the industry as a whole can’t. First, because there is just not enough such people (and I can tell you this from first hand experience – for the last 5 years I’ve been almost constantly looking for good people to hire and I can tell you that no more than 1 in 10 are truly interested in what they do – the rest try to fake it). Second, because good people tend to gravitate towards interesting projects, new languages, challenging problems – stuff they can brag about to their peers over beer. Many software projects don’t meet this description, but still have to be done.

Just to be clear about one thing: I’m not trying to judge people and their attitudes here. While some may believe that approaching work as an unwelcome necessity is inherently bad this concept of work is deeply ingrained in our societies. In fact, up until the last (20th) century work as such was an activity of lower classes/castes and elites looked at it with disdain.

The industrial society only made it worse. Until the 19th century work meant primarily farming. Farming is very hard, but it can be very satisfying because one can see and use the (literal) fruits of his labor. The assembly line of the industrial age factory doesn’t give the worker even that – the only real benefit is the pay. A corporate office is not much different. So people adjusted – they train in skills needed to get a job, work to get paid, then devote the rest of their time to things they really do care about. This is the societal default. Blaming people – or anyone – for this is a waste of time.

So the real question that agile practitioners, trainers and coaches, now face is how to convert teams of people less willing than the initial early adopters. This is the real test of agile as a method for “changing the world of work” – whether it can work with “the rest” – also because old tricks may not work. They shouldn’t give up, of course, but realize it will take a long time to change old habits, convert people, re-ignite their interest in work (if they ever had any) and make them more willing to be part of self organizing, self managing teams.

However, looking at it from my experience as a manager I think that it well may be that agile is simply not appropriate for some teams. And instead of struggling to adopt something that is squarely against who they are some people would be better off practicing well whatever they are practicing now. At least they would not have to waste their energy faking practices they abhor.

A pragmatic manager should recognize such teams and balance the potential benefits from trying to force agile methods there against the costs and risk (incl. potentially loosing people who will leave). Even if agilists will call it heresy.