Problems with APM

Although I think Agile methodologies yield better results than serial ones, there are some aspects which are not explicitly addressed. This is my subjective list of these issues:

Mixture of feature development and maintenance

Many companies have large, legacy systems that need both new features and maintenance to address bugs. Most agile textbooks and papers only talk about greenfield software development or new features only. Many companies (in some regions) do not even start new projects and have to integrate agile into the combo of new features and maintenance. For maintenance, I feel agile does not really offer a reasonable alternative in some cases.

Of course, one can use sprints for tackling bugs. However, fixing bugs in a large, complex codebase can be very time consuming. Due to this, two weeks sprints may not be delivering as much value as they should be. Also, when bug fixes are extremely important to ship, two week sprints may be too long. The delivery of emergency patches, in a matter of days, is pretty hard to integrate into the well-known agile universe. Bugs are also hard to plan, they can come at any time, so a sprint planning meeting can not cope with such uncertainty.

One partial solution could be assigning some teams to solve bugs while the rest delivering new features, provided that the bugs are not extremely urgent. This, however, can be problematic, as bug solving teams can be locked up in maintenance work, which can cause early team members’ burnout.

Hire the best talent

The textbook suggests that an agile organization should aim for smaller, but more talented teams. In theory, this idea makes perfect sense, but in real life situations, this may not be possible. Only very well known companies with high reputation can afford to be “picky” enough for this. In my current company (a firm that believes in agile values) we have been trying to fill two positions for more than three months now, without success. However, our case is somewhat fortunate, we can cope with the workload and don’t have to hire someone just for the sake of hiring them.

Finding the best talent is also really vague. Who do we consider talented? Someone with the capability to acquire new skills? Someone knowledgeable in several programming languages, frameworks, and libraries? Everybody understands “talent” in a different way, as it is a relative term.

Customer value. All the time

From time to time, it is not easy to deliver customer value. Let’s take Twitter as an example. Twitter used to be written in Ruby, but after the application became really popular in 2012, massive scalability issues have arisen. As an answer, Twitter’s engineers switched from Ruby to Java to solve scalability and performance issues. As a result, the system managed to cope with the US election in 2012. Changing libraries and languages does not lead to direct customer value. Had Twitter stuck with Ruby, the user experience would have degraded considerably; however, at the time of the switch, there were no issues yet. Hence no direct customer value was provided.

The textbook mentions scalability and performance issues, but only at the beginning of a new, greenfield project. In these days, it is hard to plan for the volume an application has to cope with. Nowadays, every day a few startups take off: some of them may become famous, others fail. However, the most important metric is time to market; there’s no time to plan with millions of transactions per second, those problems have to be sorted out as time goes by. User value may not always be possible to ship.

Agile in large, traditional companies

As mentioned before, agile books usually revolve around greenfield projects. A major problem is how to port agile values into large, traditional organizations so much of these values are preserved. Let’s take a telecom company as an example, like Ericsson. Software development units can be agile, however, from the point of view of network operators it is not desirable to roll out new software every two weeks.

To implement full agile, Ericsson would have to convince all the network providers to be agile enough to roll out every time new features are ready. However, in such organizations network downtime may affect thousands of users, and software upgrade comes with downtime. Also, a fault slip-through may have even more severe consequences.

Release management

This might be only my personal experience, but I’ve never seen a transparent, agile project, where not only sprint but a product backlog is kept and updated. In my experience product managers give in very quickly to all the requirements, pushing the work on development teams.

When clients demand new features, many product owners treat those as a top priority, making the product backlog a huge mess. Not to mention that these backlogs are not transparent across the organization: development teams usually know the current sprint backlog only, or one or two sprints ahead, but nothing more.

However, I think this is not an issue in the model, it is a problem in the implementations. The textbook offers several ways of overcoming this problem, but many companies never get any further than implementing two weeks iterations; I personally have never seen an organization where upper levels followed the agile practices too. But again, this is a personal experience and not a fact.

What extra challenges and possibilities does adaptability post for a leader?

Adaptability is a two-edged sword, as it can create both new possibilities and disruptions in an agile project.

Adaptability and flexibility mean that a project is able to keep up with changing requirements and user needs. End users do keep changing their minds from time to time, and this can have an adverse impact on the project: too many changes make the project a mess, with no clear priorities, frustration, and awful technical solutions. This, in turn, implies slower progress, messy spaghetti code, and high fluctuation. In such situations, the activity of value creation may be put in danger, and the product owner has to step in in order to establish a constructive way of change inflow.

On the other hand, however, adaptability also represents a chance to deliver exactly what is most valuable to the users. Many time customers do not know what they want until they see some first release. By expressing their opinion, the agile team is able to create new, valuable features and make sure the software contains exactly what end users want to have included in the software. In this context, adaptability is a means of delivering customer value, which is the primary end goal of agile project management.

Of course, as a developer, being flexible and adapting all the time may not be easy. Ever-changing requirements may produce a feeling of frustration which can result in conflicts. That is a serious problem, and that’s why it is crucial for the project lead to find the right team members: right from the point of view of technical and personal traits. Failing to staff the project with the right team members can result in a failed product.

Agile and Traditional team management

As in the previous topics, agile and traditional project management take a radically different view on team management.

Traditional project management has its focus on very strong processes that allow the least deviation for product teams. The idea behind traditional project management is that the project manager is responsible for delivery, and in the delivery process, a team is a tool. As such, the most time is spent on assigning tasks to team members, following up on them, monitoring and controlling their progress. Traditional methodologies try to reduce uncertainty using detailed project plans, as they believe that the more plans are in place to stick to, the less uncertain can happen, so negative surprises can be avoided.

In contrast with these techniques, agile project management places a high emphasis on shared responsibility: delivery (creating customer value) is everybody’s responsibility. The team has to manage their workload, which means that they commit to a set of features to be delivered until a date, and they can be held accountable for their commitment. In agile, the project manager should thrive to create self-organizing, empowered teams. This means that teams are able to work with little guidance. Thus the micromanagement of traditional ways of working can be taken out of the picture. Also, empowered teams have the possibility to decide among themselves in important questions, like architecture, deployment or design. This does not mean that at times the project lead should not step in and say no (4 different architectural styles with 4 teams would be chaos, rather than empowerment), but the idea is that the leader assumes a facilitator role, rather than a micromanager one. To build self-organizing teams, the right people have to be selected. Unlike in traditional PM where it may have less impact due to the rigid processes, in APM the personal traits of the team members are crucial besides the technical ones. People who can not accept shared responsibility, commitment or are not accountable should not be allowed on the team.

Values in Traditional and Agile PM

Agile project management takes an entirely different view on values than traditional project management. The latter believes in delivering value in one, big chunk, and does not make customer value part of the project lifecycle. Although traditional project management defines effect goals, it does not talk about how valuable the final delivery would be to end users. To measure the product’s usefulness, a new, follow-up project has to be started in order the evaluation to be done.

Agile project management, on the other hand, is radically different and makes value delivery a number one priority. This methodology believes in a continuous flow of values on both sides. On the incoming side, feature ideas are coming in and are placed on the product backlog, on the outgoing side, complete, working, valuable features are delivered.

It is also important to take a look at how the two methodologies think about work itself. Traditional management considers the unit of management a task: tasks are created, assigned, followed up on, in one word: managed. A task can be highly technical (building the object model necessary for Hibernate), which the end user does not have to care about and has no interest whatsoever in. In agile, the unit of management is a feature: a functionality of the software that creates value to the end user. Such a way, traditional PM cares more about the project, agile about the product.

As a result of the previous differences, the delivery methods are entirely different. Traditional project management believes in delivering one final product, designed and analyzed at the beginning of the project. Changes are hard enough to carry through. Agile, on the other hand, delivers working software every 14 – 30 days. This means end users can enjoy a limited set of features after an extremely short time. Changes are welcome anytime which makes the process adaptive and flexible.

Agile project management also employs lean thinking: everything that does not add customer value is waste, and therefore has to be eliminated. Reports, bureaucracy, useless meetings are all considered waste, as they add no value. It is important to remember that activities like code refactoring, even though create no immediate customer value, are considered value added tasks, as they have a positive impact on the future product.

What is agile project management?

If I had to use one word to characterize agile project management, it would be VALUE (in capitals). The whole project management process is centered around creating value to the customer in the form of a great, fully functional product; working software delivered as fast as possible.

In real life situations, the value creation process is not easy: priorities, users, trends, and technologies change extremely rapidly; due to that it is not possible to plan very far ahead in the future. As a response to that, agile project management values the principles of adaptability and flexibility. Very short release cycles employed in agile project management make it much easier to respond to users’ needs, adapt to changing requirements and as such, deliver working products.

Compared to traditional project management the agile way does not try to apply the smarts up front. Very fast time to market accompanied by frequent, small releases are valued much more than trying to figure out and solve problems at the beginning. Also, agile project management values early and regular customer involvement. Practitioners understand that the clients, end users can and do change their minds frequently and may not even be aware of what they want differently until they see an example. Hence agile employs product and sprint backlogs for capturing user requirements, and demo meetings for early and highly valued feedback.

The agile ways of working also believe that no process is ever perfect. Rather than trying to craft a sound process from day one, agile believes in empowering the teams. They are free to tailor their ways of working in a continuous manner, in a constructive and inclusive way. Agile gives a guide to newly formed teams but also gives them liberty and room to experiment and find the ways that are best suited for them. The agile mindset does not believe in the ‘one size fits all’ principle, but acknowledges that each team and situation is different, with unique needs.

To conclude, agile project management is about:

  • Delivering value
  • Embracing change in all phases of development
  • Being flexible, trusting and empowering teams
  • A continuous improvement cycle based on feedback (be it internal – retros – or external -demos)

Agile values, SU-DSV courses

The Agile manifesto addresses four central values: individuals and interactions, working software, responding to change and customer collaboration.

Individuals and interactions over processes and tools: In my opinion, this is everything the communication course was all about. Involving everyone in the decision process, empowering teams, non-violent communication, small groups as complex systems: these are all about interaction and placing people in front of everything else. Agile takes communication and involvement to a whole new level: sharing knowledge, collaboratively improving the process further, keeping all the participants updated all the time.

Working software over comprehensive documentation, responding to change over following a plan: these principles seemingly contradict courses like Practical Project Management or Business Strategies. These courses were talking about the role of the project manager, project plans composed of different chapters, like risk assessment, communications plan, activity plan, etc. Agile does not address the project manager role and places documentation and plans on the right, making them a non-top priority. However, in real life conditions, I think, it is on the contrary: Agile has actual value, but for most projects, it cannot be implemented ‘by the book.’ Letting the team build the product in isolation is not possible unless the product is a relatively new one (not yet introduced to the market). For more mature projects some sort of coordination and planning is required: that’s where the project manager comes to the picture. Project managers take off a huge load from the back of the team, by dealing with coordination, empowerment, planning, staffing, etc.

These two courses also pointed out the importance of responding to unforeseen circumstances: monitoring and controlling, keeping the project plan up to date, taking action when something goes off-track.

Customer collaboration over contract negotiation: I think this is the principle which has not yet been explicitly covered in the courses. Communication, PPM and BS have all addressed smaller parts of it: effective communication (collaboration), some bits of negotiation techniques, productive meetings. However, collaborating with customers as such was not yet part of the curricula.

Agile and Waterfall – a SWOT analysis

Many things have already been said about the two methodologies, so consider these lists my own, subjective opinion.

Agile

Strengths:

  • Early customer involvement: end users can have an enormous impact on the final product; product backlogs, demos, early feedback loops, all make it possible for end users to have a say about the final product.
  • More efficient communication. agile is all about communication: sprint plannings, daily standups, retrospectives; all these meetings are meant to share information, update each other and make the whole process even better.
  • Faster product development. Being an iterative process, agile tries to deliver increments which are valuable to the business. These minimum marketable features make it possible for end users to start using the product in as little time as one sprint (in an ideal case).

Weaknesses:

  • In real life situations, it is pretty hard to abide by the “no change to the sprint backlog” rule. Business priorities change fast; thus the backlog has to change too, causing smaller or larger scope changes. (Kanban can address this issue, but many people consider it a Lean methodology, rather than Agile)
  • Hard to take full advantage of it most of the times. Startups may benefit the most of agile, but in cases when releases have to be carefully planned and executed (banks, insurance companies, telecom companies) only parts of it can be implemented.

Opportunities:

  • High emphasis on product, principles and people rather than rigid processes.
  • Agile encourages empowered teams, which can boost productivity as team members are allowed to take decision and act on them.

Threats:

  • Geographically distributed teams cannot make full use of agile due to communication difficulties, cultural differences or client-vendor relationships.
  • The focus is far from being placed on documentation in agile. This can make it hard for new team members to join a mature product (many times agile projects employ ad-hoc documentations only, using wiki pages).
  • Possible dissatisfaction resulting from the mixture of traditional and new ways of working, roles and points of view.

Waterfall

Strengths:

  • Engineered way of software delivery by engineers, for engineers. Very rigorous planning up front aims at eliminating bugs right before they get introduced. In theory, this can result in major bug free implementations.
  • Phases do not mix, so it is always obvious what phase the project is in. This way the project lifecycle is easy to understand, staffing needs are clear.

Weaknesses:

  • Analysis-paralysis: in a traditional waterfall setup coding cannot start until the analysis phase completes. For a large enough project, this can take up extremely long time. Such a way, no fast results can be delivered.
  • Due to the hard constraints of the phases, it is somewhat hard to add a new feature or requirement once the analysis phase completes. This way the customers have to think well up front about their needs and there is little room for change.

Opportunities:

  • Keeps engineers disciplined and focused: rules are much harder to break than in agile.
  • Provided it is done right; Waterfall can minimalize waste. Since analysis and design are done much before any code gets written, in theory, it is ensured that no unnecessary software (in terms of features) gets created.

Threats:

  • Too much effort is put into the process itself and values and people are not explicitly addressed.
  • Too rigid for many industries: web based development, games development (entertainment, in general), startups are prime candidates where waterfall may be too slow to work with (except for several games, which are released only after 5-10 years).