Monday, July 14, 2008

Sustaining Capacity in Maturing Agile Software Teams - Part 4: Counter Measures

Mature agile practices optimize learning and communication, enlisting all project artifacts and processes into the effort.

By creating a learning culture that defends its means of communication from entropy and obstruction, and that rigorously eliminates waste, a team can continue to optimize its performance, avoid bottlenecks, and satisfy customer without degrading throughput.

Smooth oscillations in team performance by:
  • Eliminating waste, and
  • Improving continuously
The obstructions and friction that an agile team deals with as it matures are often the result of waste. Waste permits the buildup of entropy that makes it difficult to improve continuously.

A production team will undergo necessary radical improvements. Radical improvements are necessary and welcome, but they can be disruptive.

The team can avoid unnecessary disruptions by taking a tough stance on waste; learning to see waste, and to counter act it by sharpening its ability to recognize opportunities for continuous, incremental improvement.

A team can learn to counteract waste by working to clear its existing entropy buildup. The following practices can be effective counter measures:

Soluble Code
Soluble code is understandable with little effort on the part of the user.

Most software code is written without consideration for the reader. Developers using that code later, either to make a change to the system or to understand the system, have to decipher the code before its meaning can be unlocked and work can be done.

Soluble code is like a book with a table of contents. It allows the user to scan the content to discover where his work site is, and to quickly garner understanding of the intended purpose and functioning of the code without forcing the reader to read and decipher code that isn't germane to his present concern.

Solubility in code is as much about writing code to allow the user to identify and skip over non-relevant code is it is about writing relevant code that instantaneous transmutes into knowledge and understanding, and guides the present work to be done.

The time spent teasing understanding from code is waste. Code can be written so that users of the code spend the least amount of time focused on things that aren't of interest to the current task, and so that learning can happen opportunistically and osmotically.

Soluble code patterns highlight the essence of the code rather than the ceremony of the programming language or the frameworks being used.

The Context/Specification pattern expresses each permutation of a use of a class as a separate test class, or context. Contexts are natural and recognizable uses of a class or subsystem or web page, etc. The pattern encourages test code, or specifications, that is much more soluble.

Contexts and specifications are written to describe the use of the system, or the experience, rather than the implementation of the system or test itself. This makes test code easier to navigate for team members who aren't familiar with any particular collection of tests and test code.

The experiential language permits the contexts and specifications to be exported and used as external documentation of the system that is usable to people who are intimately familiar with its design and operation, as well as people who are trying to gain familiarity, or to assess the correctness of the system against expectations.

Tests written by developers and by quality assurance are written in the same experiential language. This reinforces the shared language that the team members use to communicate with each other and with customers. This consistency helps to surface misunderstandings that may lead to defects and rework.

Context/Specification places a heavier onus on the precision of user stories and the expression of acceptance criteria. This causes the team to go deeper into analysis and design during planning, and to surface assumptions and misunderstandings that often cause missed expectations and rework.

Test-Driven Development
Unit testing is a quality assurance activity. Unit testing proves the correctness of the software under development and helps to protect the team from creating new defects when making necessary changes.

Unit testing is often done after a developer has implemented a feature of function, or after a change has been made.

Tests are written before functional code when practicing Test-Driven Development (TDD). By writing tests first, developers are naturally caused to think through their intended design ideas and implementations.

Test-first programming allows programmers to design and code in smaller chunks. Object-oriented code is often best served when units of design are kept small. Loose coupling and high cohesion are the result of using fine-grained designs, and these qualities in turn cause code to be easier to understand, easier to maintain, and easier to test.

Smaller form factors help reduce duplication, which is the principal source of errors and defects stemming from inconsistency, and small factors also tend to lend themselves to greater reusability.

Developers use Test-Driven Development to create right-sized designs from the start, leaving a legacy of code that is supple, and easier to tune and adapt.

Test-Driven Development prevents the design rigidity that acts as an entropy attractor and amplifier.

Design Improvement
When entropy has accumulated, more radical design changes are often required.

It's not desirable to keep a product team mired in renovations and repairs that put business and customer imperatives on the back burner. At the same time, it's very ineffective for a team to be working on business features while simultaneously doing design improvements on the code that these features are built upon.Design improvement teams can work one iteration ahead of feature teams, softening up the rigidity that the feature teams will be working on in the following iteration to the extent that is possible based on the current design and architecture.

There are reasonable limits of what a design improvement team can do since its work tends to focus on aspects, frameworks, and patterns that affect the implementation of a good deal of the application code.

Design improvement teams introduce seams and shims into frameworks and architectures to try to isolate feature teams from widespread destabilization of common code. They then do whatever they can to break ground in areas that will be affected in subsequent feature work.

Design improvement requires a greater effort in planning and design. A design improvement team works one iteration ahead of the current iteration. It doesn't implement the future iteration stories, but considers necessary changes to frameworks, aspects, and architecture than will enable the feature team to create code attracts less entropy, and makes changes to those areas of the system that it can without upsetting the iteration in-progress.

Synchronized Teams
Software development teams ideally work as synchronized units, however some activities naturally happen sequentially.

Quality assurance testing naturally happens after software has been created, however the amount of development work that is batched up before testing should be kept to a minimum in an effort to avoid waste.

Batches of untested software are inventory, and inventory has a material carrying cost as well as a impact on the ability to clearly see and understand the work in progress.

To reduce inventory, quality assurance testers must begin their test plans and implementation concurrently with the beginning of the design and implementation of a feature. Developers and testers begin work on a feature by planning together, and a feature team made of developers and tester(s) work together throughout the production of the feature to remain synchronized.

Developers or testers slide in and out of development and testing functions in an effort to keep one function from getting too far behind the other, which leads to queuing and batching that leads to more waste.

A developer or developer pair that finishes his work well ahead of the testers should assist the testers. Developers remain abreast of testers' work so that this transition is as smooth as possible.

Developers should avoid moving on to the next feature until the testing is done on the current feature. Moving on to the next feature before testing is done is a synthetic progress, and while it might feel like an accomplishment in productivity for a developer, it often sub optimizes the whole of production.

Friday, July 11, 2008

Sustaining Capacity in Maturing Agile Software Teams - Part 3: Recognizing Entropy

The visible and tangible differences between traditional phase-based development and agile development are much more obvious than optimizations made later within an agile practice. The next round of improvements can be harder for agile teams to embrace than the original effort to embrace agile development.

Each successive improvement is increasingly subtle relative to the previous improvements. Product development organizations usually have to first start seeing signs of emerging entropy before they have the necessary context to consider new, subtle improvements.

When a product development organization starts sensing entropy, its first decision should be to decide when it needs to act upon it, rather than to decide to presume to begin to act upon it immediately. Acting too early can mean that the team might be acting without sufficient context and understanding of the sources of entropy.

If the team recognizes frustration with its current ability to perform, and if it has some shared understanding of its constraints, then it can potentially begin to remove the constraints immediately.

The following are common issues for agile teams, and represent areas where bottlenecks and entropy collect:
  • Code. Code is often written merely to work and to execute, without consideration for the waste incurred by forcing teammates to decipher code in order to get at its purpose, and to work with it. Code is one of the primary means of communication on an agile team. It must be written to communicate to other programmers on the team, and must quickly generate understanding.
  • Design and Architecture. Systems and software work but are not readily understandable and adaptable by all members of the team. A small set of well understood and familiar patterns are often over-used. Object-oriented design fundamentals are often esoteric and impenetrable during the building of a first or second-generation product. Developers come to understand their significance often only by introducing design friction that is only seen in retrospect as a series of violations of software design laws and principles.
  • Tests. Initially, teams tend to craft tests as quality assurance efforts rather than design, specification, and documentation efforts. Tests should be easy to scan, enabling developers to get an immediate understanding of the system and the impact of the work they need to do, as well as an understanding or where to find their work site (or sites) within the code and supporting artifacts. Tests are the most important form of documentation on an agile team.
  • Development and QA Test Synchronization. Without greater synchronization between development and QA testing, valuable input often comes too late in a development or release cycle to be effective. Test design and test architecture are valuable inputs to development, and software design and architecture are valuable inputs to QA testing. Teams often loose unrealized capacity by not pursuing means to do more development and testing in parallel.
A software team continues to adjust and refine its practices based on the friction it faces and the observations it makes. Entropy still collects.

A critical mass is inevitably reached and the team makes course corrections that are often broader than the iteration-to-iteration practice and tool calibrations.

Thursday, July 10, 2008

Sustaining Capacity in Maturing Agile Software Teams - Part 2: Entropy

Teams that have adopted an agile approach to software development adopt new disciplines in an attempt to reach a level of productivity and effectiveness above their achievements with previous approaches or methods.

There are common practices and disciplines found on most agile teams, including:
  • Developer testing (unit testing, and possibly Test-First Programming)
  • User stories (analysis and scoping)
  • Time-boxed delivery cycles (iterations)
  • Continuous Integration
  • Automation
  • Collaborative work
  • Deep customer involvement
  • Rapid feedback
Agile teams use these skills and tools to deal effectively with the inevitable change that is part of software development.

Previously, the team may have obstructed necessary change because of an inability to use effective counter measures that accommodate the business and the team in adapting to new opportunities and new constraints.

Counter measures often produce inherent bottlenecks that point the to the need for subsequent optimizations. Bottlenecks manifest in the following ways:
  • Code mass growth. By adopting developer testing, the team has volunteered to double the amount of code and systems artifacts that it maintains. It maintains this added code without increasing the team's resources, often gradually degrading the team's capacity.
  • Design rigidity. The team uses traditional software designs that aren't optimized for the rate of change that the surrounding business and the team itself has become acclimated to.
  • Growth constraints. The pace of an agile team and the incremental successes that allows for advancement of a product drives a need to grow the team and its people. New people are brought in to naturally increase capacity, but it takes longer than expected or desired to become meaningful contributors to the effort.
These constraints are symptoms of the entropy that subsequent optimizations should address.

Without addressing the production entropy, it will ultimately constrain the business's ability to take advantage of new opportunities, which will ultimately lead to decreased throughput and entropy in the business.