How Should Companies Best Staff the Teams That Perform Software Maintenance on Their Systems?

With some 80% of a system’s cost over its life cycle stemming from maintenance, this can easily be a million-dollar question, especially for larger companies spending sizable sums on maintenance every year.

Michel Benaroch, professor of information systems, and Kalle Lyytinen, the Iris S. Wolstein Professor of Management Design at Case Western Reserve University's Weatherhead School of Management, provide some answers in a paper recently published in IEEE Transactions on Software Engineering.

Their study considers how certain team attributes—instability and knowledge-diversity—interact with different levels of software complexity. Even as a system evolves with bug fixes and added or expanded features over a life span of as many as 25 or 40 years, “the system side is challenging but relatively stable and manageable,” says Benaroch. The teams, by contrast, are typically instable, frequently changing their membership as individuals leave for new jobs or promotions or join to bring in specific expertise.

“Theory and intuition say that perhaps you need to have a stable team to work on a more complex system, with many members who have worked on it and with each other in the past, so their productivity is going to be higher,” says Benaroch. “But a number of previous studies actually found the opposite. This is where our study jumps in.”

Instead of looking at a single complex system in a lab setting, as prior studies did, the authors worked with rich, real-world data from a large financial institution. The extensive portfolio covers over 400 systems maintained by thousands of personnel in teams across 30 countries, allowing the researchers to see how software complexity varies within and across systems.

Benaroch and Lyytinen also assumed a nonlinear exponential, rather than linear, relationship between software complexity and maintenance productivity. Above a certain level, even a small increase in complexity results in a big drop in productivity. This updated view, supported by past research, allowed them to distinguish between systems with high and low levels of complexity. “That's where we find that the results correspond with intuition and with theory,” says Benaroch. “When system complexity grows very high, the instability of the team is detrimental to maintenance productivity.” 

In addition, the researchers examined the diversity of knowledge and skills within teams, for example, specialists in database design, information security or system resilience. They found that in high complexity systems, high diversity of skill sets lowers productivity, as experts in different fields, speaking different specialized languages, need more information exchanges and coordination to get on the same page.

“The takeaway for companies is that you can’t treat all systems the same,” Benaroch summarizes. “You really need to pay attention when things are very complex, because that's when you're going to start paying the price for poorly staffing teams based on who is available at the moment and based on what additional expertise are needed on the team to make up for lack of familiarity with a system.” Conversely, “if our findings can allow an organization to become just 5 or 10% more productive on maintenance, that can be translated to large amounts of money on an annual basis.”

Benaroch, M., and Lyytinen, K., "How Much Does Software Complexity Matter for Maintenance Productivity? The Link Between Team Instability and Diversity," IEEE Transactions on Software Engineering, 2002, 1-17.