This post will analyze the cost of technical debt in software development niche.

It will explain how teams can accrue technical debts and opinions on whether it is a good or bad thing.

Software development teams balance several constraints to create software. These constraints can require compromises in the software’s design. 

Such compromises, especially in the developmental and operational qualities of the software system, are often referred to as technical debt, a metaphor coined by Ward Cunningham that draws on how we reason about financial debt to explain these compromises.

Cruft — deficiencies in internal quality make it harder than it would ideally be to modify further and extend the system.

In TechnicalDebt, Martin Fowler

A Story of AwesomeProduct 

We are building an MVP of a project called AwesomeProduct. We are to release version 1 next Friday, and we shall have cake and nuts for celebration. We are called into a quick meeting and informed that we need to make some late changes.

AwesomeProduct has an exporter module that exports users’ data in either CSV or JSON format.

Now we need to add a third format, XML (yuck!).

These new changes provoke our righteous frustration.

1. Why is this change coming at this time? 
2. We can’t get that in before Friday, and we’ve already frozen development! 
3. These changes will require modification of our design, and we didn’t see it coming.

Our Product Manager explains that we’ve just got three new customers willing to pay big for our product. Our demo impressed them. But they have some systems they would like to integrate with it.

One has an existing system for managing data exports in XML; if we provide a DTD, they can integrate our system with theirs easily. Another customer would like to automate their data export; only if they can fetch the data from an API… Of course, we can give them that.

These new requirements are not complex, but our current design does not accommodate them nicely. We have three options here. 

1. Extend the launch date and do a proper redesign to provide more flexibility in the exporter. 
2. Make a quick fix and launch as planned. We can improve the design later. 
3. Offer to provide the extra functionalities in a future release.

Only (1) and (3) gives us time to reconsider the system’s design properly. However, there are several reasons why (2) may be chosen, and it often is. 

By relaxing our design process to allow for an early launch, we have incurred technical debt. It’s like borrowing cash to aid our launch, but we borrow time and velocity from our future.

Deciding How Much to Borrow 

As with financial debt, we must choose how much we borrow at a time. It is a bargain among stakeholders. 

In the AwesomeProduct story above, the development team took on more work within the organizational cycle and compromised design to enable an early launch. As with any multi-party relationship, we must balance such compromises to maintain harmony.

It is also an exercise in prioritizing several needs. Here we take on debt to speed delivery, but this trade-off can yield its supposed benefit only within limits, as Martin Fowler explained in his DesignStaminaHypothesis: 

You can save short-term time by neglecting design, but this accumulates technical debt, which will slow your productivity later. Putting effort into your software’s design improves your project’s stamina, allowing you to go faster for long

Martin Fowler, 2007

However we choose, and whatever we prioritize, we must make a choice.

Trading off good design deliberately or inadvertently for another goal creates Technical Debt.

• Some teams disregard deliberate system design as an artifact of a bygone waterfall era. In return, they get a system design whose properties are accidental results of opaque interactions of various parts of the system; such designs are challenging to understand. 
• Economic pressures can force a pragmatic decision: ship first and deal with the consequences later. 
• Ignorance of good design practices can result in technical debt. Maybe the team doesn’t know better, and the organization disregards principled software development practices. 
• Novel problems and unfamiliar domains often leave some technical debt in their first solutions because solving the problem is an experiment that yields insight into better solutions after creating the first solution.

Technical Debt Is Not Technical 

The metaphor “technical debt” is biased towards the effects of degraded design on software development. Still, several factors contribute to technical debt, as we have seen from the preceding discussion, and these factors are not all technical.

Economic Constraints 

The AwesomeProduct example presented earlier shows how the need for sales can influence decisions on design quality. 

The team developing AwesomeProduct may be part of an organization that needs funding.

They’re probably a startup that is taking a chance at getting into the market.

That could mean that they have less bargaining power than the customer who showed up with lots of cash, and early success is a welcome tradeoff for them.

Organizational Structure 

The organizational structure of a development team influences their outcomes. As stated by the famous adage “Conway’s law”,. 

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure”. 

Conway, 1968

We can observe that loosely-coupled development teams produce more modular architectures, while tightly-coupled teams produce tightly-coupled architectures. 

Teams that depend on water cooler chats to share knowledge can develop faster initially. 

Still, the assumptions of their systems become implicit tribal knowledge rather than explicit design features and subsequent development, and onboarding new team members becomes problematic with the many assumptions held in mind while working on any part of the system. 

DevOps, DevSecOps, and other variants intend to change organizational communication structures in advantageous ways to the development process.

Development Approach and Methodologies 

How a development team approaches software development — the processes, practices, and tools (including programming languages) that they adopt — influences the quality of design they produce. 

• Test-driven development (TDD) can sustain code quality as development progresses, but, alone, it is insufficient; organizational circumstances can cripple it. 
• Although testing can inform design, inevitable design mistakes are caught by only experienced software developers’ careful observation. Code reviews are meant to catch such errors, but small teams may not have the necessary structure and conduct to conduct good code reviews. 
• Code reviews are not meant to catch every code factor that contributes to technical debt. Enforcing code formatting, dependency graph restrictions, code coverage by tests, naming conventions, dead code removal, cyclomatic complexity measures, etc., manually can be drudgery. These are better done in continuous integration pipelines. 
• Inconsistent language (how things are named) in a project increases the cognitive burden required to understand the system. It often occurs when the stakeholders communicate using different languages, and this incoherence of language makes its way into the source code. Domain-driven design tackles this problem by advocating the use of ubiquitous language throughout the code. As Einar Høst explained in his DDD Europe, 2019 talk, this is a modeling problem; technical debt isn’t technical. 
• Some programming languages are poorly suited for large software projects. Some organizations enforce the use of specific languages, even in cases where they don’t fit. As projects grow more complex, the limitations of the programming language inspire “workarounds” that may seem ingenious in the present but will cause lots of pain in the future.

The Cost of Technical Debt in software development

Technical debt would not be a problem if it cost us nothing; unfortunately, it costs a lot. Every new feature we build on the existing debt counts as interest on the debt because it compounds the effort needed to fix it. Unlike financial debt, technical debt isn’t easy to measure.

Here are some ways technical debt can cost you precious resources.

Reduced Development Velocity 

When the design of a software system is substandard, it becomes unclear where things should go. 

Developers spend more time trying to understand the system than actually extending it. If they try to develop it blindly or work around the bad design, they lose an opportunity to fix the problem, and the cost of future fixes increases. 

If the design is not fixed soon enough, further development may become too expensive to undertake. It can grind the project to a halt, resulting in wasted time and money. 

A Hacker News commenter shared their experience working on Oracle Database 12.2, describing it as unimaginable horror. Their account of the mess is instructive to any organization developing software systems.

Team Attrition 

If the software’s design continues to deteriorate, members of the development team may feel helpless and lose interest in the project. 

It is very likely if the project’s leadership does not prioritize design work and continually sacrifices design improvement for more features. Eventually, team members will leave, taking with them all the experience they have on the project. 

Such systems are almost certainly poorly documented (how can they write what they don’t understand?), making such personnel expensive.

Increased Onboarding Cost 

During the life cycle of a project, team members leave, and new team members join. 

Getting the new team members to understand the system well enough to work on it is necessary. If the technical debt in the system is too much, new team members will take much longer to get productive. 

This loss of productive person-hours is one of the opportunity costs of technical debt. If word gets out about the state of things inside, people may not want to work on your product at all.

The commenter in the Hacker News submission above closed his entry thus: 

Contagion 

In A Taxonomy of Tech Debt, Bill Clark outlined three metrics used in measuring technical debt on his team at Riot Games. The three metrics are viz, impacted, fixed cost, and contagion. Of the three, contagion is an interesting one. 

Contagion refers to how much technical debt can spread to other parts of a project. Highly contagious debt can reach throughout a project and become too burdensome to pay off. 

As it spreads, it increases both the cost of rework and, likely, its impact. An example of a very contagious technical debt is a bad data model. Whether in the persistence layer or an API, a bad data model forces every data model user to understand its quirks and deal with them. 

Imagine that this data model is in the database schema. If a new table is added, and that table has to reference data in the tables with the bad data models, it gradually gets the plague the more it references them.

Every system that consumes data from those tables gets the plague. If the data goes into logs, the logs get it too. Don’t let it escape into responses to clients. And know that data migrations are probably the most expensive and risky refactorings to attempt. 

If not fixed early, technical debt can spread so much that it becomes a system feature.

Conclusion 

Sometimes it is necessary to compromise on the internal quality as a tradeoff for other goals. Such compromises are known as technical debt, a valuable metaphor for framing how we think about such compromises. 

Several factors, including social and economic factors, contribute to the development of technical debt. If not paid back on time, the cost of technical debt in software development can be unpleasant effects on a software project. 

With diligence and a combination of several techniques and practices, technical debt can be managed effectively.