Software is conquering the world. It is ubiquitous, representing the foundation of nearly every industry today. Our modern society would be unthinkable without the trillions of code lines that move machines, bank transactions, planes and smartphone messages. All of this marks a great disruption, where the “old economy” struggles for having to compete with digitally born companies: Volkswagen against Tesla, and Deutsche Bank against the new FinTechs. The old economy’s only chance sits in creating self-written software systems, the magic ingredient that also lies in the heart of every digitally born company, enabling them to offer unseen services to their customers.
Along with the transformation of business models, the anatomy of a traditional company changes so that it evolves into a software company itself. A standard company comes to contain a software development organization within, bringing the need to optimize resources due to a current shortage of developers on the contemporary job market. The strategy for doing so usually consists of having as many developers employed as possible, while renting additional engineers from development service providers.
It goes without saying that these companies’ survival and victory depends upon building up software development organizations that run like a well-oiled machine, where developers can be as productive as possible as they perform their tasks. However, the concept of developer “productivity” needs to be handled with care; we consider it as something that depends less on the developer as a person, and more on the context in which the developer must work, such as the user requirements to be implemented, the existing code to be touched, or the processes to be followed. Often this context forms the main impediment to running an efficient software development organization. They impact negatively on the budget but many of them can be remedied through software analytics.
In our series on how analytics creates extra power for your software development organization we focus on the 3 main factors that drain the software development organization’s budgets: 1) Defect fixing, 2) actively managing technical debt, and 3) bad knowledge distribution. While we deep dived into defect fixing already, this blog post explores the most frequently given answer when developers are asked about which aspects slow them down the most: Technical Debt. In Image 1 you can see that the process of the developer stepping into technical debt and removing it can easily drain a significant part of your software development budget.
Image 1: What is using up your software development budget?
The term technical debt refers to problematic code structures in the existing code base that cause chronic headaches for developers, but also can’t be removed so easily from the code base. As there isn’t direct benefit for the customer/user from the clean-up investment, developers don’t get the time for the clean-up process. Using language from the financial domain, developers try to persuade that they were forced to accumulate technical debt in the past due to feature pressure, and that they should now find the time to clean up the code. This process functions in the same way as accumulating debt with a bank: it’s inevitable that you’ll have to pay back your debt at some point in the future.
However, there is no feasible method yet of measuring technical debt in financial terms. Besides, if you compare technical debt and financial debt the two concepts are very different. In the case of technical debt, the interest rate is unknown, constantly changing, and able to vary from 0% to an absurdly high value. Because of this degree of uncertainty, removing technical debt is much like entering the lottery; it could transpire that you never have to pay any interest or money for the debt at all. On the other hand, it is also possible for the interest rate to jump to an extremely high level, and for you to promptly curse yourself that you were silly enough to take on this burden in the first place. The reason for this fluctuation is that you can never know if you will have to touch your “finished” code again.
In an ideal world, a developer would take a user requirement, translate it into code, add it to the code base and then move on to the next requirement. In practice, however, the code for any new user requirement needs to be implanted into the existing code base, meaning that the existing code needs to be understood and touched by a developer. The uncertainty of the interest payments comes from the fact that you’re unaware of which user requirements will come up in future; if you’re lucky, the ugly code areas with technical debt are just lying somewhere unseen, where no developer will have to look at them again. In a rather less fortunate situation, fresh user requirements might be of a nature that forces the developers to open Pandora’s box right in the areas containing the ugly code.
Managing technical debt successfully relies on broadening the view, so that you’re simultaneously managing the highly fluctuating interest payments as well as the technical debt itself.
The focus on the interest payments lies in the operational dimension, where you come to understand how the technical debt is currently affecting your developers’ productivity. It’s important to have a method for real-time monitoring in place, which allows for measuring the daily interest rate. With this, you’re able to react quickly in situations of a sudden interest increase: much like a surgeon, you’re able to perform small and focused emergency operations on the lines of code responsible for the high payments. At the same time, real-time monitoring provides you with value information that you can collect and use for reducing technical debt on a strategic level. Code areas frequently involved in interest payments are good candidates for the code clean-ups planned on more of a long-term scale.
The focus on the actual technical debt is the strategic dimension, which concerns where you should plan and execute smart, long-term investments in view of reducing technical debt. Again, the precise goal is not to reduce the overall amount of technical debt; the aim is rather to minimize the future interest payments that are caused by technical debt.
As a first, basic requirement, you should obtain an overview of the existing debt to allow you to notice the sleeping risks in your code. It’s then important to predict the parts of the code that might be affected by upcoming work on new user requirements, a process made possible by the following steps:
1. First, using the observations from measuring the daily interest rate (operational dimension), which can be considered as a highly reliable source of information. Code areas that have often been the root cause of high interest payments in the past are just as likely to bring about high payments in the future.
2. As a second source, you can also seek the advice of your product roadmap. Choose each upcoming user requirement, draw associations between existing features and functional proximity, and determine the code units that implement these features. It’s worth noting that some form of analytics is required in order to be able to do so.
Given that you can now anticipate which code areas will be touched in the upcoming development cycles and understand if these code areas contain technical debt, you can plan and execute a highly accurate strategy for reducing technical debt.
Moving forward from this more abstract introduction on interest payments on technical debt, I will now demonstrate how you can boost your developers’ productivity with appropriate management of technical debt. Some astonishing numbers demonstrate how much productivity you can carve out of nothing, provided that you’ve understood the concept of interest payments. These numbers are the result of analysis performed for the open-source software system Hadoop, for which you can find details below:
● 18 developers
● They encounter technical debt and experience efficiency loss during 27% of the time.
● The loss corresponds to 2.5 team members
● De-facto team power = 15.5 developers
After applying analytics and eliminating technical debt at a few areas in the code:
● No more interest payments
● 18 developers can work without efficiency loss
→Productivity boost of 16% (15.5 developers → 18 developers)
I have picked out one KPI here to illustrate how you can boost the productivity of developers. If you want to learn more, then download my whitepaper on “Actively managing technical debt”.
For the purpose of getting a grip on technical debt, I have chosen a particularly prominent flavor of it, which is black-and-white to developers across the world and exists as an issue regardless of the programming language that they use. This issue is that of nesting complexity.
Endless discussions between developers can arise as you argue that a specific code characteristic is generally harmful. No such discussion surrounds the concept of nesting complexity, where common sense in the developer community dictates that there is absolutely no reason to have a code line that is wrapped into more than four nesting levels.
The KPI for Complexity on the Seerene analytics platform takes the number of code lines in those deep nesting levels as an indicator for the (intra-modular) complexity of a code unit (Image 2):
Image 2: Example of nesting complexity in a code.
This map provides the basic information needed to understand where sleeping risks lie in the code base (Image 3):
Image 3: This is how the KPI for Complexity distributes across the code landscape. The ‘buildings’ represent the code units, which are organized according to the modular structure of the architecture to form ‘city districts’. The height of each building reveals the amount of complex, nested code that can be found in the respective code unit.
However, this information alone is not actionable, and it’s difficult to know whether you’ll gain a return-on-investment (ROI) if, for example, you clean up the orange CBZip2OutputStream.java file. In addition, you need to look at where the coding efforts flow exactly and then in combination the effort (in person days) that flows into a code unit where complexity exists. Based on the code commits found in the code repositories, Seerene’s analytics platform analyses activities per developer, distributes each developer’s coding time across their individual actions, and finally distributes the actions times across the modified code units in the architecture. If you now combine the information on effort flow and where it hits complex code, you’ll receive a very actionable map that tells you where developers’ brains needed to cope with technical debt (Image 4). The map shows you exactly the code that makes the developers slow.
Image 4: This is how the KPI Effort (developer time) distributes across the code landscape. More precisly, it only shows the fraction of the overall effort that is spent in complex code. Hence, the map reveals which code units are the reason for slowing down the developers. Reducing complexity (technical debt) at these code units will boost the overall developer productivity.
For a more detailed description how the KPIs Effort (developer time) and Complexity (technical debt) work together, you can look at this whitepaper.
To summarize, in part 2 of our series of articles on optimizing the software development organization, I have elaborated on the concept of technical debt and how having the appropriate analytics tools at hand provides a straight-forward process for identifying the code units that force your developers to waste their time on paying interest on technical debt. With these tools, technical debt issues swiftly present themselves as the low-hanging fruits that should be removed. An astonishing boost in productivity comes as the reward and thereby enables software development teams securing time needed to innovate their systems and make progress. Find out more about how you can successfully implement software analytics in your company.