Modeling Technical Income
March 06, 2020
For some technical debt is just a list of shouldn’t and best practice a list of should. I find this to be a poor framing and one prone to producing bitterness.
My own perspective is that technical debt and best practices are actually the same thing. That must sound strange - how can they be the same thing: one is supposed to be good and the other bad, right?
Nope - it just seems that way because the utility generation is subject to a horizon effect. In my mental model both lists are derived from how effort gets converted into accomplishment. They are both good. They are both bad. Time windows control the utility perception.
Let me explain what I mean by that.
As a first step, I’m going to ditch the word best practice. I don’t consider it helpful for explaining this, because it doesn’t prima facie suggest its complement.
A lot of best practices and bad practices in software engineering can be better understood through the concepts of technical debt and technical income.
Technical debt is often understood as things which reduce the amount of work that can be completed over a time period per unit of effort expended. Technical income is a term I’ve made up which is the opposite of technical debt. Technical income is those things which increase the amount of work that can be completed over a time period per unit of effort invested.
Often times gaining technical income takes a bit of extra work: it has an acquisition cost beyond the cost of the work being done. In other words, it is the approach that requires a bit more planning and diligent effort. Meanwhile, technical debt is the hasty approach, doing things directly; just accomplishing what is asked as quickly as possible rather than accomplishing it in the way that is best over the long term.
With these definitions in place, we can model both technical debt and technical income as an effort scalar with an acquisition cost. Interestingly, a lot of the common wisdom about engineering excellence and technical debt emerges from the model.
Notice that in short projects all the usual caveats people give about technical debt being fine and the practice of engineering excellence being premature are present. As you add technical debt to a project, it increases the amount of effort needed to accomplish a given unit of work. If a project only needed ten units of accomplishment, you accomplish it more quickly with a hasty approach. The added accomplishment from easing future work doesn’t produce enough benefit to pay for its creation.
However, the benefits of technical income are immense in long term projects. Over short time frames the hasty approach is fine, but as an ongoing trend it will mean disaster. Eventually the amount of work accomplished per effort invested will approach zero with the hasty approach. In contrast, as you add technical income to a project, over the long term it will mean an abundance of accomplishment. Eventually large amounts of work will be accomplishable with minimal amounts of effort investment.
This model shows us that short projects aren’t helped by paying for technical income. The gains from the effort investment take too long to accrue for the investment to be paid back by the time the project ends. The model also shows that longer projects benefit greatly from technical income and are killed by technical debt. Over the long term taking on technical debt without paying it down before the effort scalar diminishes progress accomplished per effort invested kills the project by making it so that effort investment doesn’t accomplish work.
So far we have talked about these things abstractly and modeled them, but without examples it might be hard to tie that to the daily practice of software engineering. Concrete examples taken from industry practices can help to show the direct relationship between the practice and the multiplier on effort.
An example of technical debt is a poorly documented and poorly understood highly manual deployment process. In dysfunctional organizations deployments can take a long time, because only one engineer knows how to do the deployment and the deployment involves many manual bash commands. In order to accomplish a release, this work needs to be done. If someone else wants to accomplish a release, they need to learn the poorly understood system and then build automation around it. This would be the acquisition cost to an easing of future efforts. Unfortunately, the path of least resistance is to just let the manual work be done again by the person who knows how to do it. So that is what will often happen when acting hastily. The manual effort will be expended on every release.
In direct contrast, an example of a best practice that is effectively technical income is not taking the previously mentioned path of least resistance and instead to do the work of automation of releases via a continuous deployment system. Now whenever someone does a commit, in addition to the effort expended to do a commit, there is also the additional work accomplished of doing a release. This is the practical impact of the scalar on effort: a bit of effort is applied, but more work gets accomplished.
Creating a long list of technical best practices could seem arbitrary, but when seen through this lens of an effort scalar it is just recommending sets of things which have the effect of increasing the amount of work done per unit of effort invested.
- Documentation increases understanding acquisition per effort investment on the part of the reader, but it also effectively the automation of an explanation from the perspective of the writer. They give the explanation once, but each time they would have been asked in the future, the documentation is able to provide the answer.
- Tests automate verification processes. So potentially expensive verification becomes relatively effortless.
- Modularity facilitating code reuse reduces the effort needed to accomplish the next derived feature. The first visualization took a bit of effort, but the second only minutes.
So lets spend a bit more time considering. What does it look like when you are in the position of having a lot of technical debt? What does it look like when you have technical income?
In a dysfunctional organization with high technical debt, the activation cost of moving from technical debt to technical income is higher, not just because of the distance that the effort scalar needs to move, but also because effort doesn’t accomplish as much so over a time period, so it can look like total accomplishment is lost over a time period by making the effort. Short sighted thinking in such an environment, even when it expands its time horizon, could still direct toward a suboptimal long term approach.
Meanwhile, in a thriving organization, the acquisition cost of gaining even more technical income is lower; effort already accomplishes a lot, with ease. There is more effort to spare and the benefit of long term thinking is more apparent.
So one of the differences between a dysfunctional organizations and a thriving organization is that the dysfunctional organization will care more about the short term and the thriving organizations will care more about the long term.
Before you start a project, ask: how long will this codebase be alive? That answer determines whether haste or diligence maximizes total output. For a weekend prototype, skip the tests. For anything you expect to maintain, invest in the effort scalar. The acquisition cost pays for itself exactly once, and then it keeps paying.