Toxic Tech Ecosystems

... when nobody can afford to change

TL;DR: Tech debt is accumulated inside organisations. Toxic technology ecosystems emerge across markets, trapping vendors as surely as customers. The challenge is therefore not upgrading legacy systems, but rather restoring the conditions in which innovation becomes possible.

There is a well-known tech debt doom loop that many organisations find themselves caught up in, and then dragged down by. It works like this. A relatively short period of underinvestment increases the backlog of systems in an enterprise that require attention or renewal. The most urgently needed systems, critical to enterprise operation, start flashing red. Because resources are tight, a ‘quick fix’ or workaround is put in place and then frozen into architecture. The complexity of the system increases, making it progressively more difficult, and hence costly, to maintain or enhance.

A good deal of the remaining resource is consumed by vendor mandated upgrades. Either upgrade, or pay enhanced maintenance, or risk managing an unpatched system. It does not help that the original business case provided for acquisition but not for the costs associated with ownership.

Eventually, of course, the urgent squeezes out any strategic or innovative capacity, and the enterprise requires more and more to do less and less, entering a negative spiral.

AI has acquired its own version of the doom loop. All the critical enterprise data remain locked inside legacy systems that also mediate the core business workflows, so what little investment remains for driving change is spent on AI pilots, pathfinders, proofs of concept and demonstrations that have no realistic chance of scaling and wider deployment.

The tech debt doom loop would be bad enough on its own. It is, after all, avoidable and remediable. Organisations can, with determination and investment, dig themselves out. The deeper problem is that there are circumstances in which even well-managed organisations remain trapped because the market around them has ceased to function effectively. Tech debt is principally an organisational problem whilst what follows is a market problem.

Many organisations work in areas that require enterprise software systems to support deeply embedded services. An example to illustrate, in an area I am familiar with, is a student record system that helps higher education institutions manage the key student data and associated workflows such as admissions.

These systems often have long lineages stretching back to the time when the idiosyncratic ‘manual’ business processes in each organisation were initially brought online. At that stage the existing models and workflows were implemented in heavily crafted software systems. It was difficult to envisage doing anything else. Subsequent technical evolution resulted in large database-centric systems surrounded by a dense thicket of bespoke code, scripts and interfaces to other systems. It seemed a sensible architecture at the time, and in any case there were not many choices. A lot of investment got locked into the systems, and they became more and more complex and difficult to change.

Each customer gradually accumulated a unique version of the software, with its own data model, customisations and upgrade history. As discussed above, change became more difficult and the system became more bottlenecked around data access, prompting ad-hoc engineering fixes. This was obviously bad for the customers who could not step away and were naturally reluctant to disrupt processes that were critical but not delivering much direct business value.

It turns out that this is not only bad for customers, it is bad for vendors too. At first it might have appeared that all was good. Customisation generated revenue and increased customer dependence, but it also transformed software companies into labour-intensive service businesses, with all the attendant problems of scale and productivity. After a time, margins increasingly depended on services rather than product innovation, and investment gradually drained away from the platform itself. In some cases financial ownership further reinforced short-term optimisation at the expense of product renewal.

Normally, this is the point at which a new entrant should disrupt the incumbents. In these markets that rarely happens. Switching costs have become so high, implementation so risky and procurement cycles so long that superior products struggle to gain adoption. Competition no longer disciplines suppliers because customers can no longer exercise meaningful choice.

Of course, there is a further consequence. Instead of managing one product, the supplier gradually finds itself maintaining many subtly different products. Configuration management substitutes for product engineering, testing becomes more expensive, upgrades become slower and innovation atrophies.

This is where the problem ceases to be simply one of tech debt and becomes instead one of market structure and dysfunction. Healthy software markets compete by building better products. Toxic technology ecosystems compete by making it harder to leave. Innovation is less profitable than customer retention. Suppliers become increasingly dependent on servicing the installed base rather than renewing the platform, while customers become progressively more dependent on suppliers who have fewer and fewer incentives to innovate. The customers are locked in and so in some sense are the vendors.

The result is a toxic technology ecosystem: customers cannot leave and vendors cannot innovate. New entrants cannot readily break in. Technical debt is accumulated inside organisations but the adverse effects of toxic technology ecosystems accumulate across markets. Tech debt is something an organisation might eventually repay, ecosystem toxicity however is something no organisation can fix on its own.

The answer is not simply to replace one legacy platform with another but rather to reduce the value of lock-in itself. That means simplifying business processes rather than endlessly reproducing historical complexity and agreeing open data standards so that information can move. It means stable interfaces and reference architectures that allow components to be replaced independently and separating data from workflow, so organisations retain ownership of one without being captive to the other.

Ironically, restoring competition requires collaboration. Individual customers negotiate contracts but collectively they can shape markets. When every organisation demands bespoke functionality, every supplier becomes a consultancy. Once organisations converge around common standards, reference architectures and genuinely distinctive requirements, suppliers can compete on the quality of their products rather than on the depth of customer lock-in.

Technology ecosystems become toxic when nobody can afford to change. The only available strategy to address this is to restore the conditions and rebuild the markets in which innovation becomes possible.