Beyond the Cloud

... how edge computing is changing where AI happens

TL;DR: AI is not leaving the cloud, but it is not confined to it. As digital systems become embedded in physical infrastructure, constraints such as latency, bandwidth, resilience and energy consumption make it necessary to process data closer to where it is generated. Edge computing extends the cloud, enabling real-time, distributed intelligence. It is one of the most important frontiers in AI and receives insufficient attention.

Naturally enough, a great deal of attention is being paid to the cloud, particularly in the context of AI. The architectural centre of gravity, it is argued, has shifted decisively away from the enterprise and into hyperscale infrastructure. This is another @profserious mild corrective: the cloud has not solved everything … no architecture ever does. What follows is not an argument against the cloud, but against assuming it is the only place that matters.

In what follows I address a much less discussed shift, the (re-)emergence of computation at the edge. That is, computing close to where the data is generated. Edge computing is not a wholesale repudiation of the cloud model, but rather a recognition that when computing is embedded in the real-world fundamental limitations associated with physics and economics necessarily assert themselves. There are four key constraints to consider:

Latency. When a computation is embedded in the physical world, milliseconds become material. Whilst a delayed chat is an inconvenience, a delayed brake signal results in an accident. The round-trip to the cloud is simply not viable.

Bandwidth. We are generating data at extraordinary scale. Cameras, sensors, telemetry streams and embedded devices produce more information than it is sensible, or affordable, to transmit in raw form. Filtering and inference must happen near the point of generation.

Resilience. A centralised architecture generally assumes permanent connectivity. That assumption does not hold in critical infrastructure, contested environments, or, for that matter, in most industrial and urban systems.

Energy. Moving data consumes power. Computing close to the data can minimise this and potentially reduce global data centre loads. It also supports broader sustainability goals through local resource optimisation.

To illustrate this, consider an electricity substation forming part of critical national infrastructure. It generates continuous telemetry: load fluctuations, frequency variation, transformer temperature, vibration signatures, switching events, and so on. AI models monitor these streams to detect anomalies, predict component failure and identify potential malicious interference. When instability occurs, response time must be immediate to prevent cascade failure. If the decision loop depends on compute in the cloud, latency gives rise to operational risk. If connectivity is degraded, whether by fault, through congestion or attack, control may be unavailable at precisely the moment it is needed. Raw telemetry at scale is neither efficient, nor necessarily appropriate, to transmit wholesale beyond secure boundaries. Processing, filtering and anomaly detection must therefore occur locally, at the network edge. The cloud still has a role: training models across substations, aggregating long-term performance data and informing planning. But operational decisions are local.

What is new here is not distribution per se. Distributed systems have been about for a long time. What is new is the scale of data accompanied by the embedding of AI inference into physical systems for applications such as vision, predictive analytics and anomaly detection, all with the expectation of near-instantaneous response. Edge computing is a shift to accommodate this.

There is an emerging conjunction of key technologies that enable this shift. First and foremost, 5G and software defined networking (SDN). 5G with SDN provides ultra-low latency with network slicing, the ability to construct multiple ‘virtual’ networks with different properties, thus enabling reliable edge workloads. These technologies are complemented by frameworks for running compute services at telecom network edges (for instance, ETSI Multi-access Edge Computing (MEC)). Network operators are deploying national-scale edge infrastructures including mini-data-centres and MEC platforms.

New generations of edge AI accelerators and inference-optimised processors (eg NVIDIA Jetson-class devices, ARM NPUs, Google Edge TPU etc.) are being developed specifically for edge contexts, yielding local compute power and energy efficiency. These are associated with standardised approaches to deploying machine learning models across platforms.

In this context then, these are the key things you need to know:

AI changes the game. Edge becomes compelling when models are deployed locally for inference. Training remains centralised, but decision-making becomes distributed. This unlocks applications hitherto impractical in autonomy, real-time analytics, immersive tech, and more.

Orchestration is the primary challenge. Running software across a large number of distributed nodes is operationally complex. You need update mechanisms, configuration management, observability and lifecycle control.

Security is structurally harder. Centralised systems can (to some extent) be ring-fenced. Edge systems multiply endpoints and the attack surface expands with every sensor deployed. Identity management, secure boot, device attestation and zero-trust architectures become foundational rather than optional.

Edge reduces bandwidth and latency costs, but increases hardware and management overhead. The trade-offs are not straightforward and necessarily must be analysed systemically. Edge decisions are optimisation problems. Latency, bandwidth, energy, hardware cost, risk exposure and regulatory constraint must be modelled together. The correct answer is rarely ‘edge everywhere’ or ‘cloud everywhere’.

Governance and oversight becomes more complex. Processing locally may reduce dependencies and improve compliance, but it complicates auditability, and potentially accountability. When computation sits in buildings, vehicles, supply chains and physical infrastructure, responsibility fragments unless deliberately re-assembled.

Distribution is not merely a technical question. Data governance, regulatory compliance and security considerations increasingly shape where processing might lawfully and strategically take place. Where computation occurs determines who governs it, who can access it, and, perhaps, who can interrupt it.

Architecture is the key. Buying ‘edge devices’ does not constitute an edge strategy. Edge computing depends less on any single so-called breakthrough technology and far more on integration across layers: network capability, distributed compute, orchestration, security, governance and operations working coherently as a system. The greatest benefits accrue to those organisations that can reason across those layers and treat the whole as an architecture rather than a collection of products.

The edge-to-cloud continuum is emerging. Rather than edge versus cloud, we now see hybrid approaches that seamlessly integrate local processing, hubs and cloud for orchestration, storage and model training, giving systems resilience and scale.

Edge computing is increasingly where technical innovation is concentrated. As systems become more distributed and more tightly coupled to the physical world, the most interesting design challenges are moving outward from the cloud to the edge. For AI in particular, the ability to deploy intelligence at the point of action is technically significant, strategically important, and merits your close attention.