Streamlining cloud migration with observability and AI

The Scottish Government is pressing ahead with its digital transformation plans. In its bid to deliver economic recovery and build more flexible, responsive and resilient IT services, cloud computing is an essential part of these plans. However, cloud isn’t a panacea and can introduce new risk if migration is not properly managed.

To stand the best chance of success, observability and AI technologies are key. Such technologies deliver visibility into distributed systems and can automatically identify performance, security and other issues.

A shared vision

Driven by the Scottish Government and the Convention of Scottish Local Authorities (Cosla), the Digital Strategy for Scotland is ambitious in scope. It promises to, “ensure services are designed to meet the needs of the user, to deliver economic recovery, to meet climate change targets and to ensure that everyone in Scotland has the skills, connectivity and devices required to fully participate in our digital nation”.

Cloud migration forms a critical pillar of the strategy. The Cloud First programme will support public sector organisations as they optimise their use of cloud, as well as promote best practices and knowledge sharing. The aim is to make public cloud the default delivery model for public services, in recognition of the fact that it can offer greater environmental sustainability, operational and cost efficiency, flexibility, resilience and cyber security, while accelerating delivery times.

It's great to see the government’s Cloud Migration Service already turning words into action with the first two pilot projects in this space.

Driving insight

However, cloud migration is no easy feat. The benefits speak for themselves but realising them is another matter. Some estimates suggest as many as three-quarters (74 per cent) of cloud projects end in failure, with applications being moved back to on-premises infrastructure. 

One of the biggest challenges facing organisations beginning this journey is the lack of observability they have in cloud environments. These systems are dynamic and highly distributed, making visibility into infrastructure and the services running on top more difficult than in traditional datacentres.

This is compounded by the fact that most organisations use multiple cloud providers in a hybrid setup. In fact, research reveals that, on average, 92 per cent of organisations have a multi-cloud strategy and 80 per cent plan hybrid deployments. That means any monitoring and migration activity must be well coordinated. 

Finally, because cloud architectures are distributed and dynamic, performance can be up and down. That demands IT managers architect their systems carefully, and continuously monitor and automatically remediate any performance issues.

This is where IT leaders need to get proactive, to streamline migration and minimise the chances of any downtime. AI and observability can be their key allies—delivering automatic discovery, topology mapping and precision analysis. 

Observability is the key

At its core, observability is about measuring a system's current state based on the data it generates. That makes it particularly suited for dynamic and complex environments like cloud, and delicate projects like migration where accuracy is essential. 

It can help first with the initial assessment and discovery phase of a migration project. Observability tools assist IT teams by revealing how employees use various apps and how frequently they do so. This can help them eliminate rarely used applications from the project and shortlist candidates for piloting. Smart auto-discovery tools can also assist by rapidly delivering a visualisation of all topological dependencies across infrastructure, process and services. 

Right-sizing infrastructure is another important part of cloud migration that helps to eliminate CPU and memory waste and keep costs down. Observability tools can reveal current and historic CPU, memory, disk, and network usage to identify if a server is optimized or not, as well as recommend CPU and memory sizes that are right-sized to match workload performance. Given that performance and capacity requirements change over time, this is a process that should be run continuously.

If performance issues are impacting user experience once the migration is complete, IT teams need to quickly address them, otherwise the migration project will be seen as a failure.

This is where causal AI tools come in, by automatically detecting performance problems in application, services and the infrastructure stack. The technology can identify root cause with a high degree of accuracy by using contextual data like topology, transaction and code-level information to find events that share a common source.

This reduces alert storms, accelerates and automates remediation, and takes the pressure off stretched IT teams. But most importantly, it will help to make a success of any public sector cloud migration project.

This article was sponsored by Dynatrace