The secrets of delivering AI at scale

In September 2024, the British Computer Society (BCS) hosted an event to explore these issues and discuss the implications of AI at scale. Sponsored by Pivotl, the event brought together an expert panel featuring Rashik Parmar, Group CEO of BCS, Yvonne Gallagher, Director of Digital for the National Audit Office (NAO), James Herbert, CEO of Pivotl , and Alan Brown, Professor of Digital Economy at the University of Exeter.

In a lively discussion with a broad audience of digital leaders and practitioners, the event raised several key points and insights

Digital transformation: gradual, not uniform

• Key point: not all organisations are advancing at the same pace in digital transformation. While industries like music and media have experienced rapid shifts, others continue to adapt more gradually
• Insight: organisations may have more time than expected to get their digital strategies right, but success requires tailored approaches and long term focus rather than short term blanket solutions

The AI bubble and scale challenge

• Key point: a growing concern is whether AI will be a bubble, similar to the dot-com bubble of the early 2000s. While AI shows great promise, there are doubts about where it delivers value and whether the current levels of investment, especially in high performance computing (GPUs, electricity, cloud infrastructure), can be sustained profitably
• Insight: if organisations cannot invest sufficiently to deliver AI at scale, especially due to high costs of compute resources, the bubble may burst. But, like the dot-com era, valuable skills, infrastructure and lessons will remain

Cost and consumption of AI technologies

• Key point: all organisations that have successfully implemented large language models (LLMs), including government agencies, are finding that the cost of maintaining these systems can be prohibitive
• Insight: the high cost of AI tools is a major bottleneck to adoption at scale. This cost barrier may slow down or even reverse some of the progress in AI deployment

Leadership and knowledge gaps in digital

• Key point: there’s a substantial leadership gap when it comes to understanding and integrating digital technologies. Educational programmes are lagging in helping leaders to gain sufficient digital knowledge
• Insight: leaders at all levels in the organisation need a better grasp of digital transformation, technology integration and the nuances of AI delivery. The lack of digital literacy, especially among top executives, is contributing to slow and inefficient adoption of AI

Technological disconnect

• Key Point: there is a disconnect between the technical teams and decision makers in large organisations, often due to outdated ideas about technology capabilities and its adoption
• Insight: both IT and data professionals and non-technical leaders must work together to address modern digital challenges. The governance and leadership structures in organisations must evolve to support new collaboration and decision processes to allow digital transformation to succeed

Scale and innovation

• Key point: there are challenges in implementing change and innovation at the right scale. Organisations often get stuck in a particular operational mode, unable to scale up or down effectively
• Insight: organisations need to develop capabilities to operate and innovate at different scales. This may mean rethinking existing processes or structures that are not designed to support flexibility in innovation efforts

Learning from the past

• Key point: just as the dot-com bubble left behind valuable infrastructure, the potential bursting of the AI bubble could lead to lasting improvements in technology and infrastructure that will benefit future developments
• Insight: even if there is a downturn in AI investment or usage in the coming months, the foundational advances made will serve as the building blocks for future innovation, especially in computing infrastructure and digital literacy

The importance of contextual decision making

• Key point: lessons from previous technological change show that organisations fail at digital initiatives when they don’t match the technology choices to the current context, scale and risk profile
• Insight: decision making around AI technology adoption needs to be informed by the specific needs and circumstances of the organisation. Early decisions, particularly around procurement and leadership, have a long term impact on whether AI at scale efforts succeed or fail

A call to action for digital leaders and practitioners

• Key point: there is a need for more robust digital leadership and professionalism when delivering effective AI systems and solutions, with organisations such as BCS stepping up to provide guidance and support
• Insight: developing a cadre of leaders and practitioners who understand the intricacies of AI and digital transformation is crucial. This is not just a technical issue but one that spans strategy, leadership, governance and education

Conclusion

As AI becomes more widely deployed, its broad use raises opportunities and opens up risks. How organisations choose to face this dilemma will be fundamental to whether they succeed or fail.

For you

Be part of something bigger, join BCS, The Chartered Institute for IT.

This event highlighted that success with delivering AI at scale requires additional focus on the complexities of adopting it, overcoming the cost barriers of AI use, and bridging the leadership gaps that many organisations face today. While AI has the potential to transform industries, its current trajectory risks creating a bubble. The lessons from past technology bubbles, however, suggest that even if some investments falter, the long term infrastructure and knowledge gains will propel future innovation. Leaders must bridge the digital knowledge gap and adapt decision making processes to ensure that digital transformation efforts are contextually appropriate and scalable.

For those interested in learning more about these topics, Professor Alan Brown’s new book Surviving and Thriving in the Age of AI - A handbook for Digital Leaders offers detailed background, examples, and practical guidance. More details are available at www.SurviveAIBook.com.

This overview was produced by an LLM, and edited by a BCS editor.