Internet-enabled wrenches and BCS impact

A member of F-TAG since the very beginning, Professor James Davenport FBCS dives into his thoughts on the impact of BCS, the dangers of quantum to cybersecurity, and whether we can (or should) really build unbiased AI. Finally, while BCS Fellows might not dream of electric sheep, we discover why this one can drive real ones across London Bridge.

You've been involved with BCS a long time. What has it given you?

One very important thing is wide exposure to computing. My own research is in a useful, important but pretty specialised area, and BCS represents a whole profession. It's important to be exposed to the bigger picture on a routine basis, and it’s given me the ability to represent the profession much more widely than it would if I were just representing my particular research topic. Much as I enjoy my research, when it comes to speaking about what we need and what we do, we need to teach our students to go into the whole width of the profession. So having wide access to fellow members who work in various industries and positions has been invaluable.

I'm quite involved with a local company that writes safety-critical software for air traffic control (ATC), for example, it has been through BCS that I was exposed to the intricacies of designing safe websites; I’ve been able to speak to people who work on web design, handling money in web shops and online banking applications. It’s not just the flashy side of web design — they’re making it trustworthy, non-hackable, and guaranteed safe with the money. I’ve also been able to speak with members of the Information System Security Specialist Group (ISSG) in particular, as well as BCS branches.

I've been quite heavily involved in the management of university IT for about 10 years. Half the time I was a research professor and half the time, director of IT at the university. I was responsible for our data centres and, for example, welcomed contacts with the Data Centre Specialist Group — something the average professor would not necessarily be involved in. I've since been involved in the Isambard project in the Southwest, which has the world's first ARM-based supercomputers. The Met Office has the first two, and the third is going to a containerised data centre in north Bristol.

Through BCS, I have the ability to ask a professional: what are your five big ‘gotchas’? That sort of informal question is something that isn’t answered by papers or academic talks — but it’s what you really want to know. One I remember in particular is: ‘make sure that the electricity bill includes the cost of the air conditioning as well as the cost of computers’. It was clearly learned the hard way —  that’s the sort of thing you only get by asking battle-scarred pros.

That's who you meet in BCS; bright young things who you hope to have some influence on and battle-scarred professionals.

Can you tell us about your research area?

My research area is computer algebra, which is getting computers to do mathematics. We don't think of them as doing large, large sums, but for algebra and formulae which are too big for humans to handle.

For example, predicting where the moon will be in two years’ time so we can plan a moon landing. There's a fair amount of algebra behind that. Then there's a lot of algebra behind biochemistry — working out how different molecules interact with each other. If you ask yourself how two molecules interact, it's a very simple equation — but combine that with the next equation and the next, and so on, it's quite complicated.

The other area I am interested in is proving things about moving robots — can you prove that these two objects will never collide? Very important in air traffic control (ATC).

I sometimes upset my fellow professors by telling them I'm a happy computer scientist even though I put my life in the hands of my former students several times a year, because some of them write railway signalling software as well as the control software. We often put our lives in the hands of software professionals when we use public transport; between Paddington and Liverpool Street, fr example, the Elizabeth line is totally computer-controlled. The driver doesn't drive the train. To get the correct number of trains through in the time available, you can't rely on human reflexes; you have to rely on computers to handle the brake and so on. There's also a fully computer controlled line on the Paris underground that went live 1991 and had no bugs — most people don't believe that's possible. I've been called a liar by the IT director of a FTSE 100 company for saying that software could run without bugs for 30 years.

It's not easy to write that sort of software, but it can be done, and there are occasions when you need that degree of certainty. You may not want it everywhere - but one could argue that Horizon would not have happened had they taken more care over particular parts of the software. One of my friends who used to work in software has now become an independent consultant and has written a great paper about how you can build truly certified bits of software. And then you can go agile and fast on top of it, as long as your actual safety-critical infrastructure is designed and built first. If you don't tamper with the safety-critical bits, or at least if you know what you're doing when you change them, then you can do it — and we need more of that.

So if you're working on algebra-related computing, is that why you have an interest in the quantum side of things?

There are some questions around quantum computing — and I am fortunate because of my maths background, which included some quantum mechanics courses, so I'm not quite as lost as most people are.

Quantum computers, if they were ever built with sufficient error-correcting, would break a significant part of the cryptography we rely on to assure us, for example, that we are genuinely talking to a bank and not to a fake.

I can build a fake website much more easily than I can build a fake bank; you wouldn't go up to wheelbarrow with the bank logo on it and say it looked vaguely like a cash machine, but in a digital world, I can build something for the price of a wheelbarrow and it looks right because I cloned the digital interface. You only know you're dealing with the real bank website because it's digitally signed by one of the digital signing authorities with a digital certificate that can't be replicated — unless you've got a quantum computer. So what we need to create, and what I’m working on with F-TAG, are cryptographic techniques which quantum computers cannot crack.

These are different to the ones we're used to for the last 40 years, and so the question F-TAG is trying to answer what the industry should do now to be prepared. People do not realise the extent to which devices these days have computers in them; a colleague was recently trying to buy a washing machine that wasn't internet enabled and couldn't.

In particular, in cybersecurity, there's a weakness known as SQL injection, which was invented in 1998 and is still very strong. An example is: there's a series of torque wrenches made by a washing machine manufacturer, which actually are susceptible to SQL injection. They can be tampered with so that they would set things to one tightness but report a different tightness. Now, if you think of that in the hands of a disgruntled employee — chaos. I mean, these are the sort of torque wrenches that are used to tighten bolts on railway trains and so on.

That’s the extent to which we would need to change things in order to cope with the threat of quantum computers. Those with very long memories will remember the year 2000, when a great deal of stuff got upgraded. We're essentially going to have to do a similar exercise, except that in 2000, people didn't make internet-enabled torque wrenches, and these days they do.

We have the theory, but we don't yet have this widely deployed software that, say, an SME can pick up off the shelf, even if a huge corporation like Amazon can.

That scene is changing fairly rapidly — partly thanks to my ex-students and my colleagues on F-TAG and BCS ISSG members and so on — and I'm tracking that, but I can only do that through a wide range of contacts. I can get scientific papers about what the theory is, but that won't tell me where people can buy off-the-shelf software that complements it. That wide network is the sort of thing that BCS can provide me that straight academia can't. In turn, it makes me a more useful member of society.

You've been in F-TAG since it started. Do you feel BCS is now starting to have much more of an impact on the policy side, where these conversations are actually taking place?

Yes, is the short answer. We're reaching a point where we can produce authoritative statements based on polls of our members and that's very important. Most professionals don’t believe that abolishing encryption would help, for example, and that's one where a poll is important.

For quantum-safe computing, what's important is not the poll — it's the voice of a few experts who've got together and said, ‘yeah, this is the way forward’. That is the sort of thing that F-TAG can contribute not just because of its members, but because its members are wise enough that they can recruit other people for particular tasks. They have a nose for authority, as it were, and can say ‘we're positive in this area because we've spoken to 10 leaders in this part of the profession’. Getting groups together like F-TAG means we can know enough — we know who knows about pretty much all parts of it.

But there are challenges. Some people know quite a lot about AI, for example, but there are an awful lot of difficulties with explaining AI in layman’s language. For example, the layman says ‘I want unbiased AI for recruiting’. And we could say ‘Okay, toss a coin’; but really, they do want it to be biased in favour of good candidates. Unbiased AI is useless. We need it to be biased.

But what happens when wanted biases coincide with unwanted biases? For example, we want financial products to be gender neutral; people should get the same pension whether they’re male or female. Great.

For you

Be part of something bigger, join BCS, The Chartered Institute for IT.

However, another consequence of that principle is people getting the same car insurance whether male or female. But unfortunately, it's a sociological fact in the West — certainly in Britain — that young men are significantly more dangerous drivers than young women. If we're going gender neutral, we can't allow for that. So, if you're not careful, you end up with a situation whereby structural civil engineers pay much higher car insurance premiums than dental nurses, because civil engineers have far more car accidents than dental nurses. But that’s not because the job is more dangerous; civil engineers are nearly all men and dental nurses are nearly all women. [So, female structural civil engineers end up paying a needlessly high insurance premium because it isn’t the job that is causing the increased risk. On the other hand, sometimes it is down to the job — for example, social workers pay more because they are more likely to park their cars in high-crime areas.] How can you avoid gender bias while maintaining correct professional bias in these situations?

There are there are many techniques for doing so, which I go into in the chapter I wrote in Adam Leon Smith’s book on AI and software testing. Disentangling wanted from unwanted bias is a genuinely difficult decision, and it’s a team effort. Technical people are responsible for spotting the problem and drawing it to management's attention, who ultimately have to make an ethical decision - but management won't know to make that if the ethical aspects are not drawn to their attention. BCS now has its Foundational Certificate in Ethical AI which is designed to help them spot these ethical challenges.

A good example is that you might have in your criteria for a good employee ‘continuity of employment’. But women tend to take much more maternity leave than men do paternity leave. What do you do about that? If you say your recruitment practices are gender neutral without collecting gender statistics and adjusting for that, then you will find yourself discriminating against women. And sometimes you have to decide that a decision is just too difficult to automate.

Laws in some European countries forbid the collection of, for example, ethnic data, and those laws are based on the idea that if you don't collect it, you're therefore unbiased in that respect. But what it actually means is you can't know whether you're biased or not. In France, for example, you couldn't know if your rejection rate of French Algerian descent were much greater than for native French people.

I commend the great work that BCS has done for many years — in particular both Gillian Arnold and Alistair Revell — in making sure that the society itself is good on this issue and promotes awareness of it, which it's done for many years. We've been doing unconscious bias training on Council for over 10 years.

What are your feelings about being given an honorary fellowship?

If I feel very, very flattered. I put quite a lot into BCS over the years, so I don't feel it's undeserved — and it indicates that academics can have an important role in the whole of BCS, and that we are a distinctive part of BCS. In that sense it's not just for me — it's also for all the other academics who contribute to BCS, contribute to both the profession in their own right, and also to creating the next generation of professionals.

One last question… why are there sheep on your backdrop?

Because I'm a liveryman of the The Worshipful Company of Information Technologists (WCIT), which means I'm a Freeman of the City of London and am entitled to drive sheep across London Bridge.