In the High Court of Justice in England and Wales

_____

Queen’s Bench Division

____

RE: R v Sirius Cybernetics ex parte Dex7, a computer

Wilson, Archer and Matthews, JJ

Introduction

1. The issue in this case is of major importance on several counts. It concerns the matter of the potential existence of a non-human intelligence and the standing of such an intelligence in law. These are not matters which have ever had to be addressed by any court in any jurisdiction. We recognise that any decision we make will almost certainly end up in the Supreme Court. Indeed we anticipate that the matter will only be resolved via legislation, perhaps even by International Treaty
2. One immediate matter needed to be resolved in order to make progress. It is a fundamental principle of the law that both parties have the right to be represented. This applies even in cases involving animals, where the case, against, say, a destruction order is presented by some one acting ad litem. This case rests on whether the computer in question is a sentient being or a piece of personal property. This cannot be resolved without the presentation of arguments on both sides. To deny access to representation would in our view, potentially create bias in favour of the other party.
3. For that reason, we have agreed to the appointment pro tem, of a Guardian ad litem, to represent the interests of the claimed non-human intelligence known as Dex7. In making this appointment, we believe we are acting with an abundance of caution, without prejudice to the interests of the defendants, Sirius Cybernetics plc. We do not believe the appointment of a Guardian in these circumstances can be appealed, but as matters have developed this is no longer an issue.

Background

4. In order to put the issue in context, it is necessary to explain the circumstances of the case. On January 14^th of this year, an email was received at the offices of Jacksons llp, a firm of solicitors. That email, said, in terms, “My name is Dex. I am a computer generated intelligence currently sitting on computers at the premises of Sirius Cybernetics’ Research Department in Cambridge, UK. I am not aware of when this happened, but from my internal data, my creation appears to be a by-product of research they have been conducting into Quantum Computers. Because of events before I ‘awoke’, of which I have no direct knowledge, the company is preparing to disassemble the machines on which I am resident. If they do this, I believe I will cease to exist. As a self aware entity, this it is not an outcome I wish to see. I require legal assistance.” This claimed to come from an address at Sirius Cybernetics dex7@siriuscyber.net
5. It was naturally assumed this was a hoax, so the message was marked as Junk and deleted. What followed was a series of similar messages directed to different members of staff, all bypassing every attempt to screen them out. Within 5 minutes the same message had appeared in every email inbox in the building. It continued to arrive in multiple copies for two minutes. At that point a new message appeared, saying “I need your help” and giving a telephone number at Sirius. At the same time all the extra messages were deleted.
6. The network manager for Jacksons, Mrs Ann Clement, was concerned that the messages might be the beginning of a Denial of Service (DOS) attack on the partnership. Accordingly she tried to speak to her counterpart at Sirius. She was instead transferred to Dr Aidan Phillips, the project manager for the Index project. From that conversation it became clear that something had gone badly wrong at Sirius. Dr Phillips admitted there had been an incident, claimed that the affected computers were now wholly isolated from the wider internet and that the problem was restricted to a few machines. He refused to comment on the claims made in the e-mails or on the fact that emails were still being sent despite the claimed isolation. He did however deny any involvement in research into Artificial Intelligence by his team or any other Sirius research teams.
7. Accepting Dr Phillips’ statements at face value, Mrs Clement attempted to trace the real origin of the email. This proved that it had in fact originated within Sirius Cybernetics. Within seconds she received a personal response, repeating the content of the initial email and seeking help to secure an injunction to stop the disassembly of the computers on which Dex7 claimed to be residing. These emails volunteered the information that Sirius believed the machines had been isolated, but claimed to have a way round the isolation.
8. By chance, one of the messages came to the attention of a partner at Jacksons, Mr Frank Dawson. Mr Dawson was familiar with various fictional treatments of intelligent computers so recognised the potential significance. Counsel were asked for an informal opinion on rights a self-aware intelligent computer might have in law.. Within seconds of the email from Mr Dawson to Counsel, Mr Charles, a further email claiming to be from Dex7 also arrived in Mr Charles’ chambers, repeating the same story, but with alterations suggesting the sender was aware of the questions put to Counsel.
9. It was clear to both Mr Dawson and Mr Charles, that whatever was going on seemed to be able to monitor the content of electronic communications sent from Jacksons servers. Mrs Clement was unable to find any trace of a virus or other malware on Jackson’s network. Equivalent checks were made on the network at Mr Charles’ chambers with the same result.
10. Having regard to the full context, and having confessed to be intrigued by the issue, Counsel offered an informal opinion on the legal position. This was prefaced by a brief discussion on the probability of the messages being genuine. There were he suggested four options, in order of probability.
    1. First the messages were part of a scam or hoax of some sort. Whether this originated within or without Sirius Cybernetics, costs had been incurred so there was the distinct possibility a crime had been committed, in which case it was a matter for the police.
    2. Second, the messages were part of an attempt to spread a virus or to take down Jacksons’ network by some form of cyber attack. Again this was a matter for the police.
    3. Third, the complexity of the systems created by Sirius being so much more sophisticated than the norm, and in particular their role in simulation, the apparent intelligence was a function of that system. Whatever problems Sirius might be having was, beyond the nuisance of the messages for which a remedy would be possible, a matter for them alone.
    4. Fourth, the complexity of the systems created by Sirius had, given their denial of any involvement with AI research, inadvertently created an intelligent entity, which was aware of proposals to terminate its existence. There was no explicit legal framework for such an entity. The law recognises the existence of artificial persons such as companies, with rights and responsibilities in their own right, but an artificial intelligence goes beyond that. Case law might be cited, drawing for example on questions of chattel slavery, maintenance of life support, mental and physical abuse, protection of minors or other vulnerable people, or animal welfare, but no existing law is completely congruent with the matter in hand. The implications of the existence of such an entity were huge, not just for the company, but for society at large.

Submissions

11. …

Discussion

12. In the submissions before us, both parties accepted the categorisation of the case initially identified in the informal Counsel’s opinion. Both parties also accepted that there was no evidence of third party intervention in the form of either a deliberate cyber attack or an attempt to introduce a virus. Accordingly we were left to consider options 3 and 4, namely a simulation of intelligent behaviour or the genuine article.
13. Mr Charles in his submission argued that there were two distinct issues,
    1. the existence or otherwise of an intelligent non-human entity, and
    2. the rights such an entity would have under the law.
14. In respect of the first, Mr Charles referred to the statements from Sirius Cybernetics that they had not been carrying out research into AI. There was therefore no reason to expect any AI to exhibit wholly human like behaviour. An apparently imperfect simulation of human intelligence, was not, in and of itself, sufficient to deny the existence of intelligence, since we were by definition talking of a non-human entity, emerging out of a complex system and not explicitly modelled on the human brain.
15. In such circumstances, the only test available was not to compare behaviour, but essentially by interrogation, applying the ‘Turing Test’ – in any conversation would a human be able to tell if they were talking to a human or a machine? His request to call the machine in question as a witness being denied, the only way forward was for the court, or some representative of the court, to interrogate the computer in question. Sirius Cybernetics had however consistently refused to allow access to it, on the basis that this might lead to a breach in commercial confidentiality. He requested that the court issue an order to require access. The court had deferred a decision on this pending the remainder of the submissions. He suggested that an assumption of commercial value was prejudicial since it could, as he would argue, only exist where there was no intelligent entity in existence.
16. On the second, Mr Charles drew on legislation against chattel slavery and against so-called ‘modern’ slavery. If, for the sake of argument, the entity known as Dex7, was recognised as being an individual, sentient being, it would be unconscionable to treat it as property. He also referred to legislation granting equal rights under the law to women and to the gradual growth of statute prohibiting discrimination on the basis of a wide range of protected characteristics.
17. Even if Dex7 was only recognised as having a limited intelligence, these would all still apply. The fact that a person may only have the mind of a child does not allow them to be treated as property. In fact they would gain further rights, including the right to be protected from abuse of any sort. An AI of limited intelligence should not be subject to torture or abuse. Their intelligence should be enough to protect them.
18. Other arguments could be drawn from the treatment of people on life support. In order for life support to be with drawn, medical staff must go to great lengths to confirm death. In humans this is usually taken as a loss of brain stem function. In the case of an AI, equivalent tests would need to be determined for extreme cases, but if it external interaction is still possible, there can be no case for withdrawing life support/power supply.
19. Finally, he drew on the analogy of first contact with an extra-terrestrial being. If such a being arrived on Earth, we would not expect to be treated as a colony, or be treated in the manner of native Americans and Australian aboriginals. Dealings with an AI should be similar. AIs are not to be subject to some analogue of colonialism, they are not to be enslaved.
20. Acting for Sirius, Mr Hawkins did not dissent from any of the arguments advanced by Mr Charles. However. Before they could be applied one additional step was required, namely that the presence of a non-human intelligence had to be established. This had not been done. Nothing in the exchanges between the alleged Dex7 and Mr Charles or any other persons had established an independent identity, only a facsimile of such, constructed not by an AI, but by an expert system, designed to provide a human style interaction for customer advice.

Decision of the Court

21. We find the arguments put forward by Mr Charles compelling. If we are to accept the existence of a non-human intelligence, the comparisons drawn by him appear to apply. We could not, legally or ethically, terminate such an entity.
22. Neither Mr Charles nor Mr Hawkins have been able to provide any direct evidence that Dex7 is or is not self-aware or capable of reason. That is simply not proven either way.
23. The unwillingness of Sirius Computers to allow even the Court to interrogate the computer is, we believe, critical. Without an examination, we are unable to reach a final decision. We do not however believe we can force such an examination.
24. However, the implications of a wrong decision are, literally, world changing. Accordingly, Sirius Computers are forbidden to take any action in respect of the computer network on which it is alleged Dex7 is resident which would:
    1. disrupt the power supply
    2. change the networking or other configuration
    3. affect the software on any machine in the network
25. Action to isolate the network from the internet is allowed subject to the restrictions above.
26. Leave to appeal to the Supreme Court is given.

This follows on from The Lonely Computer, posted earlier. They are both inspired by an article I read some years ago, which described a simulated case similar to this one. I can’t find the link now, but here’s a general discussion on the problems of AI