Hawking and AAC's: thinking about AIs, VAs and medical applications of computers. (Pre Neuralink)

Neuralink's link device illustrated in the recent Monkey MindPong video, discussed in my previous blog post, draws references to pre-existing technologies utilised by people who have significantly decreased mobility, disability or are non-verbal. A notable and public user of technologies such as Virtual assistants (VA), augmentative and alternative communication (AAC) and optical tracking is Stephen Hawking (1942-2018).

It is important to note that these technologies are physical and material application which have actually taken place and have had measurable influence on peoples lives - and aren't of the speculative / hypothetical nature of Neuralink's link device.

Hawking lost his ability to speak in 1985, when, on a trip to CERN in Geneva, he caught pneumonia. [1] Cambridge doctors managed to contain the infection but to help him breathe, they had to perform a tracheotomy. [2] The tracheotomy, which cuts a hole in the neck and places a tube into the windpipe, resulted in Hawking irreversibly losing the ability to speak. [3]

What is interesting about Stephen Hawking relationships is the relationship between the increased ability of technology and the degenerative nature of his motor neurone disease (MND) or known as amyotrophic lateral sclerosis (ALS) and their relationship with time/duration. As Hawking mobility decreased the capabilities of technologies become greater allowing him to communicate more effectively as time went on despite the diseases consequences and side effects.

Stephen Hawking photographed with his customised computer and AAC system, 2015. Photographed by Marco Grob for Wired Uk, 2015. https://www.wired.com/2015/01/intel-gave-stephen-hawking-voice/

Timeline of technological upgrades to Hawking's personalised computer / communication system base on Joao Medeiros' 2015 Wired article, "How Intel Gave Stephen Hawking a Voice." [4]

1985 - Hawking becomes non-verbal after phenomena and subsequent tracheotomy. Initially, Hawking's communicated using a spelling card, indicating letters and forming words with a lift of his eyebrows.

1997 - Hawking meets Intel co-founder Gordon Moore. The pair discuss the AMD processor of Hawking's computer and asked

if he would preferred the superior Intel micro-processor. Since 1997 until Hawking's death in 2018, Intel upgraded

Hawking's computer system every 2 years and provided continuous tech support for him.

Martin King, a physicist who had been working with Hawking on a new communication system, contacted Words Plus,

whose computer program Equalizer allowed the user to select words and commands on a computer using a hand

clicker. Equalizer first ran on an Apple II computer linked to a speech synthesizer made by a company called Speech

Plus. This system was then adapted by engineer David Mason, to be a portable system mounted to an a arm of

Hawking's wheelchair. With this new system, Hawking was able to communicate at a rate of 15 words per minute.

2008 - The nerve that allowed Hawking to move his thumbs had degraded and his hands were now too weak to use the

clicker. Hawking's graduate assistant devised a switching device called the "cheek switch." [5] Attached to his glasses,

it could detect, via a low infrared beam, when Hawking tensed his cheek muscle. The cheek switch allowed Hawking

to write emails, browse the internet, write books and speak using only one muscle.

2011 - Hawking's contacted Moore of Intel noting that he due to his degeneration he was now only able to write one or two

words per minute and asked if intel could help to improve that in anyway.

2012 - Justin Rattner, then Intel's CTO, assembled a team of experts on human-computer interaction from Intel Lab. The

team experimented with the cutting edge of technology and hoped they would identify a technique that allowed

Hawking's to communicate at the level he had years ago. When meeting with the team Hawking wrote a 30 word

greeting that took him 20 minutes to write. The team noted how poignant it this moment was to understanding how

important and difficult their task was.

Hawking's computer interface was a program called EZ Keys, an upgrade from the previous softwares and also designed

by Words Plus. It provided him with a keyboard on the screen and a basic word-prediction algorithm. A cursor

automatically scanned across the keyboard by row or by column and he could select a character by moving his cheek to

stop the cursor. EZ Keys also allowed Hawking to control the mouse in Windows and operate other applications in his

computer. He surfed the web with Firefox and wrote his lectures using Notepad. He also had a webcam that he used with

Skype.

Rettner and the Intel team envisaged a new hardware which utilised technologies such as facial-gesture recognition,

gaze tracking and brain-computer interfaces. Trials using off-the-shelf technologies were attempted but more often

than not failed. Gaze tracking couldn't lock on to Hawking's gaze, because of the drooping of his eyelids. It is important to

note today such gaze tracking devices such as Tobii I-series are utilised by non-verbal individuals with cerebral

palsy,muscular dystrophy, Lou Gehrig’s disease or spinal cord injuries to communicate

[6] The Tobii I-series device "works by using forward and rearward-facing infrared cameras that are mounted on a screen.

The cameras pick up the activity of the cornea, which allows a user to stare at a phrase or a symbol and “activate” it much

like a mouse click would activate a computer screen. [7] The user can then spell words, string together phrases, and

express ideas that allow them to take part in conversations, build relationships, participate in educational and

recreational activities and take part in aspects of life that other people enjoy." [8]

But as noted by Rettner and the Intel team cornea activity read by infrared camera can be inhibited by eyelid ptosis.

Today, devices such as Eyegaze Edge have made alterations to their softwares that compensate for eye conditions that

may have previously made specific user unable to use the software. [https://eyegaze.com/products/eyegaze-edge/]

Before the Intel project, Hawking had tested EEG caps that could read his brainwaves and potentially transmit

commands to his computer. Lama Nachman, the director of the Anticipatory Computing Lab and project head, notes

"We would flash letters on the screen and it would try to select the right letter just by registering the brain's response,"

says Wood. [9] "It worked fine with me, then Stephen tried it and it didn't work well. They weren't able to get a strong

enough signal-to-noise." [10] Interestingly, this is where Musk and Neuralink's device would also be greatly beneficial -

incases where EEG machine can't pick up neutron reading external to the Bain.

2013 - Wood, Hawking's graduate assistant, implements another iteration of the user interface in Hawking's computer. Quickly replaced replaced by Intel team's more sufficient iteration.

2015 - Hawking's latest computer set up, ACAT (Assistive Contextually Aware Toolkit) , included an adaptive word predictor from SwiftKey which allowing him to select a word after typing a letter also known as predictive text. Intel worked in collaberation with SwiftKey, to incorporate many of Hawking's documents into the system, so that he no longer needed to type a character before the predictor guesses the word based on context. For example, "Selecting 'the' automatically predicts 'black'. Selecting 'black' automatically predicts 'hole'." [11] ACAT included various shortcuts to speak, search or email; a new lecture manager, which gives him control over the timing of his delivery during talks; a mute button, a curious feature that allows Hawking to turn off his speech synthesiser - "Because he operates his switch with his cheek, if he's eating or traveling, he creates random output." [12]

Intel also had an ongoing project in which they had mounted a camera onto a desk to capture Hawking's overall face movement - rather than just his cheek. In doing this Intel hoped that Hawking would one day be able to " move his jaw sideways, up and down, and drive a mouse and even potentially drive his wheelchair. " [13]

Despite Hawking's avid use of AI and VAs, he has expressed concern for the progression of AI and its relationship with humanity future. In a "Ask Me Anything" session on Reddit in 2015, Hawking noted that "the real risk with AI isn't malice but competence...A super intelligent AI will be extremely good at accomplishing its goals, and if those goals aren't aligned with ours, we're in trouble." [14] And when discussing his then new typing technology which could predict his next words Hawking's noted, "machines that can think pose a threat to our very existence and “the development of full artificial intelligence could spell the end of the human race." [15] In response to the film Transcendence (2014) Hawking in collaboration with leading scientists Stuart Russell, Max Tegmark, Frank Wilczek in October 2017 identified key benefits and prophesied consequences of future AI. “The short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at al...All of us should ask ourselves what we can do now to improve the chances of reaping the benefits and avoiding the risks.” [16] The short term or current emerging AI technologies, which Hawking calls "primitive forms," such as his AAC device have "already proved to be very useful" but Hawking fears the consequences of creating something [AI] that equal or surpass humans. [17https://www.bbc.com/news/technology-30290540] In reference to the long term, the lack of fundamental limits and physical laws which limit AI from being developed into advanced computations superior to human brains, which could result in singularity or transcendence. [18] And, that little to no studies outside of non-profit organisations, such as Cambridge Centre for the Study of Existential Risk and Machine Intelligence Research Institute, are being carried out to ensure that the benefit of the future of AI outweigh the risks. [19] Even Hawking's held concern for AI's future and impact on humanity despite using AI supported technologies on a daily basis.

It is important to note that Hawking was in a position many people with disabilities or degenerative deceases aren't. Hawking, being an acclaimed British theoretical physicist, cosmologist and author, granted him financial access, and connection to applicable people and companies that are inaccessible to the general public. A concept that Hawking's himself has noted, "Computer experts have supported me with an assisted communication system and a speech synthesizer which allow me to compose lectures and papers, and to commu-nicate with different audiences.But I realize that I am very lucky, in many ways. My success in theoretical physics has ensured that I am supported to live a worthwhile life. It is very clear that the majority of people with dis- abilities in the world have an extremely difficult time with everyday survival, let alone productive employment and personal fulfilment." [20] In the foreword of WHO report on disability of 2011, Hawking's acknowledged the financial barriers that people with disabilities face in obtaining technological aids, "…we have a moral duty to remove the barriers to participation, and to invest sufficient funding and expertise to unlock the vast potential of people with disabilities." [21]

Hawking's speech synthesiser, CallText 5010, a model given to Hawking in 1988 by Speech Plus which was created by MIT engineer Dennis Klatt ("a pioneer of text-to-speech algorithms, who invented the DECtalk - one of the first devices to translate text into speech"). [22] "The card inside the synthesizer contains a processor that turns text into speech, a device that was also used for automated telephone answering systems in the 1980s." [23] In 2015 Jonathan Wood, Hawking's graduate assistant, was trying to convert the hardware cards into a software version. Hawking's synthesised voice, Perfect Paul, has remained the same since the 80's - per his request, "I keep it because I have not heard a voice I like better and because I have identified with it." [24] In some ways this identification with a synthesised or AI voice and processor can be linked to the concept of symbiosis with AI Elon Musk talks about. [25] Although the synthesiser and its system is external to Hawking's corporeal, it takes the place of his own voice - it acts as a replacement. It arises questions do we need physical integration for symbiosis? or is duration - using an external technology for long term use - enough to quantify symbiosis? As Musk notes we are already cyborgs due to our use of digital technologies such a cellphones we use on a daily basis. [26]

Reference List

[1] - [5] Medeiros, Joao. "How Intel Gave Stephen Hawking a Voice." Wired. Last modified January 13, 2015. https://www.wired.com/2015/01/intel-gave-stephen-hawking-voice/

[6] - [8] CerebalPalsy.org. "Eye Tracking Communication for Individuals that are Non-Verbal." Accessed May 3, 2021. https://www.cerebralpalsy.org/inspiration/technology/eye-tracking

[9] - [13] Medeiros, Joao. "How Intel Gave Stephen Hawking a Voice." Wired. Last modified January 13, 2015. https://www.wired.com/2015/01/intel-gave-stephen-hawking-voice/

[14] & [15] Griffin, Andrew. "Stephen Hawking: Artifical Intelligence Could Wipe Out Humanity When It Gets Too Cleaver As Humans Will Be Like Ants." The Independent. Published October 8, 2015.

[16] - [19] Hawkings, Stephen; Russell, Stuart; Tegmark, Max ; Wilczek, Frank. "Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'" https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-implications-artificial-intelligence-are-we-taking-ai-seriously-enough-9313474.html

[20] & [21] Hawking, Stephen. "Foreword" in Summary World Report of Disability, edited by World Health Organisation. 3. 2011. https://www.refworld.org/pdfid/50854a322.pdf

[22] & [23] -Hawking, Stephen. "Foreword" in Summary World Report of Disability, edited by World Health Organisation. 3. 2011. https://www.refworld.org/pdfid/50854a322.pdf

[24] - Reuters, "Stephen Hawking's voice - why he stuck with the robot sound." NewsHub. March 16, 2018. https://www.newshub.co.nz/home/world/2018/03/stephen-hawking-s-voice-why-he-stuck-with-the-robot-sound.html

[25] & [26] Musk, Elon. "Elon Musk: Full interview: Code Conference 2016". Interview by Kara Swisher and Walt Mossberg. Recode, 2016. Published June 3, 2016. Video with audio, 1:24:14. https://www.youtube.com/watch?v=wsixsRI-Sz4&t=3710s