Genius Makers

Key idea 1

Early research into artificial intelligence was met with skepticism.

On July 7, 1958, a group of men gathers around a colossal computer, roughly the size of a refrigerator, situated deep within the confines of the United States Weather Bureau in Washington, DC. Their attention is fixed on Frank Rosenblatt, a young professor from Cornell University, who proceeds to present a series of cards to the computer.

Each of these cards features a black square printed on one side, and the task at hand is for the machine to distinguish between those with the mark on their left side and those with it on the right. Initially, the computer struggles to make this distinction. However, as Rosenblatt continues to display the flashcards, the accuracy of the machine’s identifications gradually improves. After approximately 50 attempts, it becomes highly proficient in recognizing the orientation of the cards.

Rosenblatt dubs this machine the Perceptron. While it may seem rudimentary by modern standards, it actually represents an early precursor to what we now refer to as artificial intelligence, or AI. Interestingly, during its time, the Perceptron was largely dismissed as a mere novelty.

The key message here is: Early research into artificial intelligence was met with skepticism.

Today, we acknowledge Frank Rosenblatt’s Perceptron and its successor, the Mark I, as early iterations of neural networks. Neural networks, which are fundamental to what we now refer to as artificial intelligence, utilize a process often described as machine learning. Essentially, these networks operate by scrutinizing extensive datasets and discerning patterns within them. As the network identifies more patterns, it fine-tunes its analytical algorithms to generate increasingly precise information.

Back in 1960, this process was characterized by slowness and a great deal of trial and error. To train the Mark I, researchers provided the computer with pieces of paper, each bearing a letter such as A, B, or C. Using a sequence of calculations, the computer would attempt to guess the letter it observed. Subsequently, a human evaluator would determine whether the guess was correct or not.

The Mark I would then adjust its calculations to enhance the accuracy of its future guesses.

Scientists like Rosenblatt drew parallels between this process and the functioning of the human brain, suggesting that each calculation resembled a neuron. They contended that by interconnecting numerous calculations that adapt and evolve over time, a computer could learn in a manner similar to humans. Rosenblatt termed this concept “connectionism.” However, there were critics, including MIT computer scientist Marvin Minsky. In his 1969 book, Minsky criticized the notion of connectionism and argued that machine learning could not effectively address more intricate problems.

Minsky’s book exerted significant influence. Throughout the 1970s and early 1980s, interest in neural network research dwindled, leading to what became known as an “AI winter.” During this period, few institutions allocated funds for neural network research, and progress in machine learning decelerated. Nevertheless, it did not come to a complete halt, as a handful of scientists continued to explore connectionism, as the next key idea will delve into.

Key idea 2

Deep learning made neural networks tech’s new favorite toy.

Geoff Hinton was somewhat of an outsider right from the beginning. In the early 1970s, he completed his PhD at the University of Edinburgh. This period coincided with the AI winter when enthusiasm for artificial intelligence was waning. Despite this, Hinton remained committed to a connectionist approach to AI. Understandably, he encountered difficulty in securing employment after earning his degree.

For the subsequent decade or so, Hinton moved through various academic positions. He held roles at institutions such as the University of California San Diego, Carnegie Mellon University, and eventually the University of Toronto. Throughout this time, he persistently honed his theories of machine learning. He held the belief that introducing additional layers of computation, a concept he termed “deep learning,” could unlock the potential of neural networks.

Over the years, he managed to persuade a few skeptics and achieved some progress, albeit at a gradual pace. Then, in 2008, he crossed paths with Microsoft computer scientist Li Deng.

The key message here is: Deep learning made neural networks tech’s new favorite toy.

Li Deng and Geoff Hinton initially struck up a conversation at NIPS, an AI conference held in Whistler, British Columbia. At that time, Deng was deeply involved in developing speech recognition software for Microsoft. Hinton, recognizing an exciting possibility, proposed that deep learning neural networks had the potential to surpass conventional approaches. Deng, although initially skeptical, found the idea intriguing, and they decided to collaborate.

During much of 2009, Deng and Hinton worked together at Microsoft’s research lab in Redmond, Washington. Together, they designed a program that utilized machine learning models to analyze extensive collections of recorded speech. The program ran on specialized GPU processing chips, typically employed in computer games. After several weeks of intensive processing, the results were astonishing. The program demonstrated exceptional accuracy in analyzing audio files and identifying individual words.

Subsequently, other technology companies began exploring similar programs. Google scientist Navdeep Jaitly, for instance, employed his own deep learning machine, achieving even more impressive outcomes with an error rate of only 18 percent. These early achievements presented a compelling case for the substantial potential of neural networks. Additionally, researchers recognized that the fundamental principles could be applied to address a wide array of problems, ranging from image search to the guidance of autonomous vehicles.

Within a few years, deep learning emerged as the most sought-after technology in Silicon Valley. Google, known for its ambition, led the way by acquiring Hinton’s research firm, DNNresearch, along with other AI startups like DeepMind based in London. However, this was just the outset, as the ensuing years would witness even fiercer competition in the field.

Key idea 3

Silicon Valley’s biggest companies poured money into AI research.

In November 2013, Clément Farabet experienced an uneventful evening at home when his phone suddenly rang. Upon answering, he anticipated a call from a friend or colleague, but to his surprise, the voice on the other end belonged to Mark Zuckerberg.

While the call caught him off guard, it wasn’t entirely unexpected. Farabet was a researcher at NYU’s deep learning lab, and for several weeks, Facebook employees had been attempting to persuade him to join their ranks. Initially hesitant, Farabet’s interest was piqued when the CEO himself extended a personal invitation.

Farabet was not alone in receiving such offers. Many of his peers were presented with similar opportunities. Silicon Valley’s tech giants were engaged in a fierce competition to recruit the best talent, each vying to establish dominance in the emerging field of AI.

The key message here is: Silicon Valley’s biggest companies poured money into AI research.

During the early 2010s, deep learning and neural networks represented relatively new technologies. Nevertheless, ambitious tech entrepreneurs at companies like Facebook, Apple, and Google were firmly convinced that artificial intelligence held the key to the future. Although the exact profit potential of AI remained uncertain, each company was eager to be at the forefront of this emerging field. Google gained an early advantage by acquiring DeepMind, but Facebook and Microsoft were quick to follow suit, investing millions in recruiting AI researchers.

But what did a social media company like Facebook see in AI? Essentially, a state-of-the-art neural network could enhance the business by extracting valuable insights from the vast amounts of data stored on its servers. It could learn to recognize faces, translate languages, predict consumer behavior for targeted advertising, and potentially even operate sophisticated chatbots for tasks such as messaging friends or placing orders. In summary, AI had the potential to bring the platform to life.

Google also had ambitious plans for its AI research. Specialists like Anelia Angelova and Alex Krizhevsky envisioned using Google Street View data to train self-driving cars for efficient navigation in real-world urban environments. Another researcher, Demis Hassabis, was developing a neural network aimed at improving the energy efficiency of the millions of servers essential for the company’s operations.

In the media, these projects were often portrayed as groundbreaking and potentially transformative. However, not everyone shared the same level of optimism. Oxford University philosopher Nick Bostrom issued warnings that advances in AI carried potential risks, cautioning that superintelligent machines might behave unpredictably and make decisions jeopardizing humanity’s well-being. Despite these warnings, the surge of investment in AI continued unabated.

Key idea 4

Neural networks have the potential to outdo humans in many fields.

Have you ever engaged in a game of Go? At first glance, it appears to be a straightforward game where two players take turns positioning black and white stones on a grid, each attempting to encircle the other’s stones. However, in reality, Go is an exceedingly intricate game. It presents an immense array of potential strategies and is so inherently unpredictable that, until 2015, no computer could outperform the finest human player.

In October of that year, Google’s AI program, AlphaGo, challenged Fan Hui, a highly ranked player. AlphaGo, a neural network system that had been trained by analyzing millions of past games, proved to be an unstoppable force. It emerged victorious in five consecutive matches. A few months later, it accomplished an even more remarkable feat by defeating Lee Sedol, the reigning human champion.

This unquestionably marked a significant turning point for AI. As scientists delved deeper into the study of neural networks, these networks continued to gain in power and capability.

The key message here is: Neural networks have the potential to outdo humans in many fields.

In the decades following Rosenblatt’s initial experiments with the Perceptron, neural networks experienced remarkable growth and advancements. This significant progress was driven by two key factors. First, computer processors continued to become faster and more cost-effective, enabling modern chips to perform significantly more calculations than their earlier counterparts. Second, the availability of data had surged, providing neural networks with a vast array of information for training.

These developments allowed researchers to apply machine learning principles in creative ways to address a wide range of challenges. Consider, for instance, the problem of diagnosing diabetic retinopathy, a common condition that can lead to blindness if left untreated. Early detection typically requires a skilled doctor to meticulously examine a patient’s eye for subtle signs such as lesions, hemorrhages, and slight discolorations. However, in regions with a shortage of medical professionals, like India, conducting such examinations for everyone can be logistically challenging.

This is where Google engineer Varun Gulshan and physician Lily Peng came in. Collaborating on a solution, they devised an efficient method for diagnosing diabetic retinopathy. Using a dataset of 130,000 digital eye scans from India’s Aravind Eye Hospital, they trained a neural network to identify the subtle indicators of the disease. After processing the data, their program could automatically analyze a patient’s eyes within seconds, achieving an impressive accuracy rate of 90%, comparable to that of a trained doctor.

Similar projects hold the potential to revolutionize the healthcare landscape. Neural networks can be trained to analyze various medical data types, such as X-rays, CAT scans, and MRIs, to efficiently detect diseases and anomalies. Over time, they may even learn to identify patterns that are too nuanced for human observers to discern.

Key idea 5

More sophisticated AI has the potential to distort our view of reality.

Picture yourself casually browsing your Twitter timeline. The usual content scrolls by – a brief jest, a precisely targeted ad, a lively debate about popular culture. Then, amidst the typical fare, something extraordinary grabs your attention: a video featuring Donald Trump. However, this isn’t the familiar Donald Trump you’re accustomed to.

In this video, Donald Trump is eloquently conversing in Mandarin Chinese. What’s more, it’s not a poorly executed voiceover – his lip movements flawlessly synchronize with the spoken syllables, and his gestures harmonize seamlessly with the cadence of his speech. Yet, upon closer examination, you begin to notice slight hiccups and incongruities. The video is a fabrication, albeit an exceptionally convincing one.

On this occasion, you managed to discern the deception. However, with the ongoing advancements in AI technology, your ability to distinguish such fabrications may become less assured in the future.

The key message here is: More sophisticated AI has the potential to distort our view of reality

In the early 2010s, the primary focus of machine learning research revolved around instructing computers to recognize patterns within datasets. AI programs, trained on extensive collections of images, demonstrated proficiency in identifying and categorizing pictures based on their content. However, a pivotal shift occurred in 2014 when Ian Goodfellow, a researcher at Google, introduced a novel concept. He proposed whether AI could be taught not only to recognize patterns but also to generate entirely new images.

To achieve this goal, Goodfellow devised the inaugural generative adversarial network, commonly referred to as GAN. This innovative approach involved the collaboration of two neural networks. The first network was responsible for producing images, while the second network employed intricate algorithms to evaluate their authenticity. Through iterative exchanges of information between the two networks, the newly generated images progressively acquired greater realism.

Although fabricated images had existed previously, GANs significantly streamlined the process of generating lifelike depictions of various individuals and objects. Consequently, early adopters swiftly harnessed this technology to create convincing videos featuring politicians, celebrities, and other prominent figures. While some of these creations, known as “deep fakes,” were harmless and amusing, others raised significant concerns, particularly those involving individuals placed into explicit or pornographic contexts.

However, the challenges within AI research extended beyond the realm of deep fakes. Critics also pointed out issues related to racial and gender bias within the field. In 2018, computer scientist Joy Buolamwini unveiled research demonstrating that facial recognition programs developed by tech giants like Google and Facebook exhibited shortcomings when tasked with identifying non-white and non-male faces. These programs had been trained on datasets that predominantly featured white males, leading to distortions in their accuracy. Such revelations have prompted valid concerns about AI’s potential to perpetuate biases and oppressive practices, a subject we’ll delve into further in the next key idea.

Key idea 6

AI can easily be misused for political ends.

In the autumn of 2017, a group of eight engineers employed at Clarifai, a startup specializing in AI research and development based in New York City, received an unusual assignment. Their task was to construct a neural network with the capability to recognize individuals, vehicles, and structures, with a specific emphasis on its performance in desert environments.

Initially, the engineers embarked on this project with some hesitation, but disconcerting reports soon began to circulate. It became apparent that their work was not intended for an ordinary client; instead, they were unknowingly contributing to a project for the United States Department of Defense. Their neural network was destined to facilitate the navigation of drones, aligning them with a government-backed military initiative.

In light of this revelation, the engineers chose to resign from the project. However, this episode marked just the initial manifestation of AI’s growing entanglement with the realm of politics and military applications.

The key message here is: AI can easily be misused for political ends.

Private corporations weren’t the sole entities eager to harness the expanding capabilities of AI. Governments, too, recognized the transformative potential of machine learning technologies. In China, the State Council initiated an ambitious plan funded by the government to secure a leading position in AI by the year 2030. Similarly, the United States government escalated its investments in AI systems, with a significant focus on military applications.

In 2017, discussions unfolded between the Department of Defense and Google regarding a substantial multi-million dollar partnership known as Project Maven. This collaboration aimed to leverage Google’s AI expertise to enhance the efficiency of the Pentagon’s drone program. Understandably, the prospect of assisting the military raised concerns among many engineers. Over 3,000 employees signed a petition urging the cancellation of the contract. While Google eventually acquiesced, the company’s executive board did not definitively rule out potential future partnerships with the military.

Meanwhile, Facebook found itself embroiled in a political controversy of its own. During the 2016 election cycle, a British startup named Cambridge Analytica illicitly collected private data from 50 million Facebook profiles, subsequently employing this information to craft deceptive campaign advertisements in support of Donald Trump. This scandal underscored the platform’s difficulties in content moderation. Given its vast user base, Facebook had become a breeding ground for diverse content, including extremist propaganda and misleading information commonly referred to as “fake news.”

In 2019, Mark Zuckerberg testified before Congress, pledging that his company would employ AI to combat harmful content. However, this solution has its limitations. Even the most advanced neural networks struggle to discern the subtleties of political discourse. Furthermore, malicious AI can generate new disinformation as quickly as it can be moderated. In essence, those employing AI for benevolent purposes perpetually find themselves engaged in a race against nefarious actors.

Key idea 7

Neural networks still don’t think or learn like humans.

On a sunny spring day in May 2018, the Shoreline Amphitheatre in Mountain View, California is brimming with attendees. The atmosphere is electric. However, the figure commanding the stage isn’t a guitar-wielding rock star; it’s Sundar Pichai, the CEO of Google, holding a smartphone.

This spectacle unfolds at I/O, Google’s annual conference, where Pichai showcases the company’s latest creation, the Google Assistant. Powered by neural net technology known as WaveNet, this assistant can place phone calls using a remarkably lifelike human voice. In the demonstration Pichai just shared, the AI successfully reserves a table at a restaurant, leaving the cafe attendant oblivious to the fact that they were conversing with a computer.

While this advancement leaves the audience in awe, not everyone is equally impressed.

The key message here is: Neural networks still don’t think or learn like humans.

Upon witnessing Pichai’s demonstration, Gary Marcus, a psychology professor at New York University, couldn’t help but express his skepticism. He shares a perspective akin to Marvin Minsky’s, harboring doubts about the true potential of machine learning. While Google’s AI assistant was showcased as possessing near-human comprehension of speech and dialogue, Marcus believed that the program’s apparent impressiveness stemmed from its performance of a highly specific and predictable task.

Marcus’s viewpoint is rooted in nativism, a school of thought positing that a substantial portion of human intelligence is ingrained in our brains through evolution. This crucially distinguishes human learning from neural net deep learning. For instance, an infant’s brain exhibits such remarkable adaptability that it can learn to recognize an animal after being exposed to only one or two examples. In contrast, a neural net must be trained on millions of images to accomplish the same task.

From a nativist standpoint, this disparity in innate capabilities clarifies why neural net AI hasn’t advanced as rapidly as anticipated, especially concerning nuanced endeavors like comprehending language. While Google Assistant’s AI can navigate basic scripted conversations, it struggles to engage in the intricate dialogues that come naturally to an average person. While an AI may be proficient at making reservations, it falls short in understanding humor or subtle nuances in language.

Nonetheless, researchers are diligently working to surmount this challenge. Teams at Google and OpenAI are presently experimenting with an approach referred to as universal language modeling. These systems train neural networks to comprehend language in a more nuanced, context-specific manner, and while they have exhibited progress, their potential as engaging conversational partners remains uncertain.

Key idea 8

Researchers continue to push AI beyond its current limits.

Google stands as one of the world’s most prosperous companies, holding the distinction of conceiving and managing numerous services that underpin the functioning of the contemporary world. Furthermore, it generates an immense wealth, tallying up to tens of billions of dollars annually. Now, consider the prospect of having not just one, but two, three, or even fifty Googles. Could artificial intelligence bring about such a scenario?

It’s a plausible notion, at least as per Ilya Sutskever, the chief scientist at OpenAI. If scientists could successfully construct artificial intelligence systems possessing the same level of capability and creativity as the human mind, it could lead to a groundbreaking transformation. A super-intelligent computer could create an even more advanced version of itself, and this process could continue indefinitely. Eventually, artificial intelligence might propel humanity to realms beyond our current comprehension.

This is undoubtedly an ambitious aspiration, but its feasibility remains uncertain. Even the brightest minds in Silicon Valley are not entirely certain about its realistic prospects.

The key message here is: Researchers continue to push AI beyond its current limits.

When Frank Rosenblatt initially introduced the Perceptron, it garnered both skeptics and optimists. Across the globe, scientists and futurists made bold predictions about computers eventually reaching or even surpassing human technical and intellectual capabilities. Herbert Simon, a Carnegie Mellon professor, even wagered that this feat would be accomplished within a mere two decades. Clearly, these prognostications did not fully materialize.

Despite the varying pace of progress, there remains a persistent belief that human-like or even superhuman intelligence, often referred to as artificial general intelligence (AGI), is attainable. In fact, some are placing significant bets on its realization. In 2018, OpenAI revised its charter to explicitly include the development of AGI as a primary goal for the company’s research. Following this announcement, Microsoft pledged to invest over one billion dollars to support the research team’s ambitious objective.

The precise path to achieving AGI remains uncertain, but researchers are exploring various approaches. Some companies, including Google, Nvidia, and Intel, are actively working on the creation of new processing chips tailored specifically for neural networks. The aim is to empower these networks with the processing capabilities necessary to overcome the current limitations faced by machine learning programs.

On a different trajectory, Geoff Hinton, an early advocate of machine learning, is pursuing a unique route. His current research is centered around capsule networks, a technology believed to closely mimic the structure and functionality of the human brain. Nonetheless, it will likely be several years before any tangible results emerge, and during that time, entirely new theories may come to the forefront. In the realm of AI, the future is perpetually marked by uncertainty.

Final summary

The key message in this book:

Recent developments in artificial intelligence (AI) have stirred significant excitement, concern, and debate. A substantial portion of contemporary AI relies on neural network models, a methodology involving intricate mathematical computations to scrutinize vast datasets and detect patterns. Governments and private enterprises have harnessed this technology for diverse applications, encompassing image retrieval enhancement, precision in online advertising, disease diagnosis, and the operation of unmanned aerial vehicles. The ultimate trajectory of AI research remains uncertain, but certain individuals are wagering that it will persist in yielding groundbreaking innovations.