This lecture focuses on the question, what is Deep Learning and what is Machine Learning? First, we'll start with the definition of artificial intelligence. Artificial intelligence is a technology that enables a machine to make an intelligent decision, or action. AI technology enables an intelligent agent to cognitively perceive its environment and correspondingly attempt to maximize its probability of success of a target action. An intelligent agent would correspond to a hardware module, software, a robot, or an application. Machine learning is the capability enabled to a computer to learn without being explicitly programmed. It is a functionality to learn and make predictions from data. It evolved from pattern recognition and computational learning theory in AI. Then we focus on deep learning. Deep learning is a machine learning technique that uses multiple internal layers of nonlinear processing units to conduct supervised, or unsupervised learning from data. Now, the multiple internal layers are the hidden layers. And the nonlinear processing units are commonly the neurons that we will be using. This is because our deep learning technology is commonly implemented using a neural network. And, that's where the concept of the neurons and also perceptrons will come from. Now, the relation between AI and ML and DL. First artificial intelligence. Automation became possible due to AI technology. Machines become capable of automated learning and making decisions due to machine learning technology. And precision details are cognitively noticed in the automated learning process and used in accurate decision making of complex problems due to deep learning technology. Now we'll focus on some examples of Human vs. Artificial Intelligence. We'll start off with this one. In 1996 and 1997 IBM's Deep Blue beats world's top ranked chess players. The estimated processing capability was 11.4 giga GFLOPS. And due to this following, the computers became so powerful at chess and they won most of the matches. Afterwards the world rankings were divided where there is a ranking for computers which is the computer chess ratings list and for grandmasters, which are humans the World Chess Federation ratings and these were divided afterwards. Examples of Human versus Artificial Intelligence. From 2015 to 2017 Google's DeepMind AlphaGo beat top ranked Go players. And in this system one of the recent ones that were used at the end in 2017 was the TPU second generation. TPU stands for Tensor Processing Unit. The overall combined processing capability of this system is 11.5 petaflops. That is 11.5 x 10 to the power of 15 FLOPS. That's a truly tremendous amount of computation capability. A CPU is a central processing unit. It executes instructions of a PC, smartphone, or various other electronic devices. It is designed to support all process types. Compared to that a GPU is a graphics processing unit. And it is a custom made CPU that is operations specified for high speed and low power operations. It's embedded in PCs, smartphones, video cards, motherboards and other CPU's and system on chips. Computer Performance Units is next what we'll look into. And the first one is FLOPS, floating-point operations per second. This is the floating-point computation based performance measure and the commonly used unit is Gflops which is giga billions of flops. Another performance unit is Instructions per Second, IPS. And this is the integer number of operations based performance measure. The commonly used unit is in millions of IPS. So therefore it's MIPS for short. Next we'll make some comparisons and we'll start with a human. And, the human being is capable of so many complex and diverse things. It's impossible to characterize it in pure computation numbers. However, to simplify the overall capability of a human being into some simple numbers, that's what I'll try to do here, and then compare those numbers with other systems. Advanced systems. First, when it comes to FLOPS. A human is approximately capable of 0.01 FLOPS and that means that it is 1 over 100 FLOPS. Another way to characterize a human being is the characteristic that a human being normally has around about 2.5 petabytes of memory and also it's running on 20 watts of power. Now, I know this is totally different for different people and the characteristics will diversely change these numbers but just for fun. These are some numbers that have been used to characterize human beings. Now what does this mean? 0.01 FLOPS? Well, 0.01 FLOPS means that it will take 100 seconds for one floating-point calculation for a human being. And then you're thinking, really? That's a little bit too slow. Well, think of it this way. Now if you were to add these two numbers these are simplified floating-point numbers. For example, 1.2345 + 0.6789. And in your mind, without using a pen or a pencil, can you compute this in your head? How many seconds would it take? And then, reconfirm it to make sure you know it. Then the answer is 1.9134. Did you get that number? Did you get it quickly enough? Approximately, you may have been ready and really good at it, but some people are not. And an average of, what we're talking about 100 seconds, for this simple floating point calculation maybe what would be required. And that's just a number. I know you can do it faster. But anyway, this is just an average number, considering the diverse range of human beings and their computational capabilities, just purely in their mind. In addition, in a human's brain there are 100 billion neurons. There are 10,000 connections per neuron and combining these two numbers up; there are 1 quadrillion synaptic connections in the human brain. And of course, like I said before, the human brain is not just for simple computations of floating-point operations. It's so much more diverse. But then once again those were just approximated numbers for fun. Now, comparing FLOPS for human, modern smartphones and PCs, and Google's DeepMind AlphaGo Master, which uses the TPU second generation one. Is humans are at the level of 0.01 FLOPS. Then, for modern smartphones or personal computers, workstations they go to the level of 10 to about 300 gigaflops. That is 10 to the power of 9. Then for Google's DeepMind AlphaGo Master system it's at for the TPU second generation system. It's at 11.5 petaflops. That's 10 to the power of 15. How is intelligence extracted from data? Among the possible ways we will focus on using Machine Learning and Deep Learning. What is this intelligence used for? Well, it's used for natural language processing, computer vision, speech recognition, robotics motion and manipulation, as well as computational creativity. These are some AI technology types that exist. And of course it includes artificial neural networks. In short we'll just call them neural networks. Also evolutionary algorithms, genetic programming, swarm intelligence and there's so much more. AI tools used to make optimal decisions or faster suboptimal decisions include optimization theory, game theory, fuzzy logic simulated annealing, Monte Carlo experiments and simulation, complex theory and there's so much more. We will study on how intelligence is obtained from data using a neuron, perceptron, neural network, a convolutional neural network, and a recurrent neural network. These are the references that are used and I recommend them to you. Thank you.
