Signs

Aye, Aye…AI, What are Thee?

Brace your senses, hold your horses and jump on couches…we are doing a crash course on Artificial Intelligence, Symbolic AI, Machine Learning, Natural Language Programming, Computer Vision and Quantum Computing.

AI and Machine Learning Crash Course

  • Features: Values on which object is to be classified on. All the audio and visual characteristics of human intelligence is reduced to features.
    • Training set: data to train algorithm.
  • Machine Learning Algorithms: Find optimal separations when two data sets overlap. Values of features that separate one data set from another is called Decision Boundary.
  • Confusion Matrix: Table showing whether the classified data has been put in right or wrong categories.
    • Confusion Matrix is visually represented using decision tree that use If function to pull values from the data set. Too many decision trees are called forests.
  • Non tree-based decision making method are called Support Vector Machine which uses arbitrary line to splice up the decision space.
    • They don’t have to straight line but can be polynomial or some other fancy mathematical function.
    • Machine Learning Algorithm provides the best line to get the most accurate classification result. If there are more than 2 features, the Cartesian plane becomes a 3D Space. In 3D space, decision lines become space planes. They don’t have to be straight planes.
  • Artificial Neural Networks: Take multiple inputs from the input layer, combine the signal and give out numbers to the output layer. (applying weighing -> summing -> biasing ->activation function)
    • Instead of taking in chemical or electrical signals, they take numbers as signal.
    • Both input and output is in form of numbers.  Hidden layer converts input into output and does the major work just like a neuron.
    • The first thing the hidden layer neuron does is to multiply each of these signal with a weigh or factor
    • Then it sums the weighed input together and applies a bias to the result. In other words it adds, it adds or subtract a fixed value.
    • These biases and input weighs are given random values until an algorithm goes in and starts tweaking all of these random values to train the neural network.
    • The data is then divided into training set and testing set. By running multiple iteration, the algorithm gradually improves accuracy.
    • The last step is to apply Activation function which is a mathematic modification to the output result based on fine tuning of algorithm.
    • Hidden layer doesn’t has to be one layer. It can be many layers deep. This where term deep layer comes from
  • Weak AI/Narrow AI: can only make one form of decision. It has task specific intelligence
  • Strong AI: truly general purpose AI, as smart as a human
  • Strong AI can not only source massive information from internet, it can train itself by recreating many of its clones. Eg: Alpha Go vs Lee Sedol where AI game beat the most competitive player.
  • Reinforcement learning: when AI system can learn from its own clones. So learning is reinforced through trial and error.

Symbolic AI

  • Symbolic AI requires no training. It uses symbols to recognize problems and logic to search for solution.
  • Symbols are codification for anything: people, food, numbers, objects, plants, etc.
  • Relations are adjectives to describe symbols. They are written outside parentheses such Chocolate (Donut).
    •  Chocolate is relation, donut is symbol. if a man is eating donut, you will write Eat (Donut,  Man) because Eat is describing what describing both Man and Donut.
    • Symbols are nouns. Relations are adjectives or verb that describe the noun together.
    • Knowledge base is collection of everything in our universe. AI searches these knowledges bases based on our query.
    • ‘And’ ‘Or’ are the connectives in the logic and using them with symbols and relations we make propositions => statements.
  • Using a set of Rules we can figure out whether these statements are true or not. This is called Proposition Logic or Boolean Logic. Truth table assess whether all these statements joined by a Connectives are truth or not
  • To make these decisions, AI needs to use some Math. So we turn decisions into mathematical equations. False=0, True= anything not zero. And mean multiply. Or means Add.
  • Other logical connectives include Not.
    • NOT switches True things to False and False things to True
  • Implications: If one proposition is true, other proposition must also be true. Basically they are IF/THEN statements. Symbolic AI makes thousands of IF/Then Statements.
    • Eg: IFCold(Me)ThenWear(Jacket); this is saying if I am cold, I must be wearing a jacket.
    • A statement is always true If the side is Then side is true or the if side is False.
  • Inference is when AI uses logic to check new propositions based on existing propositions
  • Expert System: make conclusion based on knowledge and reason not just trial and error or guess work like neural network system
  • Expert System can make decision by showing which parts were evaluated as True or False.

Computer Vision: Crash Course

  • Images on computer are stored as a big grid pixels. Each pixel is comprised of a of 4×4 matrix. Each pixel is defined as a color. (1.16) Each pixel has combination of 3 primary colors: Red, Green, Blue. By combining different intensities of primary color, any color can be made. Combination of these different intensities are called RBG values.
  • Basic vision algorithm track a colored object like a black paint ball. For that we will find RBG value of the ball and ask the algorithm to pick the pixel with closest color match.
  • The algorithm searches for color match in all the pixels in the photo and in all the frame in a video.
  • This technique works for tightly controlled environment were no two objects have similar color. But this technique does not work for features larger than single pixel such as edges of object which have so many pixels.
  • Therefore the algorithm looks for small regions of pixel called patches.
  • To help AI decide between edges of object, it creates a rule. That is the higher the difference in intensities of two pixel, most likely means that pixel is on the edge of object.
  • Mathematical notation of this rule is called a kernel/filter.
  • A Kernel takes pixel wise multiplication, sum of which is saved into center pixel.
  • Applying kernel to a patch of pixels is called Convolution.
  • Kernels that recognize horizontal edges and vertical edges are called Prewitt Operators.
  • Kernels can also be used to sharpen or blur an image. They can also be used as a cookie cutter to find anything that we like in an image. Can also be used for finding lines with edges on both side and even island of pixels surrounded by contrasting colors.
  • Kernels can also be used for facial recognition. Tip of the nose is brighter than sides of the nose (color difference kernel). Center of eyes are dark surrounded by lighter colors.
  • Each kernel recognizes a certain feature of human face that is eyes, nose, ears but combined these kernels can recognize the entire face. This is basis of Viola Jones Algorithm
  • Convolution Neural Network: a neuron works like a convolution just that it develops its own kernals
    • Convolution Neural Network forms bands of neurons, each process a certain data and gives a resulting image.
    • These outputs are than processed by subsequent layer of many neuron, allowing convolution after convolution.
    • So one layer of neuron may recognize edges, other shapes, other objects such as mouth or eye brow until a layer of convolution puts it together and the final layer of neuron recognizes it as a face.
  • Convolution neural network go several layers deep and it is called Deep Learning
  • Convolution neural network can be used to recognize hand writing, traffic flow, spotting tumors in CT scans and facial expression.
  • Facial Recognition. More specialized algorithms can be used to pinpoint facial landmarks such as tip of the nose and corners of the mouth.
    • Algorithm calculates distance between landmarks to read expression. Relative position of eyebrows to eyes can mark surprise or delight, smiles is difference in shape of mouth.
  • Context Sensitive. Reading facial features allow computers to intelligently adapt their behavior and become context sensitive.
  • Biometric Data. Algorithm also monitor geometry of face i.e distance between eyes and height of forehead. CCTV cameras and unlocking smartphone use these algorithms
  • Landmarking also takes place hands and whole body giving algorithm ability to read body language.

Natural Language Programming

  • Natural Language: using language as humans do with different words, accents, interesting wordplay. A filed combining Computer Science and Linguistic.
  • Parse tree: using parts of speech and associated rules to show how the sentence is constructed.
  • Knowledge Graph. Structures that contains relationships between words, adjectives and the command being asked.
  • Chat bot. Work through parsing and generating text. Chatboxes are rule based mapping what user many say and how a program may respond.
    • Ex: ELIZA. A chatbot that took the role of Therapist. Created by MIT in 1960. This was unwieldy to maintain and limited the sophistication. ELIZA used basic syntactic rules to identify contexts and ask humans about it.
  • Today real human communication is used to train chat bots. People train chatbots and in a Facebook experiment, chatbots have evolved to use their own language.
  • Speech Recognition. Using sound and converting it in text.
  • Audrey: The Automatic digit recognizer. It recognized all digits if they were said slowly enough.
  • HARPY: First system to recognize over a 1000 words developed by Carnegie Mellon University.
  • Evolution of Algorithm and advances in computing made is possible for algorithm to analyze thousands of speech data sets using machine learning. Software these day use neural networks to process speech.
  • Acoustic Signals. Using vowel sounds to turn waveforms into that identify letters
  • Spectrogram for Speech Recognition. Different frequencies that make up each sound on y axis vs time on x-axis. The brighter the color the louder is that frequency.
  • Fast Fourier Transform. An Algorithm that converts waveforms to frequencies. The algorithms looks for patterns of frequencies in speech and checks resonances of vocal tracts
  • Formants. Places where we see peak in the spectrum. These are labelled and two sounds have different formant(different peaks). This allows algorithms to recognize spoken vowels and whole words.
  • Phonemes. Sound pieces that make up word. There are 44 phonemes in English and pattern matching takes place to recognize sound for each word.
  • Language Model. Because people pronounce words with different accents and mispronunciation, transcription accuracy is greatly improved with Language Model which contains statistics like most commonly used word in a sentence and replacing it with less commonly used similar sounding word.
  • Speech Synthesis: giving the computer ability to output speech. Text is broken down into its phonetic components and computer says it in monotones
  • Improvements in monotones came about advancement in computing and using voice interfaces in Google, Siri, Cortana which makes positive feedback loop to train the algorithm.

Deep Learning Crash Course

  • Role of Neuron on Deep Learning Neural Network. Neuron hold number between zero and one.
  • The network of neuron consists of many neuron each holding the grayscale value of the pixel that make the image. In a 28×28 image, there are 784 pixel(28*28) and therefore 784 neurons. hence the number of neurons is equal to the number of pixels in image. Value in neuron is the number of gray scale color carried by pixel. These values correspond to 0 for black pixel and 1 for white pixels
  • Activation of Neuron. The number inside each neuron, the gray scale value is called activation.
  • The last layer is the output layer which has 10 neurons each for a number from 0,1,2,3,4,5,6,7,8,9. The value inside each neuron is the probability that the grayscale value of each the pixel represents one of the numbers from 0,1,2,3,4,5,6,7,8,9.
  • Structure of hidden layer.  Neural network based on how activation in one layer leads to activation in another layer and information is transferred from one layer to another.
  • Activation of Hidden Layer. In biology, one neuron fires the other neurons and based on brightness of neurons information sis transferred. In this case, the neural network is about recognizing one digit in a handwritten text.
  • Forming patterns in the hidden layer. So when we feed an image all the 784 neurons in the input layer light up according the value of grayscale in the pixel of the image.
  • This brightness trigger a pattern in the next layer which triggers the pattern in the next layer which finally leads to a pattern in the output layer.
  • Interconnection between hidden layers. The idea is to break down each image into edges, edges into patter and patterns into digits. Each image is broken into many pixels and analyzed for component like edges and then edges are matched with one another for patterns and then patterns are corresponded into the most likely digit represented by these patterns.
  • Calculations inside the input layer. The hidden layer assign some weights to the numbers from input layer. These weights are arbitrarily given. Then all od these weights are summed together. These weights are group in grid of their own, Green pixels are positive and red are negative pixels.
  • Sigmoid function. Since the weighs are positive and negative, we want to squeeze them in between 0 and 1 because those are grayscale value within each neuron. So a common mathematical function that reduces scale of number is sigmoid function. Sigmoid function changes distribution of weighs from negative number to more positive number.
  • Mathematical Expression for Activation of Neuron. Activation of neuron is the mathematical function of positive the sum of weighs is.
  • Introducing bias calculation. But maybe we don’t want to know how positive the number is, we just want to check if the sum of weighs is greater than 10. In this case we will subtract 10 from the sum of weighs. This called introducing bias in the calculation. We add bias before plugging it into sigmoid function.
  • Role of Weigh and Bias. The weigh tells you what pixel pattern neuron in the layer picking up on. The bias tells you how high the weighted sum has to be before the neuron layer to be activated.
  • Interconnection between input layer and hidden layer. For 784 neurons, there are 784 sum of weighs with their biases. One neuron in the hidden layer is connected to all 784 neurons from the input layer.
  • Learning to get the right weighs and biases. Each layer of neuron focus on one part of the output. Initial layer focuses on identifying edges, the other layer focuses on looking at the lines and the other layer connects the two together to identify digits of handwriting.
  • Matrix notation of writing down the summation of weighs. All the activations are written down in one column, all the weighs are written down in rows. Their multiplication results in one of the sum of activation with weighs.

The Indistinguishable You!
https://phys.org/news/2019-05-high-quality-photons-quantum.html