Question

Neural Networks

We will now build some neural networks to represent basic boolean functions. For simplicity, we use the threshold function as our basic units instead of the sigmoid function, where threshold(t) +1 if the input is greater than 0, and 0 otherwise, we have inputs xi (+1, 0) and weights yī (possible values-l, 0, 1). Suppose we are given boolean input data xi where 1 represents TRUE and 0 represents FALSE. The boolean NOT function can be represented by a single-layer, single-unit neural net such that the output is +1 if and only if TRUE: wl Find the appropriate weights for wl, w2, w3 for the neural network above to represent the OR function a) Find the appropriate weights for wi, w2, w3 for the neural network above to represent the AND function b) Recall that the XOR function for x1 and x2. A single perceptron cannot successfully represent the XOR function. However, a neural net consisting of multiple perceptron units should be able to. Devise a multi-unit neural net (also multi-layer if needed) that computes the XOR of two inputs x1 and x2 using the same basic unit as the neural nets above. Draw the layout diagram below and specify the weights of different edges clearly. (hint, using answers from a and b) c)

Answer 1

Let the threshold function be denoted by g(t)

a) OR

Output table for OR Function

x1	x2	y
0	0	0
0	1	1
1	0	1
1	1	1

g(w1*0+w2*0+w3)=0 (x1=0,x2=0) $\Rightarrow$ g(w3)=0 ...................(1)
g(w1*0+w2*1+w3)=1 (x1=0,x2=1) $\Rightarrow$ g(w2+w3)=1 ...................(2)
g(w1*1+w2*0+w3)=1 (x1=1,x2=0) $\Rightarrow$ g(w1+w3)=1 ...................(3)
g(w1*1+w2*1+w3)=1 (x1=1,x2=1) $\Rightarrow$ g(w1+w2+w3)=1 ...................(4)

From (1), g(w3)=0 $\Rightarrow$ w3=0
From (2), g(w2+w3)=1 $\Rightarrow$ w2+w3>0 => w2>0 $\Rightarrow$ w2=1
From (3), g(w1+w3)=1 $\Rightarrow$ w1+w3>0 => w1>0 $\Rightarrow$ w1=1

Hence. w1=1,w2=1,w3=0
These can be verified by putting in (4)

a) AND

Output table for AND Function

x1	x2	y
0	0	0
0	1	0
1	0	0
1	1	1

g(w1*0+w2*0+w3)=0 (x1=0,x2=0) $\Rightarrow$ g(w3)=0 ...................(1)
g(w1*0+w2*1+w3)=0 (x1=0,x2=1) $\Rightarrow$ g(w2+w3)=0 ...................(2)
g(w1*1+w2*0+w3)=0 (x1=1,x2=0) $\Rightarrow$ g(w1+w3)=0 ...................(3)
g(w1*1+w2*1+w3)=1 (x1=1,x2=1) $\Rightarrow$ g(w1+w2+w3)=1 ...................(4)

From (1), g(w3)=0 $\Rightarrow$w3=-1 as w3=0 will not satisfy equation (4).
From (2), g(w2+w3)=0 $\Rightarrow$w2+w3<=0 $\Rightarrow$ w2+(-1)<=0 $\Rightarrow$ w2=1 as w2=0,-1 will not satisfy equation (4).
From (3), g(w1+w3)=0 $\Rightarrow$w1+w3<=0 $\Rightarrow$ w1+(-1)<=0 $\Rightarrow$ w1=1 as w1=0,-1 will not satisfy equation (4).

Hence. w1=1,w2=1,w3=-1

c) XOR

Output table for XOR Function

x1	x2	y
0	0	0
0	1	1
1	0	1
1	1	0

Boolean Representation of XOR -
x1x2'+x1'x2 (x1' represents x1 complement)
Simplification -
$\Rightarrow$x1'x2 + x1x2' + x1'x1 + x2'x2
$\Rightarrow$x1(x1' + x2') + x2(x1' + x2')
$\Rightarrow$(x1 + x2)(x1' + x2')
$\Rightarrow$(x1 + x2)(x1x2)'
It is a combination of 1 OR, 2 AND and 1 NOT function.

1 D う 1 90224