Question

4.7. Consider a two-layer feedforward ANN with two inputs a and b, one hidden unit c, and one output unit d. This network has five weights (wca, Wcb, Wco, Wse, Wao). where wro represents the threshold weight for unit x. Initialize these weights to the values (.1,.1,.1,.1,.1), then give their values after each of the first two training iterations of the BACKPROPAGATION algorithm. Assume learning rte '-.3, momentum α-: 0.9, incremental weight updates, and the following training examples: 0 1 0 AGATION algorithm from Table 4.2 (p.98). This entails that you should assume that the hiddern unit c and the output unit d are sigmoid units. Use stochastic gradient descent. This means that in iteration 1, you should present the first training example and update the weights. In iteration 2, you should present the second training example and update the weights again. It is in iteration 2 that momentum starts playing a role. BACKPROPAGATION(training examples, η, nin , nout , n hidden) Each training example is a pair of the form (i,i ), where i is the vector of network input values, and i is the vector of target network output values. η is the learning rate (e.g., .05). nin is the number of network inputs, nhidden the number of units in the hidden layer, and nout the number of output units. The input from unit i into unit j is denoted xji, and the weight from unit i to unit j is denoted tD Create a feed-forward network with nin inputs, nhidden hidden units, and nour output units. Initialize all network weights to small random numbers (e.g., between -05 and.05). Until the termination condition is met, Do For each (z,i) in training examples, Do Propagate the input forward through the network 1. Input the instance i to the network and compute the output ou of every unit u in the network. errors backward through the network. 2. For each network output unit k, calculate its error term δk (T4.3) 3, For each hidden unit h , calculate its emor term (T4.4) keoutputs 4. Update each network weight wji where (T4.5)

Answer 1

-ixut H# Giventhat, Yate n 0 3 C 6:07 Wod. CwOC Wac wcdy d1 Ound Hee, Input a =1,66 Signoi d (a. wac ttw bc-o9woc - s'gnoid Lo o = signoid xol+ 0.S0JS 199 +9.990 t0.01 M0.505 autput of c Signoid (c.wcd to: xwod) noid Lo.oso09 x0 Ito863 e 0.5 35 fo Problem Ace&ding -6.535 = 0 465 Step t Initialization set weights blu all +2.4 whene Fi To ta inputt of no heuron ne twort the

Step a AcIiNation layer w (P)] dp of Hidden calculate neuroni actual snoid 4:CP) iCP hidden the peuron Jayer actvation func ton signoiel and the actual Calculate the outputs the neuron au tput by no. of inputs of A N olplayer inthe neunon k step-3 wcight taining gradien toy Calculate the the neuiDnS olp Layer SCP)CP) 1-9( ekc P) -YkC kp)=4 dk C Lorrections Calulate se ight A ukep) ). de( wcights neurons output update the at the Auykp wik for calaulate the adient neuyons eyYon Hcen layers wjkCP lonnection weight calculate wijCp the pdate at weights hi dden neurvons wij (P

Step H step a and interalions Con tineLe go back Catirfied topmection until selec ted PYDCess weight taining NOw .535 o.533) 1-0 535) Lo46 =.Yd C-4de LoYTection eght n.ydt Sd Awcd 0-3+ 0.016 6.0310s Colis) wed eC-yd 0.00S11 x b.008 Awac 0-3 XIX b.008 nx axse - Awbc A 8n(o.9) xfc NOo + b.0084 wa c wact wac 0.1 o9 o.D6756-6 S007 .c d twcd+ Autd . 0.018 0.148 keoc Woct AWoc wod Wod +Awod Awoc nxO 9 x cD.3 0.9xo.OD 8 xb.x d 0-3 xo.9 YOS 660075 Awod= 0. 83165

O. O0DO 7S D10075 wod 6 0.030 S O13 input will be weits NOLO these get signoid Coxo 08 we l xoo 0008 1) (D900) -0. 000 1S 0.506 1-9 1+ oto.099 to.90 PR signoid (o.506 xo408 (o.993 C0.0 Dr Sa6 tt0-90 Signoid (o.5aG o.lH8(o993) (0-035 Signoid Co.o t) eo 1+0.96 51o o.510 ce) weight training dd NOco e =0.sto (o-49)o,T) -D12 .3x .sto x-0.12 nyddd Awcd Ae n (o. 3 C-0.1 0.037 C1-님d Sa wcd de = 0.526 (1- OS) o. (0.148) o.636 X -0. 00d Awbcn x bxc 6-31 Cb DoJ Ca zo) Awac -d

(0.90) C-0 00 Dwocnx b.9 x Sc °3 x0-9x cD.00) -0.00108 Awod= nx 0.9x Sd o -3. X Co.9) (-0.la 0039 wac- 0.1084 +6 Uact Awac wbc wh +Awb= +CO.0012 o9788 o.9007 C. ool o 8)- O.89 O48-0.01 0 129 wed wed + Awcd 96 b.13 + CO.0D l08 =0.99 wac WgC+ Jwoc o(0075 0olo woa' wod touod .1305 0 034 016YS . After Lompletion training the se the eye weight hd training No this input weghs will the above b0 a d (o. 8) Co.) lx D 1084 te-b 020 1 1+0.48 98 + 1.8 Co99) (D16us signoid Lo.065-o.16 te Go-0 b.47 l084 tD.0ol 2 wad wac t Awac -01096 .9988 1 O.89 t0.00f =0 89/ wcd wcd tcd b.0179 146 99 0 037 O.099 +0.DO 1.b27 wod Wod Auod 0.164 tO.00/ M0.165

NOco this beights get 9cE Signoi d (oxo.lo96 + x 6.9988 -o.81 xo1) in pats wil be a-o, We signoid Ca 97 88 0.0891): (6.9097 1 e6.3077 -t 1-9 Signoid Co.52 KO-146 t10 (O D4SD16) O165) signold 2t e 0.244 t0 78 enor ce) 6-.S6 O.SA (PR-PP o 137 Co 6 yd a dd = 0 3 x0 56 x (-o.137)- -D 023 Awcd : -D.0366 nco.8) x C-D.l3 Aa, d) .S2 (0.48) (131) Co 146) -o.604 ewac nx axde D Ix C-0.0dH) -o00 nx Co-6olx Co-004) 1-6-0000 o/2 Awocnx b0 C-o 00) 6-6000/2 AWod n 0.037 x C-0-001) 6 0006 444 inal weight atter the raining - waot wac 1o96 t6 1o7 wad b.9948 wbo o9988 o.60 +Awb= C-0 60000 12 ) -0.870 0.023) (-D.0366 wcd cd Awcd b123 t027 69964 woc woc+ swoc- o. b.o00 l2 = 6.1000/2 b.IGS to.0000444 = 6.1650444 wo d wod tdwod