Question

2. Steepest descent for unconstrained quadratic function minimization The steepest descent method for minimize f(x) is the gradient descent method using exact line search, that is, the step size of the kth iteration is chosen as Ok = argmin f(x“ – av f(x)). a20 (a) (3 points) Consider the objective function f(x):= *xAx- Ax - c^x + d. where A e RrXnCER”, d E R are given. Assume that A is symmetric positive definite and, at xk, f(x) = 0. Give a formula of or in terms xk, A, c, d. (b) (5 points) Consider R2. Starting from xº = (0,0), perform two iterations of the steepest descent method for minimize f (11,22) := 5(+ zì) +222 +1. +

Answer 1

2. Steepest descent for unconstrained quadratic function minimization The steepest descent method for minimize f(x) is the gr

Aus: - a) Given a Function is fi1rn IR that differentiable the at xo, direction of Stee epest descent is the vector -of (xo) to See take the Function Oct) f(xottu) U is unit Vector Llull = 1 → 0(t) = df. dxi dx, de t. f df dan dan df df dxi 11 uit de un + dxn so) 600) of(xo). u All of cxo) Il Caso of Cao) u= : 0=TT Il cf Croll d'Co)--(17f(xo) Il

Continuing this process, we have Caf C) = of Co,-2) = (0,0) Now, $ct) = (6-2) --Co)) Ef Co,-2) & Co+ult(u)+1 2-uti =al It is constant so we stop our iteration. if &ct)= function of t then we prooced as Above.

b) starking From - Covo) minimize f(2, (82) = 2(x2+x2) +262 tl Of C..22) = (31,82 +2) Of (20)= (0,2) He then minimi ze the function 06+) = f(Coro)-t62) = f(-; -2+) by computing O'(t) = -of Et,-2t). C012) -6-t, -2t+2).(0, 2) - - Co, -uttu) aut-u ut-u=0 &t=1 $"(t)= uso & mimima. =Xi = Xo -_ f (20)= (0,0)-1(0, 2) (6,-2) 2= No-1 0f (20) = Co)-1012) = 0,-2)

define doct)= f(xort Ef (x0)) 4,=xo-to of (ko) and Continue the process, by searching From ni in the direction of obtain by Minimizing Ø, 1th -f 100,- tow) ) and f (xi) to X2 - So on This is the method of given an steepest descent guess to the method initial Computer a sequence of iterates {Xx] Where XK+ = XK-tk of Gre) ; K= 0,112,- Whare tko minimizes the function okCt)= f[XK-t ofQx)]