MA332 lecture notes, Fall 1998
Week 9 Tuesday

Non-linear systems (continued)

Faires & Burden Chapter 10

1) midterm evaluation

1.5) bound variables

     In mathematical formal language:

     x = 3
     for all x, x+7 = 7+x

     The second x does not refer to the first x, because it
     is bound by the "for all" clause.

     In computer formal languages:

     x:=17;
     square:=x->x^2;

     The second x does not refer to the first because it is
     an argument of the functional operator (also a local
     variable).

     Similarly:

     square := proc(x) x^2; end:

     This is relevant to the presentation of Broyden's method, both
     in the book and in these notes.

2) Broyden's method

   a) don't use J at each location, estimate J based on observed
      change in the direction of motion and previous estimate of J

   b) don't invert the estimated J (called A) at each step, instead,
      calculate the inverse based on the inverse of the previous A

3) steepest descent method

   a) minimize g(p) = || F(p) ||2

   b) easy to find direction of maximum descent = -gradient of g

   c) art form to decide how far to go in that direction

   d) nice thing about steepest descent is that it works for
      overconstrained systems (more equations than variables).
      g(p) is never zero, but we can still minimize it.

4) Levenberg-Marquart method

   a) Newton is fast but squirrely

   b) steepest descent is slow but very reliable

   c) use a weighted average of the two.  Start with the
      emphasis on steepest descent and gradually shift
      toward Newton if things are behaving well