Continuous Optimization Course

For the SCiML Group, 2021

Nick Gould and Jari Fowkes

Optimization deals with the problem of minimising or maximising a mathematical model of an objective function such as cost, fuel consumption etc. under a set of side constraints on the domain of definition of this function. Optimization theory is the study of the mathematical properties of optimization problems and the analysis of algorithms for their solution. The aim of this course is to provide an introduction to nonlinear continuous optimization specifically tailored to the background of scientists with a basic mathematical training.

The major pre-requisites for the course will be some knowledge of both linear algebra and real analysis, while an appreciation of methods for the numerical solution of linear systems of equations will be helpful.

The course consists of five lectures (Mon-Fri 9--10ish) and practicals (10:30-12).

The lectures will be based on the slides, while more material is available in an accompanying booklet. Both of these will be updated to fix any typos and errors found!

The practicals can be found in the following jupyter notebooks, please download these and run them in your own jupyter instance or on Google Colab:

  1. Introduction (answers)

  2. Linesearch (answers)

  3. Trust Region (answers)

  4. Penalty Methods (answers)

  5. Interior Point (answers)

The following books contain useful background material. The first is particularly recommended:

  • J. Nocedal and S. Wright, Numerical Optimization, Springer Verlag 1999

  • J. Dennis and R. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, (republished by) SIAM (Classics in Applied Mathematics 16) 1996

  • P. Gill, W. Murray and M. Wright, Practical Optimization, Academic Press 1981

  • R. Fletcher, Practical Methods of Optimization, 2nd edition Wiley 1987, (republished in paperback 2000)

  • A. Conn, N. Gould and Ph. Toint, Trust-Region Methods, SIAM 2000

RCUK