{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction\n",
    "First load the necessary packages for this training course"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Vector Norms\n",
    "Recall the following vector norms:\n",
    "\n",
    "$\n",
    "\\begin{align}\n",
    "&\\lVert x \\rVert_1 = \\sum_i \\lvert x_i \\rvert \\\\\n",
    "&\\lVert x \\rVert_2 = \\sqrt{\\sum_i x_i^2} \\\\\n",
    "&\\lVert x \\rVert_\\infty = \\max_i \\lvert x_i \\rvert\n",
    "\\end{align}\n",
    "$\n",
    "\n",
    "**Coding Task:**\n",
    "\n",
    "For each of the above norms, find and plot the boundaries of the regions in 2D where $\\lVert x \\rVert \\le 1$ for $x=(x_1,x_2)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create figure window\n",
    "plt.figure()\n",
    "\n",
    "# Plot regions \n",
    "plt.plot(...)\n",
    "plt.axis('equal')\n",
    "\n",
    "# Show plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Convex Quadratics\n",
    "Consider a convex quadratic of the form\n",
    "\n",
    "$\n",
    "\\begin{align}\n",
    "f(x) = \\frac{1}{2} (ax_1^2 + bx_2^2)\n",
    "\\end{align}\n",
    "$\n",
    "\n",
    "for $a,b > 0$.\n",
    "\n",
    "#### Coding Task:\n",
    "\n",
    "Plot the contours of $f(x)$ above using Matplotlib's plt.contour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scaling parameters\n",
    "a = 10\n",
    "b = 1\n",
    "\n",
    "# Objective function and gradient\n",
    "fun = lambda x: \n",
    "\n",
    "# Plot function contours\n",
    "plt.figure()\n",
    "X = np.linspace(...)\n",
    "Y = np.linspace(...)\n",
    "Z = np.meshgrid(X,Y)\n",
    "plt.contour(...)\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises:\n",
    "\n",
    "1. Calculate the gradient and Hessian of $f(x)$.  \n",
    "\n",
    "2. Recall that $f(x)$ is Lipschitz continuous in $X$ if there exists a $\\gamma > 0$ such that for all $x,y \\in X$:\n",
    "\n",
    "    $\n",
    "    \\lvert f(x) - f(y) \\rvert \\le \\gamma \\lVert x - y \\rVert_2 \n",
    "    $\n",
    "\n",
    "    Is the convex quadratic $f(x)$ above Lipschitz continuous when $X=[\\alpha,\\beta]^2$? \n",
    "    What about when $X = \\mathbb{R}^2$? You may find it helpful to plot $f(x)$ in 1D.\n",
    "\n",
    "3. Similarly, $\\nabla f(x)$ is Lipschitz continuous in $X$ if there exists a $\\gamma > 0$ such that for all $x,y \\in X$:\n",
    "\n",
    "    $\n",
    "    \\lVert \\nabla f(x) - \\nabla f(y) \\rVert_2 \\le \\gamma \\lVert x - y \\rVert_2 \n",
    "    $\n",
    "\n",
    "    Is the gradient $\\nabla f(x)$ Lipschitz continuous when $X=[\\alpha,\\beta]^2$? What about when $X = \\mathbb{R}^2$?\n",
    "\n",
    "4. Similarly, is the Hessian $\\nabla^2 f(x)$ Lipschitz continuous?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Rosenbrock Function\n",
    "Consider the Rosenbrock function defined as\n",
    "\n",
    "$\n",
    "f(x) = (a - x_1)^2 + b(x_2 - x_1^2)^2\n",
    "$\n",
    "\n",
    "where usually $a=1$ and $b=100$.\n",
    "\n",
    "#### Coding Task:\n",
    "\n",
    "Plot the contours of $f(x)$ above using Matplotlib's plt.contour. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters\n",
    "a = 1\n",
    "b = 100\n",
    "\n",
    "# Objective function and gradient\n",
    "fun = lambda x: \n",
    "\n",
    "# Plot function contours\n",
    "plt.figure()\n",
    "X = np.linspace(...)\n",
    "Y = np.linspace(...)\n",
    "Z = np.meshgrid(X,Y)\n",
    "plt.contour(...)\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises: \n",
    "1. Calculate the gradient and Hessian of this function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}