Stanford EE 364A: Convex Optimization

Author

Chao Ma

Published

December 1, 2025

All Lectures

My notes from Stanford EE 364A: Convex Optimization. This course covers the theory and applications of convex optimization, including least squares, linear programming, and convex optimization problems.

Lecture 1: Introduction to Convex Optimization Introduction to constraint optimization problems: least squares ($\|Ax-b\|_2^2$), linear programming ($c^\top x$ with linear constraints), and convex optimization (convex objective and constraints). Convex optimization generalizes both while maintaining polynomial-time solvability: $O(\max(n^3, n^2m, F))$ where $F$ is derivative computation cost.

Lecture 2: Convex Sets Convex sets (affine combinations, convex combinations, convex hull, convex cone), hyperplanes and halfspaces, Euclidean balls and ellipsoids, norm balls and norm cones (including second-order cone), polyhedra, positive semidefinite cone, operations preserving convexity (intersection, affine functions, perspective, linear-fractional), generalized inequalities (proper cones, componentwise and matrix inequalities), and minimum vs minimal elements.

Lecture 3: Convex Functions Separating and supporting hyperplane theorems, dual cones and generalized inequalities, convex function definition, first-order condition (tangent underestimates), second-order condition (Hessian PSD), extended value extension, restriction to a line, epigraph and sublevel sets, Jensen’s inequality, and operations preserving convexity (nonnegative sums, affine composition, pointwise maximum).

Lecture 4 Part 1: Operations preserving Convexity Pointwise maximum and supremum (support function, distance to farthest point, maximum eigenvalue), composition with scalar functions (exponential, reciprocal), vector composition (log-sum-exp), minimization over convex sets (Schur complement, distance to set), and perspective functions with examples.

Lecture 4 Part 2: Conjugate and Quasiconvex Functions Conjugate functions $f^*(y) = \sup_x (y^\top x - f(x))$ are always convex, with examples including negative logarithm and quadratic functions. Quasiconvex functions have convex sublevel sets with modified Jensen inequality $f(\theta x + (1-\theta)y) \leq \max\{f(x), f(y)\}$. Examples include linear-fractional functions and distance ratios.

Lecture 5 Part 1: Log-Concave and Log-Convex Functions Log-concave functions satisfy $f(\theta x+(1-\theta)y)\ge f(x)^\theta f(y)^{1-\theta}$. Powers $x^a$ are log-concave for $a \ge 0$ and log-convex for $a \le 0$. Key properties: products preserve log-concavity, but sums do not. Integration of log-concave functions preserves log-concavity.

Lecture 5 Part 2: Monotonicity with Generalized Inequalities K-nondecreasing functions satisfy $x \preceq_K y \Rightarrow f(x)\le f(y)$. Gradient condition: $\nabla f(x) \succeq_{K^*} 0$ (dual inequality). Matrix monotone examples: $\mathrm{tr}(WX)$, $\mathrm{det}X$. K-convexity extends convexity to generalized inequalities with dual characterization and composition theorems.

Chapter 4.1: Optimization Problems Basic terminology (decision variables, objective, constraints, domain, feasibility, optimal values, local optimality), standard form conversion, equivalent problems (change of variables, slack variables, constraint elimination, epigraph form), and parameter vs oracle problem descriptions.

Chapter 4.2: Convex Optimization Convex problems in standard form (convex objective/inequalities and affine equalities), convexity of feasible sets, local optimality implying global optimality, first-order optimality for unconstrained, equality-constrained, and nonnegative-orthant problems, equivalence transformations (change of variables, monotone transformations, slack variables), and quasiconvex problems solved via bisection on feasibility.

Chapter 4.3: Linear Optimization Linear programs (LP) minimize linear objectives subject to affine inequalities and equalities. Feasible sets are polyhedra; optimum occurs where level curves orthogonal to objective vector touch the feasible region. Applications: diet problem (minimize cost subject to nutritional constraints), piecewise-linear minimization via epigraph formulation, Chebyshev center (largest inscribed ball), linear fractional programs (converted to LP by change of variables $y=xz$, $z=1/(e^\top x+f)$). Von Neumann growth model maximizes minimum sectoral growth rate via quasiconvex bisection.

Chapter 4.4: Quadratic Optimization Problems Quadratic programs minimize convex quadratic objectives with affine constraints. QCQPs generalize QP by allowing quadratic inequalities, while SOCPs capture norm constraints and unify LP and QCQP. Robust LP reformulates uncertainty with ellipsoidal or probabilistic constraints, yielding SOCPs with norm terms and Gaussian chance constraints.

--- title: "Stanford EE 364A: Convex Optimization" author: "Chao Ma" date: "2025-12-01" --- ## All Lectures My notes from Stanford EE 364A: Convex Optimization. This course covers the theory and applications of convex optimization, including least squares, linear programming, and convex optimization problems. --- ::: {.content-grid} ::: {.content-card} **[Lecture 1: Introduction to Convex Optimization](ee364a-lecture1-intro.qmd)** Introduction to constraint optimization problems: least squares ($\|Ax-b\|_2^2$), linear programming ($c^\top x$ with linear constraints), and convex optimization (convex objective and constraints). Convex optimization generalizes both while maintaining polynomial-time solvability: $O(\max(n^3, n^2m, F))$ where $F$ is derivative computation cost. ::: ::: {.content-card} **[Lecture 2: Convex Sets](ee364a-lecture2-math-foundations.qmd)** Convex sets (affine combinations, convex combinations, convex hull, convex cone), hyperplanes and halfspaces, Euclidean balls and ellipsoids, norm balls and norm cones (including second-order cone), polyhedra, positive semidefinite cone, operations preserving convexity (intersection, affine functions, perspective, linear-fractional), generalized inequalities (proper cones, componentwise and matrix inequalities), and minimum vs minimal elements. ::: ::: {.content-card} **[Lecture 3: Convex Functions](ee364a-lecture3-convex-functions.qmd)** Separating and supporting hyperplane theorems, dual cones and generalized inequalities, convex function definition, first-order condition (tangent underestimates), second-order condition (Hessian PSD), extended value extension, restriction to a line, epigraph and sublevel sets, Jensen's inequality, and operations preserving convexity (nonnegative sums, affine composition, pointwise maximum). ::: ::: {.content-card} **[Lecture 4 Part 1: Operations preserving Convexity](ee364a-lecture4-part1-operations-preserving-convexity.qmd)** Pointwise maximum and supremum (support function, distance to farthest point, maximum eigenvalue), composition with scalar functions (exponential, reciprocal), vector composition (log-sum-exp), minimization over convex sets (Schur complement, distance to set), and perspective functions with examples. ::: ::: {.content-card} **[Lecture 4 Part 2: Conjugate and Quasiconvex Functions](ee364a-lecture4-part2-conjugate-quasiconvex.qmd)** Conjugate functions $f^*(y) = \sup_x (y^\top x - f(x))$ are always convex, with examples including negative logarithm and quadratic functions. Quasiconvex functions have convex sublevel sets with modified Jensen inequality $f(\theta x + (1-\theta)y) \leq \max\{f(x), f(y)\}$. Examples include linear-fractional functions and distance ratios. ::: ::: {.content-card} **[Lecture 5 Part 1: Log-Concave and Log-Convex Functions](ee364a-lecture5-part1-log-concave-convex.qmd)** Log-concave functions satisfy $f(\theta x+(1-\theta)y)\ge f(x)^\theta f(y)^{1-\theta}$. Powers $x^a$ are log-concave for $a \ge 0$ and log-convex for $a \le 0$. Key properties: products preserve log-concavity, but sums do not. Integration of log-concave functions preserves log-concavity. ::: ::: {.content-card} **[Lecture 5 Part 2: Monotonicity with Generalized Inequalities](ee364a-lecture5-part2-monotonicity.qmd)** K-nondecreasing functions satisfy $x \preceq_K y \Rightarrow f(x)\le f(y)$. Gradient condition: $\nabla f(x) \succeq_{K^*} 0$ (dual inequality). Matrix monotone examples: $\mathrm{tr}(WX)$, $\mathrm{det}X$. K-convexity extends convexity to generalized inequalities with dual characterization and composition theorems. ::: ::: {.content-card} **[Chapter 4.1: Optimization Problems](ee364a-lecture6-optimization-problems.qmd)** Basic terminology (decision variables, objective, constraints, domain, feasibility, optimal values, local optimality), standard form conversion, equivalent problems (change of variables, slack variables, constraint elimination, epigraph form), and parameter vs oracle problem descriptions. ::: ::: {.content-card} **[Chapter 4.2: Convex Optimization](ee364a-lecture7-convex-optimization.qmd)** Convex problems in standard form (convex objective/inequalities and affine equalities), convexity of feasible sets, local optimality implying global optimality, first-order optimality for unconstrained, equality-constrained, and nonnegative-orthant problems, equivalence transformations (change of variables, monotone transformations, slack variables), and quasiconvex problems solved via bisection on feasibility. ::: ::: {.content-card} **[Chapter 4.3: Linear Optimization](ee364a-lecture8-linear-optimization.qmd)** Linear programs (LP) minimize linear objectives subject to affine inequalities and equalities. Feasible sets are polyhedra; optimum occurs where level curves orthogonal to objective vector touch the feasible region. Applications: diet problem (minimize cost subject to nutritional constraints), piecewise-linear minimization via epigraph formulation, Chebyshev center (largest inscribed ball), linear fractional programs (converted to LP by change of variables $y=xz$, $z=1/(e^\top x+f)$). Von Neumann growth model maximizes minimum sectoral growth rate via quasiconvex bisection. ::: ::: {.content-card} **[Chapter 4.4: Quadratic Optimization Problems](ee364a-lecture9-quadratic-optimization.qmd)** Quadratic programs minimize convex quadratic objectives with affine constraints. QCQPs generalize QP by allowing quadratic inequalities, while SOCPs capture norm constraints and unify LP and QCQP. Robust LP reformulates uncertainty with ellipsoidal or probabilistic constraints, yielding SOCPs with norm terms and Gaussian chance constraints. ::: :::