Functions¶

This section of the tutorial describes the functions that can be applied to CVXPY expressions. CVXPY uses the function information in this section and the DCP rules to mark expressions with a sign and curvature.

Operators¶

The infix operators +, -, *, / are treated as functions. + and - are affine functions. * and / are affine in CVXPY because expr1*expr2 is allowed only when one of the expressions is constant and expr1/expr2 is allowed only when expr2 is a scalar constant.

Indexing and slicing¶

All non-scalar expressions can be indexed using the syntax expr[i, j]. Indexing is an affine function. The syntax expr[i] can be used as a shorthand for expr[i, 0] when expr is a column vector. Similarly, expr[i] is shorthand for expr[0, i] when expr is a row vector.

Non-scalar expressions can also be sliced into using the standard Python slicing syntax. For example, expr[i:j:k, r] selects every kth element in column r of expr, starting at row i and ending at row j-1.

CVXPY supports advanced indexing using lists of indices or boolean arrays. The semantics are the same as NumPy (see NumPy advanced indexing). Any time NumPy would return a 1D array, CVXPY returns a column vector.

Transpose¶

The transpose of any expression can be obtained using the syntax expr.T. Transpose is an affine function.

Power¶

For any CVXPY expression expr, the power operator expr**p is equivalent to the function power(expr, p).

Scalar functions¶

A scalar function takes one or more scalars, vectors, or matrices as arguments and returns a scalar.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
geo_mean(x) geo_mean(x, p) $p \in \mathbf{R}^n_{+}$ $p \neq 0$	$x_1^{1/n} \cdots x_n^{1/n}$ $\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}$	$x \in \mathbf{R}^n_{+}$	positive	concave	incr.
harmonic_mean(x)	$\frac{n}{\frac{1}{x_1} + \cdots + \frac{1}{x_n}}$	$x \in \mathbf{R}^n_{+}$	positive	concave	incr.
lambda_max(X)	$\lambda_{\max}(X)$	$X \in \mathbf{S}^n$	unknown	convex	None
lambda_min(X)	$\lambda_{\min}(X)$	$X \in \mathbf{S}^n$	unknown	concave	None
lambda_sum_largest(X, k) $k = 1,\ldots, n$	$\text{sum of $k$ largest}\\ \text{eigenvalues of $X$}$	$X \in\mathbf{S}^{n}$	unknown	convex	None
lambda_sum_smallest(X, k) $k = 1,\ldots, n$	$\text{sum of $k$ smallest}\\ \text{eigenvalues of $X$}$	$X \in\mathbf{S}^{n}$	unknown	concave	None
log_det(X)	$\log \left(\det (X)\right)$	$X \in \mathbf{S}^n_+$	unknown	concave	None
log_sum_exp(X)	$\log \left(\sum_{ij}e^{X_{ij}}\right)$	$X \in\mathbf{R}^{m \times n}$	unknown	convex	incr.
matrix_frac(x, P)	$x^T P^{-1} x$	$x \in \mathbf{R}^n$ $P \in\mathbf{S}^n_{++}$	positive	convex	None
max_entries(X)	$\max_{ij}\left\{ X_{ij}\right\}$	$X \in\mathbf{R}^{m \times n}$	same as X	convex	incr.
min_entries(X)	$\min_{ij}\left\{ X_{ij}\right\}$	$X \in\mathbf{R}^{m \times n}$	same as X	concave	incr.
mixed_norm(X, p, q)	$\left(\sum_k\left(\sum_l\lvert x_{k,l}\rvert^p\right)^{q/p}\right)^{1/q}$	$X \in\mathbf{R}^{n \times n}$	positive	convex	None
norm(x) norm(x, 2)	$\sqrt{\sum_{i}x_{i}^2 }$	$X \in\mathbf{R}^{n}$	positive	convex	for $x_{i} \geq 0$ for $x_{i} \leq 0$
norm(X, “fro”)	$\sqrt{\sum_{ij}X_{ij}^2 }$	$X \in\mathbf{R}^{m \times n}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, 1)	$\sum_{ij}\lvert X_{ij} \rvert$	$X \in\mathbf{R}^{m \times n}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, “inf”)	$\max_{ij} \{\lvert X_{ij} \rvert\}$	$X \in\mathbf{R}^{m \times n}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, “nuc”)	$\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)$	$X \in\mathbf{R}^{m \times n}$	positive	convex	None
norm(X) norm(X, 2)	$\sqrt{\lambda_{\max}\left(X^T X\right)}$	$X \in\mathbf{R}^{m \times n}$	positive	convex	None
pnorm(X, p) $p \geq 1$ or `p = 'inf'`	$\\|X\\|_p = \left(\sum_{ij} \|X_{ij}\|^p \right)^{1/p}$	$X \in \mathbf{R}^{m \times n}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
pnorm(X, p) $p < 1$, $p \neq 0$	$\\|X\\|_p = \left(\sum_{ij} X_{ij}^p \right)^{1/p}$	$X \in \mathbf{R}^{m \times n}_+$	positive	concave	incr.
quad_form(x, P) constant $P \in \mathbf{S}^n_+$	$x^T P x$	$x \in \mathbf{R}^n$	positive	convex	for $x_i \geq 0$ for $x_i \leq 0$
quad_form(x, P) constant $P \in \mathbf{S}^n_-$	$x^T P x$	$x \in \mathbf{R}^n$	negative	concave	for $x_i \geq 0$ for $x_i \leq 0$
quad_form(c, X) constant $c \in \mathbf{R}^n$	$c^T X c$	$X \in\mathbf{R}^{n \times n}$	depends on c, X	affine	depends on c
quad_over_lin(X, y)	$\left(\sum_{ij}X_{ij}^2\right)/y$	$x \in \mathbf{R}^n$ $y > 0$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$ decr. in $y$
sum_entries(X)	$\sum_{ij}X_{ij}$	$X \in\mathbf{R}^{m \times n}$	same as X	affine	incr.
sum_largest(X, k) $k = 1,2,\ldots$	$\text{sum of } k\text{ largest }X_{ij}$	$X \in\mathbf{R}^{m \times n}$	same as X	convex	incr.
sum_smallest(X, k) $k = 1,2,\ldots$	$\text{sum of } k\text{ smallest }X_{ij}$	$X \in\mathbf{R}^{m \times n}$	same as X	concave	incr.
sum_squares(X)	$\sum_{ij}X_{ij}^2$	$X \in\mathbf{R}^{m \times n}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
trace(X)	$\mathrm{tr}\left(X \right)$	$X \in\mathbf{R}^{n \times n}$	same as X	affine	incr.
tv(x)	$\sum_{i}\|x_{i+1} - x_i\|$	$x \in \mathbf{R}^n$	positive	convex	None
tv(X)	$\sum_{ij}\left\\| \left[\begin{matrix} X_{i+1,j} - X_{ij} \\ X_{i,j+1} -X_{ij} \end{matrix}\right] \right\\|_2$	$X \in \mathbf{R}^{m \times n}$	positive	convex	None
tv(X1,…,Xk)	$\sum_{ij}\left\\| \left[\begin{matrix} X_{i+1,j}^{(1)} - X_{ij}^{(1)} \\ X_{i,j+1}^{(1)} -X_{ij}^{(1)} \\ \vdots \\ X_{i+1,j}^{(k)} - X_{ij}^{(k)} \\ X_{i,j+1}^{(k)} -X_{ij}^{(k)} \end{matrix}\right] \right\\|_2$	$X^{(i)} \in\mathbf{R}^{m \times n}$	positive	convex	None

Clarifications¶

The domain $\mathbf{S}^n$ refers to the set of symmetric matrices. The domains $\mathbf{S}^n_+$ and $\mathbf{S}^n_-$ refer to the set of positive semi-definite and negative semi-definite matrices, respectively. Similarly, $\mathbf{S}^n_{++}$ and $\mathbf{S}^n_{--}$ refer to the set of positive definite and negative definite matrices, respectively.

For a vector expression x, norm(x) and norm(x, 2) give the Euclidean norm. For a matrix expression X, however, norm(X) and norm(X, 2) give the spectral norm.

The function norm(X, "fro") is called the Frobenius norm and norm(X, "nuc") the nuclear norm. The nuclear norm can also be defined as the sum of X’s singular values.

The functions max_entries and min_entries give the largest and smallest entry, respectively, in a single expression. These functions should not be confused with max_elemwise and min_elemwise (see Elementwise functions). Use max_elemwise and min_elemwise to find the max or min of a list of scalar expressions.

The function sum_entries sums all the entries in a single expression. The built-in Python sum should be used to add together a list of expressions. For example, the following code sums a list of three expressions:

expr_list = [expr1, expr2, expr3]
expr_sum = sum(expr_list)

Functions along an axis¶

The functions sum_entries, norm, max_entries, and min_entries can be applied along an axis. Given an m by n expression expr, the syntax func(expr, axis=0) applies func to each column, returning a 1 by n expression. The syntax func(expr, axis=1) applies func to each row, returning an m by 1 expression. For example, the following code sums along the columns and rows of a matrix variable:

X = Variable(5, 4)
col_sums = sum_entries(X, axis=0) # Has size (1, 4)
row_sums = sum_entries(X, axis=1) # Has size (5, 1)

Elementwise functions¶

These functions operate on each element of their arguments. For example, if X is a 5 by 4 matrix variable, then abs(X) is a 5 by 4 matrix expression. abs(X)[1, 2] is equivalent to abs(X[1, 2]).

Elementwise functions that take multiple arguments, such as max_elemwise and mul_elemwise, operate on the corresponding elements of each argument. For example, if X and Y are both 3 by 3 matrix variables, then max_elemwise(X, Y) is a 3 by 3 matrix expression. max_elemwise(X, Y)[2, 0] is equivalent to max_elemwise(X[2, 0], Y[2, 0]). This means all arguments must have the same dimensions or be scalars, which are promoted.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
abs(x)	$\lvert x \rvert$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
entr(x)	$-x \log (x)$	$x > 0$	unknown	concave	None
exp(x)	$e^x$	$x \in \mathbf{R}$	positive	convex	incr.
huber(x, M=1) $M \geq 0$	$\begin{cases}x^2 &\|x\| \leq M \\2M\|x\| - M^2&\|x\| >M\end{cases}$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
inv_pos(x)	$1/x$	$x > 0$	positive	convex	decr.
kl_div(x, y)	$x \log(x/y) - x + y$	$x > 0$ $y > 0$	positive	convex	None
log(x)	$\log(x)$	$x > 0$	unknown	concave	incr.
log1p(x)	$\log(x+1)$	$x > -1$	same as x	concave	incr.
logistic(x)	$\log(1 + e^{x})$	$x \in \mathbf{R}$	positive	convex	incr.
max_elemwise(x1, …, xk)	$\max \left\{x_1, \ldots , x_k\right\}$	$x_i \in \mathbf{R}$	$\max(\mathrm{sign}(x_1))$	convex	incr.
min_elemwise(x1, …, xk)	$\min \left\{x_1, \ldots , x_k\right\}$	$x_i \in \mathbf{R}$	$\min(\mathrm{sign}(x_1))$	concave	incr.
mul_elemwise(c, x) $c \in \mathbf{R}$	c*x	$x \in\mathbf{R}$	$\mathrm{sign}(cx)$	affine	depends on c
neg(x)	$\max \left\{-x, 0 \right\}$	$x \in \mathbf{R}$	positive	convex	decr.
pos(x)	$\max \left\{x, 0 \right\}$	$x \in \mathbf{R}$	positive	convex	incr.
power(x, 0)	$1$	$x \in \mathbf{R}$	positive	constant
power(x, 1)	$x$	$x \in \mathbf{R}$	same as x	affine	incr.
power(x, p) $p = 2, 4, 8, \ldots$	$x^p$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
power(x, p) $p < 0$	$x^p$	$x > 0$	positive	convex	decr.
power(x, p) $0 < p < 1$	$x^p$	$x \geq 0$	positive	concave	incr.
power(x, p) $p > 1,\ p \neq 2, 4, 8, \ldots$	$x^p$	$x \geq 0$	positive	convex	incr.
scalene(x, alpha, beta) $\text{alpha} \geq 0$ $\text{beta} \geq 0$	$\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
sqrt(x)	$\sqrt x$	$x \geq 0$	positive	concave	incr.
square(x)	$x^2$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$

Vector/matrix functions¶

A vector/matrix function takes one or more scalars, vectors, or matrices as arguments and returns a vector or matrix.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
bmat([[X11, …, X1q], …, [Xp1, …, Xpq]])	$\left[\begin{matrix} X^{(1,1)} & \cdots & X^{(1,q)} \\ \vdots & & \vdots \\ X^{(p,1)} & \cdots & X^{(p,q)} \end{matrix}\right]$	$X^{(i,j)} \in\mathbf{R}^{m_i \times n_j}$	$\mathrm{sign}\left(\sum_{ij} X^{(i,j)}_{11}\right)$	affine	incr.
conv(c, x) $c\in\mathbf{R}^m$	$c*x$	$x\in \mathbf{R}^n$	$\mathrm{sign}\left(c_{1}x_{1}\right)$	affine	depends on c
cumsum(X, axis=0)	cumulative sum along given axis.	$X \in \mathbf{R}^{m \times n}$	same as X	affine	incr.
diag(x)	$\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]$	$x \in\mathbf{R}^{n}$	same as x	affine	incr.
diag(X)	$\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]$	$X \in\mathbf{R}^{n \times n}$	same as X	affine	incr.
diff(X, k=1, axis=0) $k \in 0,1,2,\ldots$	kth order differences along given axis	$X \in\mathbf{R}^{m \times n}$	same as X	affine	incr.
hstack(X1, …, Xk)	$\left[\begin{matrix}X^{(1)} \cdots X^{(k)}\end{matrix}\right]$	$X^{(i)} \in\mathbf{R}^{m \times n_i}$	$\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)$	affine	incr.
kron(C, X) $C\in\mathbf{R}^{p \times q}$	$\left[\begin{matrix}C_{11}X & \cdots & C_{1q}X \\ \vdots & & \vdots \\ C_{p1}X & \cdots & C_{pq}X \end{matrix}\right]$	$X \in\mathbf{R}^{m \times n}$	$\mathrm{sign}\left(C_{11}X_{11}\right)$	affine	depends on C
reshape(X, n’, m’)	$X' \in\mathbf{R}^{m' \times n'}$	$X \in\mathbf{R}^{m \times n}$ $m'n' = mn$	same as X	affine	incr.
vec(X)	$x' \in\mathbf{R}^{mn}$	$X \in\mathbf{R}^{m \times n}$	same as X	affine	incr.
vstack(X1, …, Xk)	$\left[\begin{matrix}X^{(1)} \\ \vdots \\X^{(k)}\end{matrix}\right]$	$X^{(i)} \in\mathbf{R}^{m_i \times n}$	$\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)$	affine	incr.

Clarifications¶

The input to bmat is a list of lists of CVXPY expressions. It constructs a block matrix. The elements of each inner list are stacked horizontally and then the resulting block matrices are stacked vertically.

The output $y$ of conv(c, x) has size $n+m-1$ and is defined as $y[k]=\sum_{j=0}^k c[j]x[k-j]$.

The output $x'$ of vec(X) is the matrix $X$ flattened in column-major order into a vector. Formally, $x'_i = X_{i \bmod{m}, \left \lfloor{i/m}\right \rfloor }$.

The output $X'$ of reshape(X, m', n') is the matrix $X$ cast into an $m' \times n'$ matrix. The entries are taken from $X$ in column-major order and stored in $X'$ in column-major order. Formally, $X'_{ij} = \mathbf{vec}(X)_{m'j + i}$.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
geo_mean(x) geo_mean(x, p) \(p \in \mathbf{R}^n_{+}\) \(p \neq 0\)	\(x_1^{1/n} \cdots x_n^{1/n}\) \(\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}\)	\(x \in \mathbf{R}^n_{+}\)	positive	concave	incr.
harmonic_mean(x)	\(\frac{n}{\frac{1}{x_1} + \cdots + \frac{1}{x_n}}\)	\(x \in \mathbf{R}^n_{+}\)	positive	concave	incr.
lambda_max(X)	\(\lambda_{\max}(X)\)	\(X \in \mathbf{S}^n\)	unknown	convex	None
lambda_min(X)	\(\lambda_{\min}(X)\)	\(X \in \mathbf{S}^n\)	unknown	concave	None
lambda_sum_largest(X, k) \(k = 1,\ldots, n\)	\(\text{sum of $k$ largest}\\ \text{eigenvalues of $X$}\)	\(X \in\mathbf{S}^{n}\)	unknown	convex	None
lambda_sum_smallest(X, k) \(k = 1,\ldots, n\)	\(\text{sum of $k$ smallest}\\ \text{eigenvalues of $X$}\)	\(X \in\mathbf{S}^{n}\)	unknown	concave	None
log_det(X)	\(\log \left(\det (X)\right)\)	\(X \in \mathbf{S}^n_+\)	unknown	concave	None
log_sum_exp(X)	\(\log \left(\sum_{ij}e^{X_{ij}}\right)\)	\(X \in\mathbf{R}^{m \times n}\)	unknown	convex	incr.
matrix_frac(x, P)	\(x^T P^{-1} x\)	\(x \in \mathbf{R}^n\) \(P \in\mathbf{S}^n_{++}\)	positive	convex	None
max_entries(X)	\(\max_{ij}\left\{ X_{ij}\right\}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	convex	incr.
min_entries(X)	\(\min_{ij}\left\{ X_{ij}\right\}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	concave	incr.
mixed_norm(X, p, q)	\(\left(\sum_k\left(\sum_l\lvert x_{k,l}\rvert^p\right)^{q/p}\right)^{1/q}\)	\(X \in\mathbf{R}^{n \times n}\)	positive	convex	None
norm(x) norm(x, 2)	\(\sqrt{\sum_{i}x_{i}^2 }\)	\(X \in\mathbf{R}^{n}\)	positive	convex	for \(x_{i} \geq 0\) for \(x_{i} \leq 0\)
norm(X, “fro”)	\(\sqrt{\sum_{ij}X_{ij}^2 }\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, 1)	\(\sum_{ij}\lvert X_{ij} \rvert\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, “inf”)	\(\max_{ij} \{\lvert X_{ij} \rvert\}\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, “nuc”)	\(\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	None
norm(X) norm(X, 2)	\(\sqrt{\lambda_{\max}\left(X^T X\right)}\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	None
pnorm(X, p) \(p \geq 1\) or `p = 'inf'`	\(\\|X\\|_p = \left(\sum_{ij} \|X_{ij}\|^p \right)^{1/p}\)	\(X \in \mathbf{R}^{m \times n}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
pnorm(X, p) \(p < 1\), \(p \neq 0\)	\(\\|X\\|_p = \left(\sum_{ij} X_{ij}^p \right)^{1/p}\)	\(X \in \mathbf{R}^{m \times n}_+\)	positive	concave	incr.
quad_form(x, P) constant \(P \in \mathbf{S}^n_+\)	\(x^T P x\)	\(x \in \mathbf{R}^n\)	positive	convex	for \(x_i \geq 0\) for \(x_i \leq 0\)
quad_form(x, P) constant \(P \in \mathbf{S}^n_-\)	\(x^T P x\)	\(x \in \mathbf{R}^n\)	negative	concave	for \(x_i \geq 0\) for \(x_i \leq 0\)
quad_form(c, X) constant \(c \in \mathbf{R}^n\)	\(c^T X c\)	\(X \in\mathbf{R}^{n \times n}\)	depends on c, X	affine	depends on c
quad_over_lin(X, y)	\(\left(\sum_{ij}X_{ij}^2\right)/y\)	\(x \in \mathbf{R}^n\) \(y > 0\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\) decr. in \(y\)
sum_entries(X)	\(\sum_{ij}X_{ij}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	affine	incr.
sum_largest(X, k) \(k = 1,2,\ldots\)	\(\text{sum of } k\text{ largest }X_{ij}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	convex	incr.
sum_smallest(X, k) \(k = 1,2,\ldots\)	\(\text{sum of } k\text{ smallest }X_{ij}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	concave	incr.
sum_squares(X)	\(\sum_{ij}X_{ij}^2\)	\(X \in\mathbf{R}^{m \times n}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
trace(X)	\(\mathrm{tr}\left(X \right)\)	\(X \in\mathbf{R}^{n \times n}\)	same as X	affine	incr.
tv(x)	\(\sum_{i}\|x_{i+1} - x_i\|\)	\(x \in \mathbf{R}^n\)	positive	convex	None
tv(X)	\(\sum_{ij}\left\\| \left[\begin{matrix} X_{i+1,j} - X_{ij} \\ X_{i,j+1} -X_{ij} \end{matrix}\right] \right\\|_2\)	\(X \in \mathbf{R}^{m \times n}\)	positive	convex	None
tv(X1,…,Xk)	\(\sum_{ij}\left\\| \left[\begin{matrix} X_{i+1,j}^{(1)} - X_{ij}^{(1)} \\ X_{i,j+1}^{(1)} -X_{ij}^{(1)} \\ \vdots \\ X_{i+1,j}^{(k)} - X_{ij}^{(k)} \\ X_{i,j+1}^{(k)} -X_{ij}^{(k)} \end{matrix}\right] \right\\|_2\)	\(X^{(i)} \in\mathbf{R}^{m \times n}\)	positive	convex	None

Function	Meaning	Domain	Sign	Curvature	Monotonicity
abs(x)	\(\lvert x \rvert\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
entr(x)	\(-x \log (x)\)	\(x > 0\)	unknown	concave	None
exp(x)	\(e^x\)	\(x \in \mathbf{R}\)	positive	convex	incr.
huber(x, M=1) \(M \geq 0\)	\(\begin{cases}x^2 &\|x\| \leq M \\2M\|x\| - M^2&\|x\| >M\end{cases}\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
inv_pos(x)	\(1/x\)	\(x > 0\)	positive	convex	decr.
kl_div(x, y)	\(x \log(x/y) - x + y\)	\(x > 0\) \(y > 0\)	positive	convex	None
log(x)	\(\log(x)\)	\(x > 0\)	unknown	concave	incr.
log1p(x)	\(\log(x+1)\)	\(x > -1\)	same as x	concave	incr.
logistic(x)	\(\log(1 + e^{x})\)	\(x \in \mathbf{R}\)	positive	convex	incr.
max_elemwise(x1, …, xk)	\(\max \left\{x_1, \ldots , x_k\right\}\)	\(x_i \in \mathbf{R}\)	\(\max(\mathrm{sign}(x_1))\)	convex	incr.
min_elemwise(x1, …, xk)	\(\min \left\{x_1, \ldots , x_k\right\}\)	\(x_i \in \mathbf{R}\)	\(\min(\mathrm{sign}(x_1))\)	concave	incr.
mul_elemwise(c, x) \(c \in \mathbf{R}\)	c*x	\(x \in\mathbf{R}\)	\(\mathrm{sign}(cx)\)	affine	depends on c
neg(x)	\(\max \left\{-x, 0 \right\}\)	\(x \in \mathbf{R}\)	positive	convex	decr.
pos(x)	\(\max \left\{x, 0 \right\}\)	\(x \in \mathbf{R}\)	positive	convex	incr.
power(x, 0)	\(1\)	\(x \in \mathbf{R}\)	positive	constant
power(x, 1)	\(x\)	\(x \in \mathbf{R}\)	same as x	affine	incr.
power(x, p) \(p = 2, 4, 8, \ldots\)	\(x^p\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
power(x, p) \(p < 0\)	\(x^p\)	\(x > 0\)	positive	convex	decr.
power(x, p) \(0 < p < 1\)	\(x^p\)	\(x \geq 0\)	positive	concave	incr.
power(x, p) \(p > 1,\ p \neq 2, 4, 8, \ldots\)	\(x^p\)	\(x \geq 0\)	positive	convex	incr.
scalene(x, alpha, beta) \(\text{alpha} \geq 0\) \(\text{beta} \geq 0\)	\(\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
sqrt(x)	\(\sqrt x\)	\(x \geq 0\)	positive	concave	incr.
square(x)	\(x^2\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)

Function	Meaning	Domain	Sign	Curvature	Monotonicity
bmat([[X11, …, X1q], …, [Xp1, …, Xpq]])	\(\left[\begin{matrix} X^{(1,1)} & \cdots & X^{(1,q)} \\ \vdots & & \vdots \\ X^{(p,1)} & \cdots & X^{(p,q)} \end{matrix}\right]\)	\(X^{(i,j)} \in\mathbf{R}^{m_i \times n_j}\)	\(\mathrm{sign}\left(\sum_{ij} X^{(i,j)}_{11}\right)\)	affine	incr.
conv(c, x) \(c\in\mathbf{R}^m\)	\(c*x\)	\(x\in \mathbf{R}^n\)	\(\mathrm{sign}\left(c_{1}x_{1}\right)\)	affine	depends on c
cumsum(X, axis=0)	cumulative sum along given axis.	\(X \in \mathbf{R}^{m \times n}\)	same as X	affine	incr.
diag(x)	\(\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]\)	\(x \in\mathbf{R}^{n}\)	same as x	affine	incr.
diag(X)	\(\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]\)	\(X \in\mathbf{R}^{n \times n}\)	same as X	affine	incr.
diff(X, k=1, axis=0) \(k \in 0,1,2,\ldots\)	kth order differences along given axis	\(X \in\mathbf{R}^{m \times n}\)	same as X	affine	incr.
hstack(X1, …, Xk)	\(\left[\begin{matrix}X^{(1)} \cdots X^{(k)}\end{matrix}\right]\)	\(X^{(i)} \in\mathbf{R}^{m \times n_i}\)	\(\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)\)	affine	incr.
kron(C, X) \(C\in\mathbf{R}^{p \times q}\)	\(\left[\begin{matrix}C_{11}X & \cdots & C_{1q}X \\ \vdots & & \vdots \\ C_{p1}X & \cdots & C_{pq}X \end{matrix}\right]\)	\(X \in\mathbf{R}^{m \times n}\)	\(\mathrm{sign}\left(C_{11}X_{11}\right)\)	affine	depends on C
reshape(X, n’, m’)	\(X' \in\mathbf{R}^{m' \times n'}\)	\(X \in\mathbf{R}^{m \times n}\) \(m'n' = mn\)	same as X	affine	incr.
vec(X)	\(x' \in\mathbf{R}^{mn}\)	\(X \in\mathbf{R}^{m \times n}\)	same as X	affine	incr.
vstack(X1, …, Xk)	\(\left[\begin{matrix}X^{(1)} \\ \vdots \\X^{(k)}\end{matrix}\right]\)	\(X^{(i)} \in\mathbf{R}^{m_i \times n}\)	\(\mathrm{sign}\left(\sum_i X^{(i)}_{11}\right)\)	affine	incr.