Norm equivalence

作者: jjx323 | 来源:发表于2020-02-28 13:12 被阅读0次

Norm equivalence
20181029norm
1.24三峡广场GRE全程班 1.24第1节课笔记大纲
2018-10-29 Norm
每日一词 norm
norm 作业练习
吴恩达深度学习笔记(52)-将 Batch Norm 拟合进神经
数学基础
pytorch api:torch.nn.utils.clip_
norm

Theorem: If $X$ is a finite-dimensional vector space over $\mathbb{F}$ , then any two norms on $X$ are equivalent.

Proof: Let $\mathcal{B} = \{e_1, \cdots, e_N\}$ be any basis for $X$ . Then, for any element of $X$ , e.g., $u\in X$ , we have $u=\sum_{i=1}^{N}u_i e_i$ with $u_i\in \mathbb{F}$ . Let $\|u\|_{\ell^{\infty}} := \sup_{1\leq i\leq N}|u_i|$ , and introduce
$D = \{u\in X : \|u\|_{\ell^{\infty}} = 1\} .$
Assume $\|\cdot\|$ be some norm on $X$ , then we just need to illustrate the equivalence of $\|\cdot\|$ and $\|\cdot\|_{\ell^{\infty}}$ . Obviously, we have the following estimate
$\|u\| = \left\| \sum_{i=1}^{N}u_i e_i \right\| \leq \sum_{i=1}^{N}|u_i|\|e_i \|\leq \sum_{i=1}^{N}\|e_i\| \sup_{1\leq i\leq N}|u_i| \leq C_1 \|u\|_{\ell^{\infty}} .$
For the reversed inequality, we decompose the proof into three steps.
Step 1: Prove $D$ is compact with respect to the norm $\|\cdot\|_{\ell^{\infty}}$ .
Let $\{ u_n \}_{n\in \mathbb{N}}$ be a sequence in $D$ , then we have $\sup_{1\leq i\leq N}|u_{ni}| = 1$ . That is to say, for each $n$ , there must be some $i\in\{ 1,2,\cdots, N \}$ such that $|u_{n,i}| = 1$ . Considering the index $i$ can only take finitely many values, there must be infinitely many $j$ such that $|u_{n_j, k}|=1$ for some fixed index $k$ . Since $\{ u_{n_j,1} \}_{j\in\mathbb{N}}$ are contained in $\{ c\in\mathbb{R} : |c|\leq 1 \}$ (closed set), we can select a converged subsequence. For the sequence selected just before, we can further select a converged subsequence when $i=2$ . Iteratively, we finally find a converged sequence. Because $|u_{n_j, k}| = 1$ , the subsequences are always contained in $D$ , which illustrates the compactness of $D$ .
Step 2: Prove $D$ is compact with respect to the norm $\|\cdot\|$ .
Let $\{u_n\}_{n\in\mathbb{N}}$ be a sequence in $D$ , then, according to Step 1, we can find a subsequence $\{u_{n_k}\}_{k\in\mathbb{N}}$ and $u$ in $D$ such that $\|u_{n_k} - u\|_{\ell^{\infty}}$ . Hence, we have
$\|u_{n_k} - u\| \leq C_1 \|u_{n_k} - u\|_{\ell^{\infty}} .$
It means that $\{u_{n_k}\}_{k\in\mathbb{N}}$ is a subsequence converges to $u\in X$ with respect to $\|\cdot\|$ , which shows the compactness of $D$ .
Step 3: The norm $\|\cdot\|$ is a continuous real valued function and $D$ is compact with respect to $\|\cdot\|$ . A real valued continuous function on a compact set must achieve a maximum and minimum on that set. Hence, there must exist two constants $m$ and $M$ such that $m\leq \|u\| \leq M$ for all $u\in X$ . Since $u\in D$ if and only if $\|u\|_{\ell^{\infty}} = 1$ , this implies that
$m\|u\|_{\ell^{\infty}} \leq \|u\| \leq M\|u\|_{\ell^{\infty}}, \quad \forall \, u\in X.$
Step 4: Show $m > 0$ . If we had $m=0$ , then this would imply that there is an $u\in X$ such that $\|u\|=0$ . So $u=0$ , and $\|u\|_{ell^{\infty}} = 0$ , which contradicting the fact that $x\in D$ . Hence, we have $m > 0$ .
Combining the statements from Step 1 to 4, we finally conclude the proof.

2020

Norm equivalence
Theorem: If is a finite-dimensional vector space over , ...
20181029norm
1）ADJ+norm accepted/estalished norm 接受的/公认的规范 V+the norm ...
1.24三峡广场GRE全程班 1.24第1节课笔记大纲
Text Completion & Sentence Equivalence Text Completion 1 ...
2018-10-29 Norm
1.verb+adj+norm: conform to+accepted/established norm of ...
每日一词 norm
搭配：参考朗文；be the norm; become the norm; social norms; cultu...
norm 作业练习
1. 常见搭配： be/become the norm 成为标准 above/below the norm 高于/...
吴恩达深度学习笔记(52)-将 Batch Norm 拟合进神经
Batch Norm 拟合进神经网络（Fitting Batch Norm into a neural netwo...
数学基础
模 norms包含多重含义L-0 norm: 查向量中的非零数L-1 norm: 曼哈顿距离L-2 norm: 欧...
pytorch api:torch.nn.utils.clip_
1. torch.nn.utils.clip_grad_norm_(parameters, max_norm, n...
norm
norm：the usual or normal situation, way of doing sth 常态，...