R&C Analysis-Integration of Complex functions

Let $\mu$ be a positive definite measure on arbitarary measurable space $X$. Let $L^{1}(\mu)$ be collection of all complex measurable functions $f$ on $X$ for which $${\int }_X|f|d\mu <\mathrm{\infty }$$ is called space of ``Lesbesgue integral functions''. $f$ measurable implies $|f|$ is measurable, hence above integral is defined. For understanding next definition, recall that a function $u$ can be split into positive $u^+=max\{u,0\}$ and $u^-=-max\{u,0\}$ parts.  
Definition If $f=u+iv$ for real measurable functions $u,v$ on $X$ and if $f \in L^{1}(\mu)$, we define $${\int }_{E}fd\mu ={\int }_E{u}^{+}d\mu -{\int }_E{u}^{-}d\mu +i{\int }_E{v}^{+}d\mu -i{\int }_E{v}^{-}d\mu$$ for every measurable set $E$.Know that $u^+,u^-,v^+,v^-$ are measurable. Then $\int_E f d\mu$ exists. Futhermore $u^+ \leq |u| \leq |f|$ for all $4$ parts of above integral. Hence, each one of them is finite. Clearly from above definition, $\int_E fd\mu$ is a complex number. Occasionally, it is desirable to define integral of $f$ with range $[-\infty,\infty]$ to be $${\int }_{E}fd\mu ={\int }_E{f}^{+}d\mu -{\int }_E{f}^{-}1d\mu$$ provided atleast one of the integrals on the right is finite. Thus LHS is a number between $[-\infty,\infty]$. Theorem 1.32 Suppose $f,g \in L^1(\mu)$ and $\alpha,\beta$ are complex numbers, then $\alpha f+\beta g \in L^1(\mu)$ and $${\int }_X(\alpha f+\beta g)=\alpha {\int }_{X}fd\mu +\beta {\int }_{X}gd\mu$$ \subsection{Proof} First we need to establish that $\alpha f+\beta g$ is measurable.Then,need to show that the integral is less than infinity - thus establishing that LHS of above belongs to esteemed Legesgue integrable functions set($L^1(\mu)$). If $f,g$ are complex measurable functions, $f+g$ and $fg$ are measurable functions. $\alpha f = f.f.\cdots(\alpha\ times)$ is measurable. Similarly $\beta g$ is measurable. The sum of measurable funcitons $\alpha f + \beta g$ is measurable. WLOG assume $o \leq f \leq g$, then $\int_E f d\mu \leq \int_E g d\mu$. Know, $$|\alpha f+\beta g|\le |\alpha ||f|+|\beta ||g|$$ This implies $${\int }_X|\alpha f+\beta g|d\mu \le {\int }_X|\alpha ||f|+{\int }_X|\beta ||g|=|\alpha |{\int }_X|f|d\mu +|\beta |{\int }_X|g|d\mu <\mathrm{\infty }$$ Thus $\alpha f + \beta g \in L^1(\mu)$. To prove $(4)$ we need to establish $${\int }_X(f+g)d\mu ={\int }_{X}fd\mu +{\int }_{x}gd\mu$$ and $${\int }_X\alpha fd\mu =\alpha {\int }_{X}fd\mu$$ Assume $h=f+g$ $$h^{+}-h^{-}=f^{+}-f^{-}+g^{+}-g^{-}\text{ implies }{h}^{+}+f^{-}+g^{-}=f^{+}+g^{+}+h^{-}$$ From Theorem $1.27$ know that if $f=\sum_{i=1}^\infty f_n(x)$ then $\int_X f \mu = \sum_{i=1}^n \int_X f_n d\mu$. Applying this theorem yields, $$\int h^{+}+\int f^{-}1+\int g^{-}=\int f^{+}+\int g^{+}+\int h^{-}$$ Since each of these integrals is finite, we can rearrange terms as we like. $$\int h^{+}-\int h^{-}=\int (h^{+}-h^{-})=\int (f^{+}-f^{-})+\int (g^{+}-g^{-})$$ Leading to $\int_X(f+g) d\mu = \int_X f d\mu + \int_X g d\mu$. To establish equation (8), the following was already proved earlier. $${\int }_X(\alpha f)d\mu =\alpha {\int }_{X}fd\mu \text{ when }\alpha \ge 0$$ All that's left is to show that equation (8) holds for $\alpha < 0$ and $\alpha=i$. $\alpha=0$ case: Notice $$-u^{+}=-max\{u,0\}=u^{-}$$ which means $${\int }_X(-1)fd\mu ={\int }_X(-1)(u+iv)d\mu ={\int }_X(-u-iv)d\mu =(-1){\int }_{X}fd\mu$$ $\alpha=i$ case: $$\int (if)=\int i(u+iv)=\int (iu-v)=-\int v+i\int u=i\int (u+iv)=i\int f$$ Proved for all $\alpha$ less than $0$ and for $i$. This shows, $${\int }_X\alpha fd\mu =\alpha {\int }_{X}fd\mu$$

Machline Learning-Kernel Smoothing methods

Chapter 5:

Kernel Smooth Methods:

The set up is as follows:

You have input data - this is called matrix X.

Dimension of this matrix are N × p where N is number of rows. Here each row corresponds to a sample from your experiment.

ie/ Inputs. p is number of features.

Your output is collected into a matrix called y. Most of the time y is a N × 1

Typical linear regression is expressed as y = Xβ where βare called coeff of linear regression.

For example you have a function y = β₀ + β₁x

These functions are linear in β. For example y = β₀ + β₁x + β₂x² is still a linear function.

If output is a real value, then it is called regression. And if output is categorical variable - ex: Obese/NonObese. We use logistic regression and its variations.

Key term here is “localization”. Idea is fit a simple model at each of the query points and from there infer a overall function f(X)

This locatization is achieved via using of “Kernels”. Basic set up and terminology is - kernel is K_λ(x₀, x) where x₀is query point and xis any arbitary point. λwhich is size of neighborhood,

In these models λis a parameter.

Simplest way to understand one dim Kernels.

One simple is to select a neighborhood λand simply average all the distances from your query point x₀to all other points within λdistance or ball. Other way is to simply fit a linear function in each neighborhood. Here again you loose continity. This lack of continuity gets resolved using Nadara-Watson average.

f(x₀) = (∑_{{i = 1}^N}K_λ(x₀, x_i)y_i)/(∑_{{i = 1}^N}K_λ(x₀, x_i))

epinichekov kernel.

K_λ(x₀, x_i) = D((|x₀ − x_i|)/(λ))

D(t) = (3)/(4)(1 − t²) if |t| < 1

else D(t) = 0

These algorithms have issues at Boundaries:

At boundaries local cluster or neighborhood of points may generate a curve that takes off in different direction compared with the direction overall function takes.

This leads to poor predictability.

We need to tackle this.

To tackle this - as a start we use linear regression for these points.

Linear regression is done for each cluster of points or neighborhood, but applied only to x₀your query point at Boundary.

min∑^N_i = 1K_λ(x₀, x_i)[y_i − α(x₀) − β(x₀)x_i]²

Then your end function or boundary function is simply f(x₀) = α(x₀) + β(x₀)x₀

Defune b(x)^T = (1, x) and define B as N × 2 and we define W(x₀) which is a diagnol weights N × N

f(x₀) = b(x₀)^T(B^TW(x₀)B)^{ − 1)B^Ty = ∑^N_i = 1l_i(x₀)y_i

where l_i(x₀) are weights.

R&C Analysis-Th 1.29

Suppose $f:X\to [0,\mathrm{\infty }]$ is measurable, and $$\phi (E)={\int }_{E}fd\mu [E\in \mathcal{R}]$$ Then $\phi$ is a measure on $\mathcal{R}$ $${\int }_{X}gd\phi ={\int }_{X}gfd\mu$$ for every measurable function $g$ on $X$ with range in $[0,\mathrm{\infty }]$.

In this set up, as we tour through different sets of sigma algebra $\mathcal{R}$, the integral of measurable function generates a measure.

To prove Measure, need to show

Let $E_1,E_2,\cdots$ be disjoint members of $\mathcal{R}$ whose union is $E$. Then, $${\chi }_{E}f=\sum _{i=1}^{\mathrm{\infty }}{\chi }_{E_i}f$$ and that $$\phi (E)={\int }_X{\chi }_{E}f\mu \phi (E_j)={\int }_X{\chi }_{E_j}fd\mu$$ Then, $$\sum _{i=1}^{\mathrm{\infty }}\phi (E_i)=\sum _{i=1}^{\mathrm{\infty }}{\int }_X{\chi }_{E_i}fd\mu$$ Using previous theorem on summation of integrals, $$\begin{array}{r}\sum _{i=1}^{\mathrm{\infty }}{\int }_X{\chi }_{E_i}fd\mu ={\int }_X\sum _{i=1}^{\mathrm{\infty }}{\chi }_{E_i}fd\mu \\ ={\int }_X{\chi }_{E}fd\mu \\ =\phi (E)\end{array}$$ Thus $\sum _{i=1}^{\mathrm{\infty }}\phi (E_i)=\phi (E)$ establishing countable additive property. Since $\varphi \in \mathcal{R}$, and $\phi (\varphi )=0$ the finiteness property of atleast one of the sets in sigma algebra is satisfied. This shows $\phi$ is a measure. Each $\chi$ in above equations is a simple function. Setting $g=\chi$ If we set $h={\chi }_E$, for any simple measurable function. then $$\begin{array}{r}hf=\sum _{i=1}^{\mathrm{\infty }}{h}_{i}f\\ \text{ and }\\ \phi (E)={\int }_{E}hd\mu \\ \phi (E_i)={\int }_{E_i}{h}_{i}fd\mu \\ \text{ then }\phi (E)\text{ is a measure. }\\ {\int }_{X}hd\mu ={\int }_{X}fgd\mu \end{array}$$ Assume, $$0\le g_1(x)\le g_2(x)\le \cdots \le g(x)$$ with $li{m}_{n\to \mathrm{\infty }}{g}_i(x)=g(x)$. for each $i$ the following is true $${\int }_X{g}_{i}d\mu ={\int }_{X}f{g}_{i}d\mu$$

from monotone conv theorem, LHS is $$li{m}_{n\to \mathrm{\infty }}{\int }_X{g}_{i}d\phi ={\int }_{X}gd\phi$$ as $f$ is positive definite, $$0\le g_1(x)f(x)\le g_2(x)f(x)\cdots \le g(x)f(x)$$ with ${\int }_X{g}_{n}fd\mu \to {\int }_{X}gfd\mu$.

Hence, the equation $${\int }_{X}gd\phi ={\int }_{X}gfd\mu$$ holds.