TheLazy.Dev
Portfolio GitHub

Blog

learning |machinelearning |beginners |tutorial

Mathematics for Machine Learning - Day 17

pourlehommes

Terra

Posted on July 25, 2024

Mathematics for Machine Learning - Day 17

Idiocy of learning fast

What an absolute clown. When it's just separate examples, it's working fine, but when combined I'm suddenly confused like a headless chicken. I need to stop finding shortcuts only to just re-learn it again the right way.

Well, another lesson learned. No use dwelling on yesterday's idiocy.

Basis Change.

It's combining a few sections we've discussed before into one. So far what I've known to be related directly are:

  1. Transformation Matrix
  2. Linear mapping

Which combined will become... Transformation mapping. I know, so original.

So what is it?

It's transforming matrices while taking into consideration linear mapping if the basis is changed.

So, with

Φ:V→W \Phi : V \to W

Consider 2 ordered basis of V and W.

B=(b1,…,bn),B~=(b~1,…,b~n)→VC=(c1,…,cn),C~=(c~1,…,c~n)→W B = (b_1, \dots, b_n), \tilde{B} = (\tilde{b}_1, \dots, \tilde{b}_n) \to V \\ C = (c_1, \dots, c_n), \tilde{C} = (\tilde{c}_1, \dots, \tilde{c}_n) \to W

Note:
If you forgot, here's what the above basis are for.

B→ Basis Change →B~C→ Basis Change →C~B→ Linear Mapping →CB~→ Linear Mapping →C~ B \to \text{ Basis Change } \to \tilde{B} \\ C \to \text{ Basis Change } \to \tilde{C} \\ B \to \text{ Linear Mapping } \to C \\ \tilde{B} \to \text{ Linear Mapping } \to \tilde{C} \\

With both basis change and linear mapping can be inversed. So anything can become anything, you just gotta believe in yourself.

Oh yeah, also

AΦ∈Rm×n and AΦ~∈Rm×n A_{\Phi} \in \reals^{m \times n} \text{ and } \tilde{A_{\Phi}} \in \reals^{m \times n}

are the transformation matrix for the basis change.

So what?

So, consider this transformation matrix:

A=[2112] A = \left[\begin{array}{cc} 2 & 1 \\ 1 & 2 \\ \end{array}\right]

with respect to the standard basis, if we define a new basis.

B=([11],[1−1]) B = ( \left[\begin{array}{c} 1 \\ 1 \\ \end{array}\right], \left[\begin{array}{c} 1 \\ -1 \\ \end{array}\right] )

we obtain the diagonal transformation matrix

A~=[3001] \tilde{A} = \left[\begin{array}{cc} 3 & 0 \\ 0 & 1 \\ \end{array}\right]

which is easier to work with than A.

Huh... How?

I hope that's what you're thinking right now, because I spent well over an hour knowing how the hell did this happen.

If you already know how... Please review to see if it's really correct or not :D I'm new in this.

Step 1: Transform the new basis into a matrix

B=([11],[1−1])=[111−1] B = ( \left[\begin{array}{c} 1 \\ 1 \\ \end{array}\right], \left[\begin{array}{c} 1 \\ -1 \\ \end{array}\right] ) = \left[\begin{array}{cc} 1 & 1 \\ 1 & -1 \\ \end{array}\right]

Step 2: Find Inverse B

B−1=1∥B∥adj(A)B−1=1−2[−1−1−11]B−1=[121212−12] B^{-1} = \frac{1}{\Vert{B}\Vert} adj(A) \\ B^{-1} = \frac{1}{-2} \left[\begin{array}{cc} -1 & -1 \\ -1 & 1 \\ \end{array}\right] \\ B^{-1} = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right]

Step 3: Calculate (Inverse B) AB

B−1AB=[121212−12][2112][111−1] B^{-1}AB = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right] \left[\begin{array}{cc} 2 & 1 \\ 1 & 2 \\ \end{array}\right] \left[\begin{array}{cc} 1 & 1 \\ 1 & -1 \\ \end{array}\right] \\

Calculate AB

B−1AB=[121212−12][333−1] B^{-1}AB = \left[\begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} \\ \end{array}\right] \left[\begin{array}{cc} 3 & 3 \\ 3 & -1 \\ \end{array}\right]

Calculate Inverse B to AB

B−1AB=[3001] B^{-1}AB = \left[\begin{array}{cc} 3 & 0 \\ 0 & 1 \\ \end{array}\right]

For some of you, this is enough.

You might already understand the steps and what does each step mean, to which I say, congratulations the other texts below aren't need for you so feel free to leave.

But honestly, I still don't get it :D.

How did I read through a bunch of sections and only now inverse is used again, what's happening here?. So I asked myself a second time.

Huh... How?

It all stems back to linear mapping, the bastard that's laughing maniacally feeling superior as I use an ineffective method on my journey to learning this.

Consider this:

T(v)std=PT(v)BT(v)std=Linear transformation on standard basisP=Change of Basis matrixT(v)B=Linear transformation on new basis T_{(v){std}} = P T{(v){B}} \\ T{(v){std}} = \text{Linear transformation on standard basis} \\ P = \text{Change of Basis matrix} \\ T{(v)_{B}} = \text{Linear transformation on new basis} \\

P is what we defined as B beforehand.

That's easy, then just inverse it to get the answer right?

T(v)B=P−1T(v)std T_{(v){B}} = P^{-1} T{(v)_{std}}

That's right. P is the B that we used on the example.

But what about the linear transformation T, what's that?

T(v)std=Astd[vstd]Astd=The matrix representation of T with respect to the standard basis[vstd]=Vectors with respect to the standard basis T_{(v){std}} = A{std} \left[v_{std}\right] \\ A_{std} = \text{The matrix representation of T with respect to the standard basis} \\ \left[v_{std}\right] = \text{Vectors with respect to the standard basis} \\

And since the vectors any vector inside of T on the standard basis, we need to map the vector to the new basis as well.

How?

[vstd]=P[vb] \left[v_{std}\right] = P \left[v_{b}\right] \\

Notice it's familiar with the T formula for changing basis right? that's because it's the same.

Add it to the formula

T(v)std=AstdP[vb] T_{(v){std}} = A{std} P \left[v_{b}\right]

Combine with the new basis linear transformation.

T(v)B=P−1AstdP[vb] T_{(v){B}} = P^{-1} A{std} P \left[v_{b}\right]

Boom. This gets the formula.

Sidenote:

I absolutely have no clue why some are able to be subscript but some aren't. I think there's something wrong with the notation or maybe there's a parameter that's messing it up. But either way, just incase, here's a clarification.

Astd=AstdT(v)std=T(v)std Astd = A_{std} \\ T(v)std = T_{(v)_{std}}

Acknowledgement

I can't overstate this: I'm truly grateful for this book being open-sourced for everyone. Many people will be able to learn and understand machine learning on a fundamental level. Whether changing careers, demystifying AI, or just learning in general, this book offers immense value even for fledgling composer such as myself. So, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, thank you for this book.

Source:
Axler, Sheldon. 2015. Linear Algebra Done Right. Springer
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge: Cambridge University Press.
https://mml-book.com

💖 💪 🙅 🚩
pourlehommes
Terra

Posted on July 25, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Mathematics for Machine Learning - Day 17
learning Mathematics for Machine Learning - Day 17

July 25, 2024

Mathematics for Machine Learning - Day 11
learning Mathematics for Machine Learning - Day 11

July 18, 2024

Mathematics for Machine Learning - Day 13
learning Mathematics for Machine Learning - Day 13

July 20, 2024

Mathematics for Machine Learning - Day 14
learning Mathematics for Machine Learning - Day 14

July 21, 2024

Mathematics for Machine Learning - Day 12
learning Mathematics for Machine Learning - Day 12

July 19, 2024