Obsidian Source: Notes / Complex Gradient
Summary
Pending synthesis from local Obsidian source.
Original source title: Complex Gradient
Extracted Preview
What Exactly am I trying to achieve?
Imagine a auto differentiation library, but it is made for keeping in mind the complex numbers as well. This can be a game-changer, as we can now have complex valued transformers, conv nets and what not. Imagine a robot that can feed in it's direction as a quaternion into a transformers, and we can predict or classify or do what not from it. It's honestly a game changer.
Kaprathy has a good tutorial in making a auto-diff library, and I'm mostly clear with all the things, and I just want to expand that to quaternions. Let's look at how does it work?
Differentiation of Complex Numbers
The derivative of a complex valued function is identical to that of a real valued function, but a point to note is that for complex valued functions, the relevant limit must exist independently of the direction from which we approach(remember the epsilon-delta definition of gradient)
- As we did notice from previous experience, if there exists one complex derivative, there exist infinitely many(You can always add 2$\pi$ in whatever direction, and you still get the same number.)
- The rules are quite similar for the complex case as well, just that it is with extra coefficients.
Let's see how that looks in code.
Integration Notes
- Source folder:
/home/yashs/Documents/Docs/Obsidian/Research-Notes - Local source:
/home/yashs/Documents/Docs/Obsidian/Research-Notes/Notes/Complex Gradient.md - Raw copy:
raw/obsidian/research-notes/Notes/Complex Gradient.md