quaternion

四元数的基础知识

$q=a+bi+cj+dk$​​

$ii=jj=kk=ijk=-1$​​

$i^2=j^2=k^2=-1$​​

$ij=k,jk=i,ki=j$​​

$ji=-k,kj=-i,ik=-j$​​

若四元数$q=[s,v]$​​,则$q^*=[s,-v]$​​

四元数$q$​​​的逆$q^{-1}=\frac{q^}{||q||^2}$​​​,当$q$​​​为单位四元数时,$q^{-1}=q^$​​​

$v'=qvq^*$​​,对任意单位四元数$q=[cos(\frac{1}{2}\theta),sin(\frac{1}{2}\theta)u]$​​

表示绕旋转轴$ u$​ 旋转$latex \theta$​度

四元数插值

我们希望找到,中间变换$q_t$​​,让初始变换$q_0$​​平滑过渡到最终变换$ q_1$​​

设旋转的该变量为$\Delta q$​​,旋转的该变量也对应一个旋转,$ q_1$​​ 可以看成旋转的复合
$$\Delta qq_0 =q_1
\ \Delta q=q_1 q^{-1}
\ \Delta q=q_1 q^
\ q_t=(q_1 q^
)^t$$

$ \Delta q$​​的旋转角度是$ q$​​和 q_1$的夹角$latex \theta$的二倍,即$$2\theta$​​​

Lerp

线性插值(Linear Interpolation)

$$
v_t=Lerp(v_0,v_1,t)=v_0+t(v_1-v_0)
\ v_t=(1-t)v_0+tv_1
$$
应用到单位四元数,得Lerp公式:
$$
q_t=Lerp(q_0,q_1,t)=q_0+t(q_1-q_0)
\ q_t=(1-t)q_0+tq_1
$$
$q_t$并不是单位四元数

image-20210721101241723

Nlerp

正规化线性插值(Normalized Linear Interpolation)

$q_t^0=\frac{q_t}{||q_t||}$​

存在问题:

旋转速度,先加速后减速,不能保证均匀的角速度

Slerp

对角度线性插值,又称球面线性插值(Spherical Linear Interpolation)
$$
\theta_t=(1-t)*0+t\theta=t\theta
\ v_t=\alpha v_0+\beta v_1
$$
左右两边分别点乘$ v_0$​​和$ v_1$​​
$$
v_0 \cdot v_t=v_0 \cdot(\alpha v_0+\beta v_1)
\ v_1 \cdot v_t=v_1 \cdot(\alpha v_0+\beta v_1)
$$

$$
\alpha=\frac{sin((1-t)\theta)}{sin(\theta)}
\ \beta=\frac{sin(t\theta)}{sin(\theta)}
$$
得Slerp公式
$$
q_t=Slerp(q_0,q_1,t)=\frac{sin((1-t)\theta)}{sin(\theta)}q_0+\frac{sin(t\theta)}{sin(\theta)}q_1
$$
其中$ \theta=cos^{-1}(q_0\cdot q_1)$​​

Slerp方法效率低,要注意$latex \theta$不能过小,否则会出现除$latex 0$​得错误

Squad

球面四边形插值(Spherical and quadrangle)

希望能以牺牲固定角速度为条件,让插值的曲线不仅是连续的,而且让它的以解甚至是高阶导数是连续的。

Squad采用一层二次插值,嵌套一层一次插值

设有一个向量序列$latex v_0,v_1,v_2,v3$
$$
v
{03}=Lerp(v_0,v3;t)
\ v
{12}=Lerp(v_1,v2;t)
$$
使用$latex 2t(1-t)$为参数,对$latex v
{03}$和$latex v{12}$作二次插值
$$
v
{0312}=Lerp(v{03},v{12};2t(1-t))
$$
image-20210721104725422
$$
Squad(q_0,q_1,q_2,q_3;t)=Slerp(Slerp(q_0,q_3;t),Slerp(q_1,q_2;t);2t(1-t))
$$
设控制点$latexsi $​和$latexs{i+1} $​​​​​
$$
Squad(q_i,si,s{i+1},q_{i+1};t)=Slerp(Slerp(qi,q{i+1};t),Slerp(si,s{i+1};t);2t(1-t))
$$
其中
$$
s_i=q_i \exp(- \frac{ln(q^iq{i-1})+ln(q^iq{i+1})}{4})
$$
image-20210721110036875

红色的点即为控制点

参考文章:

https://krasjet.github.io/quaternion/

Autograd

自动求导

import torch

# 设置requires_grad=True,开启 autograd 跟踪设置
x=torch.tensor([5.],requires_grad=True)
y=x**3

# 将代码块包装在 with torch.no_grad(): 中,
#来阻止 autograd 跟踪设置了 .requires_grad=True 的张量的历史记录
with torch.no_grad():
    z=y*2
print(x,y,z)
print(x.requires_grad)
print(y.requires_grad)
print(z.requires_grad)

# .grad_fn引用了创建tensor的function
print(x.grad_fn)
print(y.grad_fn)
print(z.grad_fn)
# 反向传播
y.backward()

print(x.grad)
print(y.grad)
print(z.grad)

image-20210713110613325

数列极限

求$\lim_{n \to \infty }nsin(2\pi en!)$​

解:

$e=1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!}+\frac{1}{(n+1)!}+o(\frac{1}{(n+1)!})$​

$=\lim_{n \to \infty }nsin(2\pi en!)$​

$=\lim_{n \to \infty }nsin(2\pi n!(1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!}+\frac{1}{(n+1)!}+o(\frac{1}{(n+1)!})))$​

$=\lim_{n \to \infty }nsin(2\pi N+\frac{2\pi}{n+1}+o(\frac{1}{n^2}))$​​

$=\lim_{n \to \infty }n(\frac{2\pi}{n+1}+o(\frac{1}{n^2}))$​

$=2\pi$​

Regression

Stock Market Forecast

Self-driving Car

Recommendation

Example Application

Estimating the Combat Power of a pokemon after evolution

Step 1: Model

) A set of function

) y = b + w * X$_{cp}$

) Linear model:

Step 2: Goodness of Function

) Loss function L:

)) input: a function, output: how bad it is
$$
L(f)=L(w,b)
\ \sum(y^n-(b+w*x_{cp}^{n}))^2
$$
))color: Value of Loss

Step 3: Best Function
$$
f^ = arg min L(f)
\w^
, b^*=arg min L(w,b)
$$

Solution: Gradient Descent

One parameter

Pick an initial value $w^0$

compute $\frac{dL}{dw}|w=w^0$
$$
w^1 \leftarrow w^0 - (Learning Rate)\frac{dL}{dw}|w=w^0
$$
two parameters

Pick an initial value

gradient

in linear regression, convex

How's the results?

Generalization

How can we do better?

Selecting another Model

$$
y = b +w1* x{cp}+ w2*(x{cp})^2
$$

$$
y = b +w1* x{cp}+ w2*(x{cp})^2 + w3*(x{cp})^3
$$

This is Overfitting

  • select suitable model

Back to step 1: Redesign the Model

  • $y = b + \sum w_ix_i$

Back to step 2: Regularization
$$
L=\sum(y^n - (b+ \sum w_iw_i))+ \lambda \sum(w_i)^2
$$
validation

Regression

Regression

Stock Market Forecast

Self-driving Car

Recommendation

Example Application

Estimating the Combat Power of a pokemon after evolution

Step 1: Model

) A set of function

) y = b + w * X$_{cp}$

) Linear model:

Step 2: Goodness of Function

) Loss function L:

)) input: a function, output: how bad it is
$$
L(f)=L(w,b)
\ \sum(y^n-(b+w*x_{cp}^{n}))^2
$$
))color: Value of Loss

Step 3: Best Function
$$
f^ = arg min L(f)
\w^
, b^*=arg min L(w,b)
$$

Solution: Gradient Descent

One parameter

Pick an initial value $w^0$

compute $\frac{dL}{dw}|w=w^0$
$$
w^1 \leftarrow w^0 - (Learning Rate)\frac{dL}{dw}|w=w^0
$$
two parameters

Pick an initial value

gradient

in linear regression, convex

How's the results?

Generalization

How can we do better?

Selecting another Model

$$
y = b +w1* x{cp}+ w2*(x{cp})^2
$$

$$
y = b +w1* x{cp}+ w2*(x{cp})^2 + w3*(x{cp})^3
$$

This is Overfitting

  • select suitable model

Back to step 1: Redesign the Model

  • $y = b + \sum w_ix_i$

Back to step 2: Regularization
$$
L=\sum(y^n - (b+ \sum w_iw_i))+ \lambda \sum(w_i)^2
$$
validation

机器怎么找函式

Speech Recognition

Image Recognition

Playing Go

Dialogue System

你想要找什么样的函式

1.Regression Classification

2.Binary Classification

3.Multi-class Classification

4.Generation

Supervised Learning

对资料做 label

Labeled Data

函式的Loss

Loss越小越好

Reinforcement Learning

机器自主学习

Unsupervised Learning

资料没有 label

机器怎么找出你想要的函式

给定函式寻找范围

Network Architecture

)RNN and CNN

函式寻找方法

Gradient Descent

前沿研究

Explainable AI

Adversarial Attack

Network Compression

Anomaly Detection

)机器如何知道自己不知道

Transfer Learning

Meta Learning

)=Learn to learn

Life-long Learning

)Continuous Learning

Python Tensor

Tensor创建

import torch
print(torch.__version__)
print(torch.cuda.is_available())
import numpy as np
# 从数组变成张量
a=np.array([[1,2,3],[4,5,6],[7,8,9]])
print("a=",a)
b=torch.from_numpy(a)
print("b=",b)
# 直接从列表变成张量
c=[[2,2,2],[2,2,2],[2,2,2]]
print("c=",c)
d=torch.tensor(c)
print("d=",d)
# 利用其它张量的形状大小进行创建
e=torch.ones_like(b)
print("e=",e)
f=torch.rand_like(b,dtype=torch.float)
print("f=",f)

# 分割线
print()
print("********************************************")
print()

# 常见的tensor创建方法
shape=(5,5)
zero_tensor=torch.zeros(3,3)
print("zero=",zero_tensor)
zero_tensor=torch.zeros(shape)
print("zero=",zero_tensor)

ones_tensor=torch.ones(shape)
print("ones_tensor=",ones_tensor)

rand_tensor=torch.tensor(shape)
print("rand_tensor=",rand_tensor)

tensor的属性

import torch

# torch的属性
# 形状,数据类型,储存的设备

a_tensor=torch.rand(3,4)
print("a_tensor=",a_tensor)

# 形状
print("shape of a_tensor :",a_tensor.shape)

# 数据类型
print("Datetype of a_tensor :",a_tensor.dtype)

# 储存设备
print("device of a_tensor :",a_tensor.device)

tensor的操作

tensor的转移

import torch

a_tensor=torch.rand(3,4)
print(a_tensor)
print("device of a_tensor :",a_tensor.device)
print(torch.cuda.is_available())

# tensor 转移到 gpu
if torch.cuda.is_available():
    a_tensor=a_tensor.to('cuda')
print("device of a_tensor :",a_tensor.device)

tensor 支持标准的numpy索引和切片

import torch
# tensor支持标准的numpy索引和切片

tensor=torch.ones(3,3)
print(tensor)
tensor[:,0]=0
print(tensor)

tensor 按行,按列合并

import torch
# 合并tensor

a_tensor=torch.ones(3,3)
b_tensor=torch.rand(3,3)
c_tensor=torch.cat([a_tensor,b_tensor])
print(b_tensor)
print(c_tensor)

# 支持两个维度
# 0是按行合并,1是按列合并

d_tensor=torch.cat([a_tensor,b_tensor],dim=0)
e_tensor=torch.cat([a_tensor,b_tensor],dim=1)
print(d_tensor)
print(e_tensor)

tensor对应元素相乘

import torch
a_tensor=torch.ones(3,3)
b_tensor=torch.rand(3,3)

# tensor对应元素相乘

c_tensor=torch.mul(a_tensor,b_tensor)
d_tensor=a_tensor*b_tensor
print("a_tensor = ",a_tensor)
print("b_tensor = ",b_tensor)
print("c_tensor = ",c_tensor)
print("d_tensor = ",d_tensor)

tensor做矩阵乘法

import torch
a_tensor=torch.ones(3,3)
b_tensor=torch.rand(3,3)

# tensor做矩阵乘法

c_tensor=torch.matmul(a_tensor,b_tensor)
d_tensor=a_tensor@b_tensor
print("a_tensor = ",a_tensor)
print("b_tensor = ",b_tensor)
print("c_tensor = ",c_tensor)
print("d_tensor = ",d_tensor)

tensor 原地操作

import torch

# tensor 原地操作
tensor = torch.ones(3,3)
print("tensor = ",tensor)
tensor.add_(tensor)
print("tensor = ",tensor)

tensor 转换到 numpy 数组

import torch

# tensor 转换到 numpy 数组
tensor=torch.ones(3,3)
arr=tensor.numpy()
print("tensor = ",tensor)
print("arr = ",arr)

# tensor 变化 numpy 数组也会变化
tensor.add_(2)
print("tensor = ",tensor)
print("arr = ",arr)

numpy 数组转化到 tensor

import torch
import numpy as np

# numpy 数组转化为 tensor
arr = np.ones((3,3))
tensor = torch.from_numpy(arr)

print("arr = ",arr)
print("tensor = ",tensor)

# numpy 数组变化 tensor 也会跟着变化
np.add(arr,1,out = arr)

print("arr = ",arr)
print("tensor = ",tensor)