1. Variable var a = 5; // declare global function myfunc(){ var a = 10; // override global { let a = 5; // local // here a = 5; } // here a = 10; } // only 'var' can override in the same block // var a = 10; // let a = 5; illegal statement 2. Constant const a = 5; // initial value must be assigned a = 3 // illegal statement // the atribution of constant object can be changed const b = ["Python", "Java", "C++"]; b[0] = "Ruby"; // legal statement // but the object itself can not be re-assigned b = ["C#", "VB"]; // illegal statment // more example const car = {type:"porsche", model:"911", color:"Black"}; car.
Prosody is underspecified by the text. (eg. rising intonation)
Add a multi-speaker model to Tacotron. (speaker embedding)
reference encoder: using a fixed-dimension embeddings.
Reference https://arxiv.org/pdf/1803.09047.pdf
1. Function Load Balance(负载均衡) and Reverse Proxy(反向代理).
https://www.runoob.com/linux/nginx-install-setup.html
2. Command nginx nginx -s reload # reload config file.
1. Create Project We can get access to Spring and download the project. Don’t forget to add ‘Spring Web’ dependency.
After that, we can run the project with IDEA. It’ll take around 20 minutes to download the necessary files. Then, we can follow the guidance from Quick Start to test the installation.
If you got trouble ‘Port 8080 was already in use’, you can create a file ‘/resource/application.yml’, and add the configuration.
1. Input There’re 3 inputs Q(query), K(key), V(value) for attention mechanism. If Q=K=V, we call it ‘self-attention’. Also, there’re several rules to calculate it.
$$Attention(Q,K,V) = Softmax(Linear([Q,K])) \cdot V$$
$$Attention(Q,K,V) = Softmax(sum(\tanh(Linear([Q,K])))) \cdot V$$
$$Attention(Q,K,V) = Softmax(\frac{Q \cdot K^T}{\sqrt{d_k}}) \cdot V$$
The ‘bmm’ is a special tensor multiply operation, batch matrices multiplication.
$$(b, n, m)*(b, m, p) \rightarrow (b, n, p)$$
Attention is usually used in seq2seq task.
import torch import torch.
1. RNN Hidden state at time t depends on the current input and the previous hidden state. Activation function’tanh’ add unlinearity and output values from -1 to 1.
$$h_t = \tanh(W_t \cdot [h_{t-1}, x_t]+b_t)$$
import torch import torch.nn as nn rnn = nn.RNN(input_size, hidden_size, num_layers) input1 = torch.randn(sequence_length, batch_size, input_size) h0 = torch.randn(num_layers*num_directions, batch_size, hidden_size) output, hn = rnn(input1, h0) RNN is useful in short sequence, but may encounter ‘gradients vanishing’ and ‘gradients explosion’ while processing long sequence.
1. Encoder $$one_hot \Rightarrow char_embedding \Rightarrow 3*Conv_layer \Rightarrow LSTM \Rightarrow Context$$
Char_level embeddings are vector representation of the original sentences. We can use conv_layers and LSTM to extract meanings of them.
2. Prenet $$[frame_t, batch, frame_dim] \Rightarrow 2*Linear(ReLU) \Rightarrow F_t$$
$$concat(F_t,~Context, F_{t-1}) \Rightarrow LSTM_Cell$$