1. Introduction 1.1 KL for two Gaussian distribution $$ KL(p,q) = \log\frac{\sigma_1}{\sigma_2} + \frac{\sigma_1^2+(\mu_1-\mu_2)^2}{2\sigma_2^2} - \frac{1}{2} $$
1.2 Reparameterization If we want to sample from $N(\mu_1,\sigma_1)$, we can first sample $z$ from $N(0,1)$ and calculate $\sigma*z+\mu$ which will help us to get partial gradients.
1.3 Single layer of VAE The variational lower bound, or the evidence lower bound(ELBO), can be derived as follows.
$$ \begin{aligned} p(x) &= \int_z p_\theta(x|z)p(z) \\ &= \int_z q_\phi(z|x)\frac{p_\theta(x|z)p(z)}{q_\phi(z|x)} \\ \log p(x) &= \log E_{z\sim q_\phi{z|x}}[\frac{p_\theta(x|z)p(z)}{q_\phi(z|x)}] \\ &\ge E_{z\sim q_\phi{z|x}}[\log \frac{p_\theta(x|z)p(z)}{q_\phi(z|x)}] \end{aligned} $$
1. Flow-Based Generatvie Model Give a datapoint $x$ and latent varible $z\sim p_{\theta}(z)$. The function $g_{\theta}(..)$ is invertible such that
$$ \begin{aligned} x&=g_{\theta}(z) \\ z&=f_{\theta}(x) =g_{\theta}^{-1}(x) \end{aligned} $$
The probability density function of the model can be written as:
$$ \begin{aligned} \log p_{\theta}(x) &= \log p_{\theta}(z) + \log | det(dz/dx) | \\ &= \log p_{\theta}(z) + \sum_{i=1}^{K}\log | det(dh_i/dh_{i-1}) | \end{aligned} $$
To simplify the calculation, we can choose transformations with Jacobian $dh_{i}/dh_{i-1}$ is a triangular matrix,
1. ELBO We define that $x$ is target and $z$ is hidden variable.
$$P(x)=\int_{z}P(x|z)P(z)dz$$
Since $P(x|z)$ is close to 0, we have to shrink the sample space of $z$. Supposed that $z \sim Q(z|x)$:
$$\begin{aligned} KL[Q(z|x) || P(z|x)] &= \Epsilon_{z\sim Q(z|x)}[\log Q(z|x) - \log P(z|x)] \\ KL[Q(z|x) || P(z|x)] &= \Epsilon_{z\sim Q(z|x)}[\log Q(z|x) - \log P(x|z) - \log P(z) + \log P(x)] \\ \log P(x) - D[Q(z|x),P(z|x)] &= \Epsilon_{z\sim Q(z|x)}[\log P(x|z)] - KL[P(z) || Q(z|x)] \\ \log P(x) &\ge \Epsilon_{z\sim Q(z|x)}[\log P(x|z)] - KL[P(z) || Q(z|x)] \end{aligned}$$
1. jQuery <!-- https://jquery.com/download/ --> <script src="{% static 'js/jquery-3.6.1.min.js' %}"></script> <script src="{% static 'plugins/bootstrap-3.4.1-dist/js/bootstrap.min.js' %}"></script> Bootstrap with jQuery will help us to create a lot of dynamic pages.
<!-- Button trigger modal --> <button type="button" class="btn btn-primary btn-lg" data-toggle="modal" data-target="#myModal"> delete </button> <!-- Modal --> <div class="modal fade" id="myModal" tabindex="-1" role="dialog" aria-labelledby="myModalLabel"> <div class="modal-dialog" role="document"> <div class="modal-content"> <div class="modal-header"> <button type="button" class="close" data-dismiss="modal" aria-label="Close"><span aria-hidden="true">×</span></button> <h4 class="modal-title" id="myModalLabel">Warning</h4> </div> <div class="modal-body"> Are you sure that you want to delete this article?
1. Install $f(x)=\lambda x$
def foo(): print("aa")
1. Quick Start <head> ... <!-- development --> <link rel="stylesheet" href="static/plugins/bootstrap-3.4.1/css/bootstrap.css"> <!-- production --> <link rel="stylesheet" href="static/plugins/bootstrap-3.4.1/css/bootstrap.min.css"> <style> .navbar{ border-radius: 0; } </style> </head> <body> <!-- button --> <input type="button" value="enter" class="btn btn-primary"/> <!-- col-xs --> <div class="col-xs-offset-2 col-xs-2" style="color:red;">1</div> <div class="col-xs-2" style="color:green;">2</div> <!-- col-sm col-md col-lg --> </body>
1. Intro 1.1 <div style="color:red;"> contents </div> 1.2 <head> <style> .c1{ color: red; } #c２{ color: green; } li{ /* global */ color: pink; } input[type='text']{ border: 1px solid red; } .v1[attr='123']{ ... } .mylist li{ /* children and grandchildren */ color: red; } .mydiv > a{ /* only children */ } </style> </head> ... <div class='c1'>contents</div> <div id='c2'>contents</div> <ul> <li> ccc </li> ... </ul> <input type="text" /> <input type="password" /> 1.