1. AlexNet

Compared with LeNet, it has some changes.

  • Add noise and avoid overfitting(data transform).
  • bigger and deeper
  • use dropout (normalization)
  • AvgPooling -> MaxPooling
  • Sigmoid -> ReLU

2. Code

net = nn.Sequential(
    nn.Conv2d(3,96,kernel_size=11,stride=4,padding=2),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2), # 
    nn.Conv2d(96,128*2,kernel_size=5,padding=2),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    nn.Conv2d(128*2,192*2,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(192*2,192*2,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(192*2,128*2,kernel_size=3,padding=1),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2), # 6*6*256
    nn.Flatten(),
    nn.Linear(6*6*256,2048*2),nn.ReLU(),nn.Dropout(p=0.5),
    nn.Linear(2048*2,2048*2),nn.ReLU(),nn.Dropout(p=0.5),
    nn.Linear(2048*2,1000),nn.ReLU(),
)

3. Q&A

  • CNN提取的特征只是针对最后的分类任务，对于人来说大部分难以理解，因此它的可解释性较差。
  • Last two 4096 Full Connected Layers is necessary.
  • A good name.