pytorch：nn与nn.functional的区别

1.两者的调用方式不同
调用nn.xxx时要先在里面传入超参数，然后再将数据以函数调用的方式输进nn.xxx里，例如：

inputs = torch.rand(64, 3, 244, 244)
conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)
out = conv(inputs)

而nn.functional.xxx则要同时输入数据和weight,bias等参数，例如：

weight = torch.rand(64,3,3,3)
bias = torch.rand(64) 
out = nn.functional.conv2d(inputs, weight, bias, padding=1)

2.nn.xxx能够放在nn.Sequential里，而nn.functional.xxx就不行
3.nn.functional.xxx需要自己定义weight,每次调用时都需要手动传入weight,而nn.xxx则不用，例如：
使用nn.xxx定义一个cnn:

class CNN(nn.Moudle)
    def __init__(self):
        super(CNN, self).__init__()
        
        self.cnn1 = nn.Conv2d(in_channels=1,  out_channels=16, kernel_size=5,padding=0)
        self.relu1 = nn.ReLU()
        self.maxpool1 = nn.MaxPool2d(kernel_size=2)
        
        self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5,  padding=0)
        self.relu2 = nn.ReLU()
        self.maxpool2 = nn.MaxPool2d(kernel_size=2)
        
        self.linear1 = nn.Linear(4 * 4 * 32, 10)
        
    def forward(self, x):
        x = x.view(x.size(0), -1)
        out = self.maxpool1(self.relu1(self.cnn1(x)))
        out = self.maxpool2(self.relu2(self.cnn2(out)))
        out = self.linear1(out.view(x.size(0), -1))
        return out

使用nn.functional.xxx定义一个与上面相同的cnn:

class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        
        self.cnn1_weight = nn.Parameter(torch.rand(16, 1, 5, 5))
        self.bias1_weight = nn.Parameter(torch.rand(16))
        
        self.cnn2_weight = nn.Parameter(torch.rand(32, 16, 5, 5))
        self.bias2_weight = nn.Parameter(torch.rand(32))
        
        self.linear1_weight = nn.Parameter(torch.rand(4 * 4 * 32, 10))
        self.bias3_weight = nn.Parameter(torch.rand(10))
        
    def forward(self, x):
        x = x.view(x.size(0), -1)
        out = F.conv2d(x, self.cnn1_weight, self.bias1_weight)
        out = F.conv2d(x, self.cnn2_weight, self.bias2_weight)
        out = F.linear(x, self.linear1_weight, self.bias3_weight)
        return out

4.关于dropout,推荐使用nn.xxx。因为一般情况下只有训练时才用dropout,在eval不需要dropout。使用nn.Dropout,在调用model.eval()后，模型的dropout层都关闭，但用nn.functional.dropout，在调用model.eval()后不会关闭dropout.

5.有一种情况用nn.functional.xxx会更好，即如果行为相同，但参数矩阵相同的两个layer共享参数，可直接多次调用nn的Module。但若行为不同，比如想让两个dilation不同但kernel相同的卷积层共享参数，就可用到nn.functional,例如：

import torch
import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()

        self.weight = nn.Parameter(torch.Tensor(10,10,3,3))
    
    def forward(self, x):
        x_1 = F.conv2d(x, self.weight,dilation=1, padding=1)
        x_2 = F.conv2d(x, self.weight,dilation=2, padding=2)
        return x_1 + x_2

建议：在构建模型框架时（nn.Module）可用nn.xxx，在训练模型时可用nn.functional.xxx。另外，如果涉及到参数计算的，那用nn.；若不需要涉及更新参数，只是一次性计算，那用nn.functional。