Transformer Layers#
Module |
MAdds |
FLOPs |
Parameters |
Memory |
|---|---|---|---|---|
torch.nn.Transformer |
x |
x |
✓ |
x |
torch.nn.TransformerEncoder |
x |
x |
✓ |
x |
torch.nn.TransformerDecoder |
x |
x |
✓ |
x |
torch.nn.TransformerEncoderLayer |
x |
x |
✓ |
x |
torch.nn.TransformerDecoderLayer |
x |
x |
✓ |
x |
torch.nn.MultiheadAttention |
x |
x |
✓ |
x |