作者：钟超  阿里集团大淘宝团队

 

 

 

 

 

 

 

 

 

 [01] https://web.stanford.edu/~jurafsky/slp3/3.pdf

[02] https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html

[03] 《自然语言处理：基于预训练模型的方法》车万翔 等著

[04] https://cs.stanford.edu/people/karpathy/convnetjs/

[05] https://arxiv.org/abs/1706.03762

[06] https://arxiv.org/abs/1512.03385

[07] https://github.com/Kyubyong/transformer/

[08] http://jalammar.github.io/illustrated-transformer/

[09] https://towardsdatascience.com/this-is-how-to-train-better-transformer-models-d54191299978

[10] 《自然语言处理实战：预训练模型应用及其产品化》安库·A·帕特尔 等著

[11] https://lilianweng.github.io/posts/2018-06-24-attention/

[12] https://github.com/lilianweng/transformer-tensorflow/

[13] 《基于深度学习的道路短期交通状态时空序列预测》崔建勋 著

[14] https://www.zhihu.com/question/325839123

[15] https://luweikxy.gitbook.io/machine-learning-notes/self-attention-and-transformer

[16] 《Python 深度学习（第 2 版）》弗朗索瓦·肖莱 著

[17] https://en.wikipedia.org/wiki/Attention_(machine_learning)

[18] https://zhuanlan.zhihu.com/p/410776234

[19] https://www.tensorflow.org/tensorboard/get_started

[20] https://paperswithcode.com/method/multi-head-attention

[21] https://zhuanlan.zhihu.com/p/48508221

[22] https://www.joshbelanich.com/self-attention-layer/

[23] https://learning.rasa.com/transformers/kvq/

[24] http://deeplearning.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/

[25] https://zhuanlan.zhihu.com/p/352898810

[26] https://towardsdatascience.com/beautifully-illustrated-nlp-models-from-rnn-to-transformer-80d69faf2109

[27] https://medium.com/analytics-vidhya/understanding-q-k-v-in-transformer-self-attention-9a5eddaa5960