We have re-organized Chapter: NLP pretraining (https://t.co/gUKNmrySw7) and Chapter: NLP applications (https://t.co/I5QvkSPEVa), and added sections of BERT (model, data, pretraining, fine-tuning, application) and natural language inference (data, model).

In this episode, Tom Kwiatkowski and Michael Collins talk about Natural Questions, a benchmark for question answering research. We discuss how the dataset was collected to reflect naturally-occurring

Anything that looks like genuine understanding is just an illusion.

01

We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time. This effect is often avoided through careful…

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...) - guillaume-be/rust-bert

Incorporating context into word embeddings - as exemplified by BERT, ELMo, and GPT-2 - has proven to be a watershed idea in NLP. Replacing static vectors (e.g., word2vec) with contextualized word representations has led to significant improvements on virtually…

The Pushshift Telegram Dataset The dataset consists of three files: Accounts.ndjson: Provides data for 2.2M Telegram users that were active in the channels we crawled. Channels.ndjson: Provides data for 28K Telegram channels that we crawled. Messages.ndjson:…

This post deals with the math behind the AdaBoost Algorithm in a, hopefully, easy way

全唐詩電子全文，全文檢索、相關於全唐詩的討論及參考資料。有簡體字版、繁體字版、英文版本。

MPNet能够在引入预测单词之间依赖的情况下看到整个句子信息语

Transfer Learning in Natural Language Processing (NLP): Open questions, current trends, limits, and future directions. Slides: https://tinyurl.com/FutureOfNLP
A walk through interesting papers and research directions in late 2019/early-2020 on:
- model size…

本文选自《当代外语研究》2020年第2期

本文选自《当代外语研究》2020年第01期