Papers › Convolutional Sequence to Sequence Learning

Convolutional Sequence to Sequence Learning

8 May 2017ICML 2017 8arXiv:1705.03122archive 2025-07-28

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent models, computations over all elements can be fully parallelized during training and optimization is easier since the number of non-linearities is fixed and independent of the input length. Our use of gated linear units eases gradient propagation and we equip each decoder layer with a separate attention module. We outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT'14 English-French translation at an order of magnitude faster speed, both on GPU and CPU.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

37 repositories listed; official and paper-mentioned ones first.

facebookresearch/fairseq officialmentioned in papermentioned on GitHubpytorchMITSyntology: not harvested report
CKPOON0619/Kaggle2Sigma mentioned on GitHubSyntology: not harvested report
DavidHarar/Siamese-Networks-for-name-nickname-similarity mentioned on GitHubSyntology: not harvested report
Deeksha96/Im2Latex mentioned on GitHubtfSyntology: not harvested report
EdinburghNLP/XSum mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to On Faithfulness and Factuality in Abstractive Summarization (arXiv:2005.00661; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
Helsinki-NLP/OpenNMT-py mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to OpenNMT: Neural Machine Translation Toolkit (arXiv:1805.11462; that paper's own run record: 20 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 1 violated, 18 with no contract checked; 0 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report
IBM/pytorch-seq2seq mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified); nothing checks that they implement this paper's method. report
Izecson/saml-nmt mentioned on GitHubmxnetSyntology: not harvested report
Izecson/sockeye-1.16.6 mentioned on GitHubtfSyntology: not harvested report
Kakoedlinnoeslovo/fairseq mentioned on GitHubtfSyntology: not harvested report
MiuGod0126/ConvS2S_Paddle mentioned on GitHubpaddleApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Cross-lingual Language Model Pretraining (arXiv:1901.07291; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report
Nick-Zhao-Engr/Machine-Translation mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (arXiv:1406.1078; that paper's own run record: 13 ran (of which 6 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 3 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report
OpenNMT/OpenNMT-py mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention's Gravitational Field:A Power-Law Interpretation of Positional Correlation (arXiv:2603.04805; that paper's own run record: 9 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report
YizhuLiu/sumlen mentioned on GitHubpytorchSyntology: not harvested report
ZurichNLP/sockeye mentioned on GitHubmxnetApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Domain Robustness in Neural Machine Translation (arXiv:1911.03109; that paper's own run record: 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
albarji/neurowriter mentioned on GitHubtfSyntology: not harvested report
alisterta/Persian-text-to-speech mentioned on GitHubtfSyntology: not harvested report
awslabs/sockeye mentioned on GitHubmxnetApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality (arXiv:2205.04022; that paper's own run record: 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report
bcmi220/esc4nmt mentioned on GitHubpytorchSyntology: not harvested report
butsugiri/shape mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Simple Recurrent Units for Highly Parallelizable Recurrence (arXiv:1709.02755; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
cove-adml/adml-anon mentioned on GitHubpytorchSyntology: not harvested report
facebookresearch/ParlAI mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning (arXiv:2212.07919; that paper's own run record: 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
lucylow/En_francais_si_vous_plait- mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to fairseq: A Fast, Extensible Toolkit for Sequence Modeling (arXiv:1904.01038; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified); nothing checks that they implement this paper's method. report
memray/OpenNMT-kpg-release mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases (arXiv:1810.05241; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
midobal/OpenNMT-py mentioned on GitHubpytorchMITSyntology: not harvested report
nrc-cnrc/sockeye-multisource mentioned on GitHubmxnetSyntology: not harvested report
oneTimePad/conv-nmt mentioned on GitHubtfSyntology: not harvested report
phanideepgampa/IM2LATEX mentioned on GitHubtfSyntology: not harvested report
richinkabra/CoVe-BCN mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learned in Translation: Contextualized Word Vectors (arXiv:1708.00107; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
shashiongithub/XSum mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks... (arXiv:1808.08745; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
siyuofzhou/CNNSeqToSeq mentioned on GitHubtorchSyntology: not harvested report
stefannatu/NLP-Translation-Models mentioned on GitHubtfSyntology: not harvested report
thinkwee/DPP_CNN_Summarization mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Neural Machine Translation of Rare Words with Subword Units (arXiv:1508.07909; that paper's own run record: 21 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 20 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report
umeiko/mindspore-seq2seq mentioned on GitHubmindsporeSyntology: not harvested report
xingniu/sockeye mentioned on GitHubmxnetSyntology: not harvested report
yinghao1019/NLP_and_DL_practice mentioned on GitHubpytorchSyntology: not harvested report
yinghao1019/NLP_and_DL_practice pytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Bangla Spelling Error CorrectionDecoderImage ClassificationMachine TranslationTranslation

2 archive task tags without a task page not shown.

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Bangla Spelling Error Correction DPCSpell-Bangla-SEC-Corpus ConvSeq2Seq Exact Match Accuracy 78.85% #3 of 4 Archive leaderboard report
Image Classification MNIST CNN Model by Som Accuracy 98.59 #51 of 81 Archive leaderboard report
Image Classification MNIST CNN Model by Som Percentage error 1.41 #51 of 81 Archive leaderboard report
Machine Translation IWSLT2015 English-German ConvS2S BLEU score 26.73 #6 of 8 Archive leaderboard report
Machine Translation IWSLT2015 German-English ConvS2S BLEU score 32.31 #6 of 15 Archive leaderboard report
Machine Translation WMT2014 English-French ConvS2S (ensemble) BLEU score 41.3 #25 of 57 Archive leaderboard report
Machine Translation WMT2014 English-French ConvS2S BLEU score 40.46 #33 of 57 Archive leaderboard report
Machine Translation WMT2014 English-French ConvS2S Hardware Burden 143G #33 of 57 Archive leaderboard report
Machine Translation WMT2014 English-German ConvS2S (ensemble) BLEU score 26.4 #58 of 91 Archive leaderboard report
Machine Translation WMT2014 English-German ConvS2S (ensemble) Hardware Burden 54G #58 of 91 Archive leaderboard report
Machine Translation WMT2014 English-German ConvS2S BLEU score 25.16 #70 of 91 Archive leaderboard report
Machine Translation WMT2014 English-German ConvS2S Hardware Burden 72G #70 of 91 Archive leaderboard report
Machine Translation WMT2016 English-Romanian ConvS2S BPE40k BLEU score 29.9 #7 of 21 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LSTMSigmoid ActivationTanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections