TensorFlow-Models/syntaxnet/g3doc/conll2017

CoNLL2017 Shared Task Instructions

We are pleased to provide a competitive baseline for the CoNLL2017 Shared Task on Dependency Parsing. Note that we are providing detailed tutorials to make it easier to use DRAGNN as a platform for improving upon the baselines.

Please see our paper for more technical details about the model.

Running the baselines

• Install SyntaxNet/DRAGNN following the install instructions.
• Download the models here
• Download the contest data and tools
• Run the baseline_eval.py to run the pre-trained tokenizer and evaluate on the dev set.

You should obtain the following results on the dev sets with gold segmentation. Note: Our segmenter does not split multi-word tokens, which may not play nice (yet) with the official evaluation script.

Language	UAS	LAS
Ancient_Greek-PROIEL	81.52	76.87
Ancient_Greek	70.96	65.13
Arabic	84.79	78.90
Basque	80.96	77.19
Bulgarian	91.33	86.77
Catalan	91.32	88.76
Chinese	77.56	71.96
Croatian	86.62	81.84
Czech-CAC	89.99	86.09
Czech-CLTT	78.25	73.70
Czech	89.55	85.23
Danish	84.69	81.36
Dutch-LassySmall	84.12	80.85
Dutch	86.68	81.91
English-LinES	82.43	78.46
English-ParTUT	83.55	79.00
English	87.60	84.20
Estonian	75.77	67.76
Finnish-FTB	87.54	83.70
Finnish	87.05	83.33
French-ParTUT	85.12	80.79
French-Sequoia	87.90	85.74
French	91.05	88.48
Galician-TreeGal	75.26	69.50
Galician	84.64	81.58
German	85.53	81.27
Gothic	81.79	74.99
Greek	86.99	84.23
Hebrew	87.79	82.18
Hindi	93.73	90.10
Hungarian	78.68	73.03
Indonesian	83.02	76.51
Irish	75.02	65.66
Italian-ParTUT	85.09	80.90
Italian	90.73	87.71
Japanese	95.33	93.99
Kazakh	28.09	7.87
Korean	81.21	76.78
Latin-ITTB	82.86	78.43
Latin-PROIEL	79.52	73.58
Latin	64.72	54.59
Latvian	76.17	70.55
Norwegian-Bokmaal	91.23	88.79
Norwegian-Nynorsk	89.32	86.67
Old_Church_Slavonic	84.96	79.65
Persian	87.70	83.98
Polish	91.32	86.83
Portuguese-BR	92.36	90.60
Portuguese	90.60	88.12
Romanian	89.41	83.00
Russian-SynTagRus	91.51	89.05
Russian	85.18	80.71
Slovak	88.08	82.64
Slovenian-SST	66.77	59.38
Slovenian	89.85	87.62
Spanish-AnCora	91.02	88.61
Spanish	90.32	87.16
Swedish-LinES	83.67	78.96
Swedish	82.45	78.75
Turkish	68.81	60.57
Ukrainian	72.19	62.79
Urdu	85.50	79.19
Uyghur	69.23	43.27
Vietnamese	65.18	55.61

Using DRAGNN for developing your own models

We hope that DRAGNN will be useful as a starting point for deep learning parsing methods. We've provided a few recipes for alternative baselines sprinkled through the tutorials and examples.