uu.seUppsala University Publications
Change search
Link to record
Permanent link

Direct link
BETA
Nivre, Joakim
Publications (10 of 183) Show all publications
de Marneffe, M.-C. & Nivre, J. (2019). Dependency Grammar. Annual review of linguistics, 5, 197-218
Open this publication in new window or tab >>Dependency Grammar
2019 (English)In: Annual review of linguistics, E-ISSN 2333-9691, Vol. 5, p. 197-218Article in journal (Refereed) Published
Abstract [en]

Dependency grammar is a descriptive and theoretical tradition in linguistics that can be traced back to antiquity. It has long been influential in the European linguistics tradition and has more recently become a mainstream approach to representing syntactic and semantic structure in natural language processing. In this review, we introduce the basic theoretical assumptions of dependency grammar and review some key aspects in which different dependency frameworks agree or disagree. We also discuss advantages and disadvantages of dependency representations and introduce Universal Dependencies, a framework for multilingual dependency-based morphosyntactic annotation that has been applied to more than 60 languages.

Place, publisher, year, edition, pages
ANNUAL REVIEWS, 2019
Keywords
dependency grammar, dependency frameworks, dependency parsing, Universal Dependencies
National Category
General Language Studies and Linguistics
Identifiers
urn:nbn:se:uu:diva-381531 (URN)10.1146/annurev-linguistics-011718-011842 (DOI)000460289100010 ()
Available from: 2019-04-11 Created: 2019-04-11 Last updated: 2019-04-11Bibliographically approved
Basirat, A., de Lhoneux, M., Kulmizev, A., Kurfal, M., Nivre, J. & Östling, R. (2019). Polyglot Parsing for One Thousand and One Languages (And Then Some). In: : . Paper presented at First workshop on Typology for Polyglot NLP, Florence, Italy, August 1 2019.
Open this publication in new window or tab >>Polyglot Parsing for One Thousand and One Languages (And Then Some)
Show others...
2019 (English)Conference paper, Poster (with or without abstract) (Other academic)
National Category
General Language Studies and Linguistics
Identifiers
urn:nbn:se:uu:diva-392156 (URN)
Conference
First workshop on Typology for Polyglot NLP, Florence, Italy, August 1 2019
Available from: 2019-08-29 Created: 2019-08-29 Last updated: 2019-08-30Bibliographically approved
Basirat, A. & Nivre, J. (2019). Real-valued syntactic word vectors. Journal of experimental and theoretical artificial intelligence (Print)
Open this publication in new window or tab >>Real-valued syntactic word vectors
2019 (English)In: Journal of experimental and theoretical artificial intelligence (Print), ISSN 0952-813X, E-ISSN 1362-3079Article in journal (Refereed) Published
Abstract [en]

We introduce a word embedding method that generates a set of real-valued word vectors from a distributional semantic space. The semantic space is built with a set of context units (words) which are selected by an entropy-based feature selection approach with respect to the certainty involved in their contextual environments. We show that the most predictive context of a target word is its preceding word. An adaptive transformation function is also introduced that reshapes the data distribution to make it suitable for dimensionality reduction techniques. The final low-dimensional word vectors are formed by the singular vectors of a matrix of transformed data. We show that the resulting word vectors are as good as other sets of word vectors generated with popular word embedding methods.

Keywords
Word embeddings, context selection, transformation, dependency parsing, singular value decomposition, entropy
National Category
Languages and Literature General Language Studies and Linguistics Computer Systems
Identifiers
urn:nbn:se:uu:diva-392095 (URN)10.1080/0952813X.2019.1653385 (DOI)
Available from: 2019-08-29 Created: 2019-08-29 Last updated: 2019-08-29Bibliographically approved
Smith, A., Bohnet, B., de Lhoneux, M., Nivre, J., Shao, Y. & Stymne, S. (2018). 82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Models. In: Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Paper presented at Conference on Computational Natural Language Learning (CoNLL),October 31 - November 1, 2018 Brussels, Belgium (pp. 113-123).
Open this publication in new window or tab >>82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Models
Show others...
2018 (English)In: Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, 2018, p. 113-123Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-371246 (URN)
Conference
Conference on Computational Natural Language Learning (CoNLL),October 31 - November 1, 2018 Brussels, Belgium
Available from: 2018-12-19 Created: 2018-12-19 Last updated: 2019-03-06Bibliographically approved
Tang, G., Sennrich, R. & Nivre, J. (2018). An analysis of Attention Mechanism: The Case of Word Sense Disambiguation in Neural Machine Translation. In: Proceedings of the Third Conference on Machine Translation: . Paper presented at Third Conference on Machine Translation, October 31 — November 1, 2018, Brussels, Belgium (pp. 26-35).
Open this publication in new window or tab >>An analysis of Attention Mechanism: The Case of Word Sense Disambiguation in Neural Machine Translation
2018 (English)In: Proceedings of the Third Conference on Machine Translation, 2018, p. 26-35Conference paper, Published paper (Refereed)
Abstract [en]

Recent work has shown that the encoder-decoder attention mechanisms in neural ma-chine translation (NMT) are different from theword alignment in statistical machine trans-lation.In this paper, we focus on analyz-ing encoder-decoder attention mechanisms, inthe case of word sense disambiguation (WSD)in NMT models. We hypothesize that atten-tion mechanisms pay more attention to contexttokens when translating ambiguous words.We explore the attention distribution patternswhen translating ambiguous nouns. Counter-intuitively, we find that attention mechanismsare likely to distribute more attention to theambiguous noun itself rather than context to-kens, in comparison to other nouns. We con-clude that attention is not the main mecha-nism used by NMT models to incorporate con-textual information for WSD. The experimen-tal results suggest that NMT models learn toencode contextual information necessary forWSD in the encoder hidden states. For the at-tention mechanism in Transformer models, wereveal that the first few layers gradually learnto “align” source and target tokens and the lastfew layers learn to extract features from the re-lated but unaligned context tokens

National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-369712 (URN)
Conference
Third Conference on Machine Translation, October 31 — November 1, 2018, Brussels, Belgium
Available from: 2018-12-17 Created: 2018-12-17 Last updated: 2019-03-06Bibliographically approved
Tang, G., Cap, F., Pettersson, E. & Nivre, J. (2018). An evaluation of neural machine translation models on historical spelling normalization. In: Proceedings of the 27th International Conference on Computational Linguistics: . Paper presented at COLING 2018 (pp. 1320-1331).
Open this publication in new window or tab >>An evaluation of neural machine translation models on historical spelling normalization
2018 (English)In: Proceedings of the 27th International Conference on Computational Linguistics, 2018, p. 1320-1331Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-369710 (URN)
Conference
COLING 2018
Available from: 2018-12-17 Created: 2018-12-17 Last updated: 2018-12-17
Smith, A., de Lhoneux, M., Stymne, S. & Nivre, J. (2018). An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: . Paper presented at The 2018 Conference on Empirical Methods in Natural Language Processing, October 31–November 4 Brussels, Belgium (pp. 2711-2720).
Open this publication in new window or tab >>An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing
2018 (English)In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, p. 2711-2720Conference paper, Published paper (Refereed)
Abstract [en]

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2711–2720Brussels, Belgium, October 31 - November 4, 2018.c©2018 Association for Computational Linguistics2711An Investigation of the Interactions Between Pre-Trained WordEmbeddings, Character Models and POS Tags in Dependency ParsingAaron Smith Miryam de Lhoneux Sara Stymne Joakim NivreDepartment of Linguistics and Philology, Uppsala UniversityAbstractWe provide a comprehensive analysis of theinteractions between pre-trained word embed-dings, character models and POS tags in atransition-based dependency parser.Whileprevious studies have shown POS informationto be less important in the presence of char-acter models, we show that in fact there arecomplex interactions between all three tech-niques. In isolation each produces large im-provements over a baseline system using ran-domly initialised word embeddings only, butcombining them quickly leads to diminishingreturns. We categorise words by frequency,POS tag and language in order to systemati-cally investigate how each of the techniquesaffects parsing quality. For many word cat-egories, applying any two of the three tech-niques is almost as good as the full combinedsystem. Character models tend to be more im-portant for low-frequency open-class words,especially in morphologically rich languages,while POS tags can help disambiguate high-frequency function words. We also show thatlarge character embedding sizes help even forlanguages with small character sets, especiallyin morphologically rich languages.

National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics; Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-371245 (URN)
Conference
The 2018 Conference on Empirical Methods in Natural Language Processing, October 31–November 4 Brussels, Belgium
Available from: 2018-12-19 Created: 2018-12-19 Last updated: 2019-03-06Bibliographically approved
Nivre, J., Marongiu, P., Ginter, F., Kanerva, J., Montemagni, S., Schuster, S. & Simi, M. (2018). Enhancing Universal Dependency Treebanks: A Case Study. In: Proceedings of the Second Workshop on Universal Dependencies (UDW 2018): . Paper presented at Second Workshop on Universal Dependencies (UDW 2018), 1st November, Brussels, Belgium (pp. 102-107).
Open this publication in new window or tab >>Enhancing Universal Dependency Treebanks: A Case Study
Show others...
2018 (English)In: Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), 2018, p. 102-107Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-371249 (URN)
Conference
Second Workshop on Universal Dependencies (UDW 2018), 1st November, Brussels, Belgium
Available from: 2018-12-19 Created: 2018-12-19 Last updated: 2019-03-07Bibliographically approved
Bouma, G., Hajič, J., Haug, D., Nivre, J., Solberg, P. E. & Øvrelid, L. (2018). Expletives in Universal Dependency Treebanks. In: Proceedings of the Second Workshop on Universal Dependencies (UDW 2018): . Paper presented at Second Workshop on Universal Dependencies, 2018, 1st November, Brussels, Belgium (UDW 2018) (pp. 18-26).
Open this publication in new window or tab >>Expletives in Universal Dependency Treebanks
Show others...
2018 (English)In: Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), 2018, p. 18-26Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-371248 (URN)
Conference
Second Workshop on Universal Dependencies, 2018, 1st November, Brussels, Belgium (UDW 2018)
Available from: 2018-12-19 Created: 2018-12-19 Last updated: 2019-03-07Bibliographically approved
Stymne, S., de Lhoneux, M., Smith, A. & Nivre, J. (2018). Parser Training with Heterogeneous Treebanks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers): . Paper presented at The 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, July 15 - 20, 2018. (pp. 619-625). Association for Computational Linguistics
Open this publication in new window or tab >>Parser Training with Heterogeneous Treebanks
2018 (English)In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, 2018, p. 619-625Conference paper, Published paper (Refereed)
Abstract [en]

How to make the most of multiple heterogeneous treebanks when training a monolingual dependency parser is an open question. We start by investigating previouslysuggested, but little evaluated, strategiesfor exploiting multiple treebanks based onconcatenating training sets, with or without fine-tuning. We go on to propose anew method based on treebank embeddings. We perform experiments for severallanguages and show that in many casesfine-tuning and treebank embeddings leadto substantial improvements over singletreebanks or concatenation, with averagegains of 2.0–3.5 LAS points. We arguethat treebank embeddings should be preferred due to their conceptual simplicity,flexibility and extensibility.

Place, publisher, year, edition, pages
Association for Computational Linguistics, 2018
National Category
Language Technology (Computational Linguistics)
Research subject
Computational Linguistics
Identifiers
urn:nbn:se:uu:diva-362215 (URN)10.18653/v1/P18-2098 (DOI)000493913100098 ()978-1-948087-34-6 (ISBN)
Conference
The 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, July 15 - 20, 2018.
Funder
Swedish Research Council, P2016-01817
Available from: 2018-10-02 Created: 2018-10-02 Last updated: 2019-12-06Bibliographically approved
Organisations

Search in DiVA

Show all publications