TreeTagger - a language independent part-of-speech tagger

The TreeTagger is a tool for annotating text with part-of-speech and lemma information. It was developed by Helmut Schmid in the TC project at the Institute for Computational Linguistics of the University of Stuttgart. The TreeTagger has been successfully used to tag German, English, French, Italian, Dutch, Spanish, Bulgarian, Russian, Portuguese, Galician, Chinese, Swahili, Slovak, Slovenian, Latin, Estonian, Polish and old French texts and is adaptable to other languages if a lexicon and a manually tagged training corpus are available.

Sample output:

The TreeTagger can also be used as a chunker for English, German, French, and Spanish.

The tagger is described in the following two papers:

• Helmut Schmid (1995): Improvements in Part-of-Speech Tagging with an Application to German. Proceedings of the ACL SIGDAT-Workshop. Dublin, Ireland.

• Helmut Schmid (1994): Probabilistic Part-of-Speech Tagging Using Decision Trees. Proceedings of International Conference on New Methods in Language Processing, Manchester, UK.

Download

This software is freely available for research, education and evaluation.

Please read the license terms, before you download the software! By downloading the software, you agree to the terms stated there.

The following steps are necessary to install the TreeTagger (see below for the Windows version). Download the files by right-clicking on the link. Then select "save file as". All files should be stored in the same directory.

1. Download the tagger package for your system (PC-Linux, Mac OS-X (Intel-CPU), PC-Linux (version for older kernels)).
2. Download the tagging scripts into the same directory.
3. Download the installation script install-tagger.sh.
4. Download the parameter files for the languages you want to process.
5. Open a terminal window and run the installation script in the directory where you have downloaded the files:
sh install-tagger.sh
6. Make a test, e.g.
echo 'Hello world!' | cmd/tree-tagger-english
or
echo 'Das ist ein Test.' | cmd/tagger-chunker-german

Parameter files

• Bulgarian parameter file (gzip compressed, UTF-8, tagset, trained on the Bulgarian Treebank)
• A Chinese parameter file and tokenizer created by Serge Sharoff are available here
• Dutch parameter file (UTF8) (gzip compressed, tagset)
• Another Dutch parameter file (gzip compressed, UTF8, trained on the Eindhoven corpus, tagset documentation)
• English parameter file (gzip compressed, UTF8, tagset)
• Estonian parameter file (UTF8) (gzip compressed, tagset documentation)
• Finnish parameter file (UTF8) trained on data provided by Joshua Waxman (list of tags used).
• French parameter file (UTF-8) (gzip compressed, tagset documentation)
• A parameter file for spoken French texts can be found here
• Galician parameter file (UTF8) (gzip compressed, UTF8, tagset documentation)
• German parameter file (UTF-8) (gzip compressed, UTF-8, tagset documentation)
• Italian parameter file (UTF-8) (gzip compressed, tagset documentation)
• Marco Baroni's Italian parameter file (gzip compressed, Latin1, tagset documentation)
• Latin parameter file (gzip compressed, tagset info in Italian)
The corpus and lexicon for training the Latin parameter file have been compiled by Gabriele Brandolini from various resources
• Another Latin parameter file (gzip compressed, tagset info) which has been trained on the Index Thomisticus Treebank which was kindly provided by Marco Passarotti.
• Mongolian parameter file (gzip compressed, ???)
created from a small Mongolian corpus by Khuder Altangerel.
• Polish parameter file (UTF8) trained on the Polish National Corpus (tagset description).
• Portuguese parameter file (UTF8) provided by Pablo Gamallo (tagset description).
• Portuguese parameter file (UTF8) with fine-grained tagset provided by Pablo Gamallo (tagset description).
• Russian parameter file (UTF8) (gzip compressed, UTF8, tagset trained on a corpus created by Serge Sharoff)
• Slovak parameter file (UTF-8) (gzip compressed)
The Slovak parameter file was trained on the Slovak National Corpus. The tagset was simplified.
• Slovak parameter file (UTF-8, full tags) (gzip compressed)
The Slovak parameter file was trained on the Slovak National Corpus. The tagset was not simplified (just a marker for typos was removed). Many thanks to Vladimir Benko for suggesting to train on the full tagset and also for his bug reports.
• Slovenian parameter file (UTF-8) (gzip compressed)
The Slovaenian parameter file was trained on the ssj500k 1.3 training corpus. The tagset is documented here.
• Spanish parameter file (UTF8) (gzip compressed, UTF8, tagset documentation)
• Swahili parameter file (gzip compressed)
The Swahili parameter file was trained on the Helsinki Corpus of Swahili (HCS) and uses a simplified version of the HCS tagset. The HCS was created by Prof. Arvi Hurskainen by means of his Swahili Language Manager (SALAMA) which uses Lingsoft's TWOL compiler for constructing morphological analysers and Connexor's CG2 parser for syntactic disambiguation. The creation of the parameter file was joint work with Gabriele Brandolini.

• English chunker parameter file (gzip compressed, UTF8, tagset info)
  Note: The English tagger parameter file is needed, as well.
• French chunker parameter file (UTF-8) (gzip compressed)
  The chunker was trained on the French treebank whose annotation guidelines are documented here.
  Note: The French tagger parameter file is needed, as well.
• German chunker parameter file (UTF8) (gzip compressed, UTF8, tagset info)
  Note: The German tagger parameter file is needed, as well.
• Spanish chunker parameter file (UTF8) (gzip compressed, UTF8)
  Note: The Spanish tagger parameter file is needed, as well.

A Windows version of the TreeTagger is available here. Unpack the zip file and follow the instructions in the INSTALL.txt file. The parameter files have to be downloaded separately. The tagger has to be invoked from a (Windows, cygwin, msys) shell. Therefore, you might want to install the graphical interface kindly provided by Ciarán Ó Duibhín.

Acknowledgments

The Russian parameter file was created on a corpus provided by Serge Sharoff. He has a webpage with various resources for Russian NLP.

The French and the Italian parameter files are provided by Achim Stein.

The parameter file for the French chunker was created by Michel Généreux.

The second Italian parameter files was provided by Marco Baroni.

The English parameter file was trained on the PENN treebank and uses the English morphological database created by Karp, Schabes, Zaidel and Egedi.

The Spanish parameter file was trained on the Spanish CRATER corpus and uses the Spanish lexicon of the CALLHOME corpus of the LDC.

The Spanish chunker was trained on the IULA Spanish treebank.

The Galician parameter file was trained on the Xiada corpus provided by the Centro Ramón Piñeiro para a Investigación en Humanidades

The Bulgarian parameter file was created by Julien Nioche on the Bulgarian Treebank. It uses UTF-8 encoding and the BulTreeBank tagset.

Michel Généreux created the parameter file for the French chunker.

The Estonian parameter file was trained on the Tartu Morphologically disambiguated corpus. Thanks to Mark Fishel for pointing me to this data!

Many thanks to Marco Baroni, Pablo Gamallo, Julien Nioche, Serge Sharoff, Michel Généreux, and Achim Stein for making their parameter files publicly available! Also thanks to Holger Wunsch and Cassio Binkowski for compiling the TreeTagger on MacOS!

Links

• Graphical Interface for the Windows version of the TreeTagger (developed by Ciarán Ó Duibhín)


• Serge Sharoffs web page where you can download a tokenizer and a parameter file for Chinese.
• Pablo Gamallos web page who provides resources for Portuguese and Galician.
• Achim Stein's web page on French and old French POS tagging with the TreeTagger


• Python Wrapper for the TreeTagger (developed by Laurent Pointal)
• Java Wrapper for the TreeTagger (developed by Richard Eckart de Castilho)
• Perl module for calling the TreeTagger and manipulating its output (developed by Aris Xanthos)
• R wrapper for the TreeTagger (developed by Meik Michalke)
• Ruby wrapper for the TreeTagger (developed by Andrei Beliankou)
• UIMA wrapper for the TreeTagger (developed at the UKP Lab)
• Java wrapper for the TreeTagger (developed by Thomas Holloway)


• Giuseppe Attardi's online interface to the TreeTagger.
• Italian Online Tagger at the University for Foreigners Perugia
• Leonardo di Donato created a Docker image which simplifies the installation process.

• Text Analysis Software developed by LinguLab
• Wikimeta