polyglot.tokenize package
=========================

Subpackages
-----------

.. toctree::

    polyglot.tokenize.tests

Submodules
----------

polyglot.tokenize.base module
-----------------------------

.. automodule:: polyglot.tokenize.base
    :members:
    :undoc-members:
    :show-inheritance:


Module contents
---------------

.. automodule:: polyglot.tokenize
    :members:
    :undoc-members:
    :show-inheritance: