A toolkit for neural sequence-to-sequence transduction


Developer Documentation


There are three types of dependencies: core dependencies, development dependencies and dependencies for generating the documentation.

Install them via

> pip install -r requirements/requirements.txt
> pip install -r requirements/
> pip install -r requirements/

Developer Guidelines

We welcome contributions to sockeye in form of pull requests on Github. If you want to develop sockeye, please adhere to the following development guidelines.

  • Write Python 3.5, PEP8 compatible code.

  • Functions should be documented with Sphinx-style docstrings and should include type hints for static code analyzers.

def foo(bar: <type of bar>) -> <returnType>:
    <Docstring for foo method, followed by a period>.

    :param bar: <Description of bar argument followed by a period>.
    :return: <Description of the return value followed by a period>.
  • Sockeye 2 uses the Gluon API.
  • When using MXNet operators, preceding symbolic or hybridizable statements in the code with the resulting, expected shape of the tensor greatly improves readability of the code:
# (batch_size, num_hidden)
data = mx.sym.Variable('data')
# (batch_size * num_hidden,)
data = mx.sym.reshape(data=data, shape=(-1))
  • The desired line length of Python modules should not exceed 120 characters.

  • Make sure to pass unit tests before submitting a pull request.

  • Whenever reasonable, write py.test unit tests covering your contribution.

  • When importing other sockeye modules import the entire module instead of individual functions and classes using relative imports:

from . import attention

Unit & Integration Tests

Unit & integration tests are written using py.test. They can be run with:

> python test


> pytest

Integration tests run Sockeye CLI tools on small, synthetic data to test for functional correctness.

System Tests

System tests test Sockeye CLI tools on synthetic tasks (digit sequence copying & sorting) for functional correctness and successful learning. They assert on validation metrics (perplexity) and BLEU scores from decoding. A subset of the system tests are run on Travis for every commit. The full set of system tests is run as a nightly Travis Cron job. You can manually run the system tests with:

> pytest test/system

Submitting a New Version to PyPI

Before starting make sure you have the TestPyPI and PyPI accounts and the corresponding ~/.pypirc set up.

  1. Build source distribution:
    > python sdist bdist_wheel
  2. Upload to PyPITest:
    > twine upload dist/sockeye-${VERSION}.tar.gz dist/sockeye-${VERSION}-py3-none-any.whl -r pypitest
  3. In a new python environment check that the package is installable
    > pip install -i sockeye
  4. Upload to PyPI
    > twine upload dist/sockeye-${VERSION}.tar.gz dist/sockeye-${VERSION}-py3-none-any.whl

    When pushing a new git tag to the repository, it is automatically built and deployed to PyPI as a new version via Travis.

Code of Conduct

This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact with any additional questions or comments.


See the LICENSE file for our project’s licensing. We will ask you confirm the licensing of your contribution.

We may ask you to sign a Contributor License Agreement (CLA) for larger changes.