# Losses and decoders for end-to-end Speech Recognition and Optical Character Recognition with PyTorch

[  ](https://travis-ci.com/artbataev/end2end)

The module focuses on experiments with CTC-loss (Connectionist Temporal Classification) and its modifications.

Under active development.

## Documentation [ artbataev.github.io/end2end/ ](https://artbataev.github.io/end2end/)

## Losses - [x] CTC (C++, CPU)

## Decoders - [x] CTC Greedy Decoder (C++, CPU) - [x] CTC Beam Search Decoder (C++, CPU) - [x] CTC Beam Search Decoder with language model (C++, CPU)

## How to install

Requirements: - Python 3.6+ - Tested with PyTorch 1.6.0+ (maybe compatible with other versions)

1. Install PyTorch from [pytorch.org](https://pytorch.org), e.g.

   `bash pip install torch `

2. Install tools to compile

   `bash sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \ sudo apt-get update && \ sudo apt-get install libboost-all-dev g++-7 -y `

3. Install the module

   `bash pip install -v git+https://github.com/artbataev/end2end.git ` or `bash git clone --recursive https://github.com/artbataev/end2end.git cd end2end python setup.py install python -m tests.test_ctc python -m tests.test_ctc_decoder `

## How to use

### CTC Loss ```python import torch from pytorch_end2end import CTCLoss

ctc_loss = CTCLoss(blank_idx=0, time_major=False,

reduce=True, size_average=True, after_logsoftmax=False)

batch_size = 4 alphabet_size = 28 # blank + 26 english characters + space

logits = torch.randn(batch_size, 50, alphabet_size).detach().requires_grad_() targets = torch.randint(1, alphabet_size, (batch_size, 30), dtype=torch.long) logits_lengths = torch.full((batch_size,), 50, dtype=torch.long) targets_lengths = torch.randint(10, 30, (batch_size,), dtype=torch.long)

loss = ctc_loss(logits, targets, logits_lengths, targets_lengths) loss.backward() ```

### CTC Decoder ```python import torch from pytorch_end2end import CTCDecoder

batch_size = 4 alphabet_size = 6 decoder = CTCDecoder(blank_idx=0, beam_width=100,

time_major=False, after_logsoftmax=False, labels=[“_”, “a”, “b”, “c”, “d”, ” “])

logits = torch.randn(batch_size, 50, alphabet_size).detach() logits_lengths = torch.full((batch_size,), 50, dtype=torch.long)

decoded_targets, decoded_targets_lengths, decoded_sentences = decoder.decode(logits, logits_lengths) for sentence in decoded_sentences:

print(sentence)

```

### Future Plans

#### Losses - [ ] Gram-CTC - [ ] CTC without blank (C++, CPU) - [ ] CTC (Cuda) - [ ] CTC without blank refactoring - [ ] Dynamic segmentation with CTC refactoring

#### Decoders - [ ] Restrict Beam Search with vocabulary - [ ] Allow custom transcriptions - [ ] Gram-CTC Beam Search Decoder