RNN example?

[alexanderkjeldaas]

I'd love to see an RNN example.

[blackgnezdo]

Do you have a learning problem an a data set in mind that would be interesting?

[alexanderkjeldaas]

A chatbot trained on the ubuntu dialogue corpus would be a great example.

[blackgnezdo]

I like the data set, it seems to have non-trivial size. I found The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems paper which references the logs. The logs are in text form. Do you know of a more structured representation of the same data? A big part of an ML system is pre-processing the raw data into a form which can be efficiently consumed by the learner. This is also a piece of the puzzle which is currently not easily expressed in TensorFlow.

I found this ubuntu-ranking-dataset-creator but it seems to be preparing the data for a ranking problem and not a chatbot.

[alexanderkjeldaas]

Could https://github.com/brmson/dataset-sts/tree/master/data/anssel/ubuntu be used?

[alexanderkjeldaas]

How should it be expressed?

[alexanderkjeldaas]

This series of blog posts uses the reddit data set http://lauragelston.ghost.io/speakeasy-pt1/
The code is here: https://github.com/Charging-Alot/SpeakEasy-AI
Maybe porting that could be a useful example? Or porting the english-french example from tensorflow that they based their system on.

[alexanderkjeldaas]

Replicating this seq2seq setup is an alternative: https://github.com/spro/intense

[alexanderkjeldaas]

Another alternative, replicating code like this:
https://gist.github.com/nivwusquorum/b18ce332bde37e156034e5d3f60f8a23
and
https://gist.github.com/nivwusquorum/160d5cf7e1e82c21fad3ebf04f039317

[ghost]

Hi @blackgnezdo and @alexanderkjeldaas , great idea,

I think starting with LSTM is not the most simplest example though, maybe the simplest RNN that works well is a GRU?

Here's some starter code,

https://github.com/suriyadeepan/rnn-from-scratch/blob/master/gru.py

Would be great for people to learn Haskel-TF from a useful example 👍

[delanoe]

Looking at the GRU example, it seems that current TF Haskell version does not support all functions used in the code above. Maybe I am wrong ? Do you have any insights to get started ?

[fkm3]

Enough is supported to implement an "unrolled" RNN with a fixed length. If you are brave, I have a WIP RNN example here: https://github.com/fkm3/tensorflow-haskell/blob/dqn/tensorflow-mnist/app/Seq.hs

It is using an LSTM, but there is also a GRU implementation in https://github.com/fkm3/tensorflow-haskell/blob/dqn/tensorflow-ops/src/TensorFlow/RNN.hs#L42

I'm distracted by my day job right now, so it might be a while before I finish it up and send the various pieces for review.

[delanoe]

Many thanks for your answer. This script helps to understand how to use the lib.

[rikvdkleij]

Can someone share a working LSTM example? Many thanks in advance!