home projects sketches open source talks

on machine learning and talking to dead people

23 December 2015

I’ve been playing around with multi layer neural networks recently. I’ve wanted a chance to experiment with recurrent neural networks for a while now. They are a type of neural network that are good at working with sequences. That could be books, midi files, speech, anything that can be represented as time time series data. So if you trained a system on a dictionary, you could tell the model a word like “synapse” and it might output th definition. I’m going to talk about a recurrent neural network I just built.

If you have ever seen Futurama, you will remember seeing the talking heads in a jar. They were from famous people in our past like Abraham Lincoln, Elvis Presley, Leonard Nimoy, and Napoleon. I remember watching those episodes when I was younger and thinking about how cool it would be to actually build something like this. I see a future where we can can consult past leaders and loved ones. We could ask them advice and get feedback or just talk to them as a way to remember them. These would be built as computational persona models. These systems would take in as much information about the person to construct a mental model about them. Information inputted into the model would be all of their writings, letters,their recorded speech, videos,photos, and any other information produced by them.

I believe with the recent breakthroughs in neural networks, we can start to build the first prototypes of these systems. Artificial neural networks have been used recently to generate shakespeare, Obama Speeches, and TED talks. These models are somewhat crude, but you can see potential.

Helen Keller model

I’ve been fascinated with Helen Keller for a while. She became a fully productive member of society while never being able to see or hear, she lived her life through touch. She wrote some several books. If she didn’t tell you, you couldn’t tell she was blind and deaf from her writing. She describes in her books what life was felt, what her environment was like, how she learned the concepts of words, her first moment of discovering she was a conscious being, and other amazing details.
I took all of her books she wrote and combined them into a single file for processing into my recurrent neural network model. The combined text of her 4 books was just under 1 MB. Unfortunately that is considered a small amount of data. Regardless of the small size, I still built the model.

I asked her a few thoughts:

seeing is like:

seeing is like or inspirents of a kiss licks, in child, for the last decting of accomplish with me for the mistakes in silence is to keep the moments filled whiter, the chaps of the house language was sends a humanise.

i wish i could:

presepred its repepenting and the days like the poor discuss of
language of the poem in the letters,
dotiment in the endless good and eager and over
the charicality of the hall of rubbings
that I hapmende the comprehend, the birds like your mind to perhaps the
not wind I should do?

my hopes and dreams are:
rebeboutt one
of opening as my eximism of my little fellt of language until my
lessons, and was our string,
The trees for her, and december is overward and beautiful.

As you can see, the results are horrible. You really need a lot more text. I tried to find other large bodies of text written in the first person. Abraham Lincoln and George Washington have hundreds of books,correspondences, and letters, but the archives are all images of cursive writing. I searched all over the place and almost gave up. Fortunately I found an awesome data set, subtitles of english movies.

I built several new models:

subtitles only
Helen Keller books mixed with subtitles
Helen Keller books mixed with subtitles, Helen Keller’s books repeated to account for ~15% of the total data set.
Helen Keller books mixed with subtitles, Helen Keller’s books repeated to account for ~15% of the total data set and then shuffling all of the data

These models took approximately 2 days each to train on a NVIDIA Telsa K80.

Helen Keller books mixed with subtitles, Helen Keller’s books repeated to account for ~15% of the total data set.

I could not see much of Helen Keller’s voice in this data. The model output mostly scenes from movies.

Life is:
Life is quite presswelled down here, so if there was nothing else there happens to you.
Let me try to.
I mean, she had everyone else to die.
Get on the whole house so he was funny.
- I haven ’t all the same.

I wish I could:
I wish I could phone sometimes, but the Honorable Tonogatist ewernal bexit…
Adken Woodtown!

seeing is like:
seeing is like people sleeping and prepared to handle this someone for you to get in there.
I need to trust you.
They’ re getting the photograph for a long time… in one place, in its phone for public occupation..

You can play with the model here: http://somatic.io/models/jtoy/helen-keller-books-ten-percent-with-opensubtitles

Helen Keller books mixed with subtitles, Helen Keller’s books repeated to account for ~15% of the total data set and then shuffling all of the data

I wanted to try shuffling the data to see how much if would effect te distribution of the data.

Life is:
Life is I do not suppose it was Sir.
And something new for a fucking idea, aren’ t you?
Find the kitchen.
It’ s going to see us.
Oh, thanks, God, grannym comes for Veronica!
When I get three scheme for her

I wish I could:
I wish I could play out of my idea, on her mother in it. If the flight
On that land they were made instead of internet, but he straightened out

seeing is like:
seeing is like the Mr. Grumple Reford, Freedom I love.
Where is she?
Fuck you, Tam!
You’ ve got before.
- No, it’ s pretty good.

You can play with the model here: http://somatic.io/models/jtoy/helen-keller-books-ten-percent-with-opensubtitles-shuffled

You can download all the model files here


It was fun to build these models, but finding good data was painful.
Adding the subtitle data did make the models look more like modern english versus Helen’s english. There is a lot of swearing in the subtitles.
Without the right data, these models are useless. Getting relevant data is going to be a key challenge to build deep learning models that work.
These particular models are nowhere near Futurama’s talking heads, but I see a lot of room for improvement. I think I can dramitically improve the performance with some different algorithms.
I have a long list of new experiments to try based off this last batch of experiments and I will publish my results here. Feel free to follow me to stay up to date or if you want to discuss other ideas for cool experiments to build.