From 26-10-2016 to 11-12-2016, Wavenet-for-Poem-Generation (code on github) trained on an 11k poem corpus simultaneously in 7 different tabs of a terminal window (on a 8-core G5 each tab occupied a core of the CPU) — each tab was using different parameter settings.
In the end only 3 settings exceeded 100k training epochs before succumbing to the exploding gradient dilemma (detailed here).
The 3 surviving threads were known as 26-03, 38-59, and 39-18 — each folder name references its time of birth, the time it began receiving models from its thread, the neural network learning as it groped its way thru the corpus. These threads alone (of many myriad attempts) lived longest and saved out hundred of models with loss under 0.7.
SILENT VIDEOS of REALTIME POEM GENERATION
Warning: these videos are long! Total viewing time: 8+ hours.
Each is a silent realtime screen-capture of neural net models generating poems.
Poems from the same model are generated side-by-side to allow for comparative viewing. Note how young models create poems that rampage logic, merge less. Mature models from 50k-110k begin to emulate deflections and balance, concealing and revealing. And ancient models (after they suffer an exploding gradient data hemorrhage) create poems full of fragments and silences, aphasia and lack, demented seeking.
Suggested viewing: put on an extra monitor and let run. Consult occasionally as if the computer were a clever oracle with a debilitating lack of narrative cohesion.
SAMPLE OUTPUT
16,548 text file poems on github
PARAMETER SETTINGS
Common to each survivor were the following parameters:
- Dilations = 1024
- SkipChannels = 4096
- Quantization Channels = 1024
Dilation channels were different for each survivor : 8, 16, 32.
Training process: complete terminal output of training runs .
FOLDER DETAILS
A subset of the models used in demo readings can be found online at github.
39-18 (2016-10-26T18-39-18)
Dilation Channels : 8
38-59 (2016-10-27T10-38-59)
Dilation Channels : 16
26-03 (2016-10-26T15-26-03)
Dilation Channels : 32