3 Survivors : 1397 Models, 16,548 txt files, 8+ hrs of video (& no poems yet): Wavenet for Poem Generation: Secondary Results (After training for 6+ weeks continuously)

From 26-11-2016 to 11-12-2016, Wavenet-for-Poem-Generation (code on github) trained on an 11k poem corpus simultaneously in 7 different tabs of a terminal window (on a 8-core G5 each tab occupied a core of the CPU) — each tab was using different parameter settings.

In the end only 3 settings exceeded 100k training epochs before succumbing to the exploding gradient dilemma (detailed here).

The 3 surviving threads were known as 26-03, 38-59, and 39-18 — each folder name references its time of birth, the time it began receiving models from its thread, the neural network learning as it groped its way thru the corpus. These threads alone (of many myriad attempts) lived longest and saved out hundred of models with loss under 0.7.


SILENT VIDEOS of REALTIME POEM GENERATION

Warning: these videos are long! Total viewing time: 8+ hours.

Each is a silent realtime screen-capture of neural net models generating poems.

Poems from the same model are generated side-by-side to allow for comparative viewing. Note how young models create poems that rampage logic, merge less. Mature models from 50k-110k begin to emulate deflections and balance, concealing and revealing. And ancient models (after they suffer an exploding gradient data hemorrhage) create poems full of fragments and silences, aphasia and lack, demented seeking.

Suggested viewing: put on an extra monitor and let run. Consult occasionally as if the computer were a clever oracle with a debilitating lack of narrative cohesion.


SAMPLE OUTPUT

16,548 text file poems on github


PARAMETER SETTINGS

Common to each survivor were the following parameters:

  • Dilations = 1024
  • SkipChannels = 4096
  • Quantization Channels = 1024

Dilation channels were different for each survivor : 8, 16, 32.

Training process: complete terminal output of training runs .


FOLDER DETAILS

A subset of the models used in demo readings can be found online at github.

39-18 (2016-10-26T18-39-18)

Dilation Channels : 8

Born: 26 October 2016 at 03:29
Died: Sunday, 11 December 2016 at 11:28
Models: 458
Epochs: 145070
Size: 80.37GB

 

38-59 (2016-10-27T10-38-59)

Dilation Channels : 16

Born: 26 October 2016 at 03:29
Died: Sunday, 11 December 2016 at 8:03
Models: 475
Epochs: 150000
Size: 130.68GB

 

26-03 (2016-10-26T15-26-03)

Dilation Channels : 32

Born: 26 October 2016 at 03:29
Died: Sunday, 11 December 2016 at 11:28
Models: 464
Epochs: 145070
Size: 98.1GB

 

Comments are closed.