Category: digital poetry

2-layer 256-cell LSTM neural net trained on a source text of moderate size

Yesterday, sitting in Hong Kong, I launched a cluster of GPUs in North Carolina and ran a neural net for 9 hours to generate proto-language.


Using modified code from the Keras example on LSTM text generation (and aided by a tutorial on aws-keras-theano-tensorflow integration), the cluster produced 76,976 words.

Many of these words are new and never-seen-before. It’s like having Kurt Schwitters on tap.

mengaporal concents
typhinal voivat

the dusial prespirals
of Inimated dootion

Here is my bill for over 11 hours of GPU cluster time:

screen-shot-2016-10-03-at-07-55-16


Neural nets learn through exposure to words just like babies.  The more they hear, the better they get. For preliminary testing of the code, a 2-layer 256-cell LSTM neural net was trained on a source text of moderate size: a draft of my book, in a 430kb text file format. So it’s as if a baby was exposed to only critical literary theory, no kids books, no conversation, just theory.

The results can be very very strange:

in a way of stumalized scabes occurs forms. and then paradigm for some control implicit and surjace of need)

And they evolve over time from simplicity:

and conceptual poetry and conceptual poetry in the same that the same that the same that the same that has been are conceptual poetry and conceptual process of the static entity and conceptual sourd subjects that in the same of the same that digital poetry in the static and conceptual sourd source of the static entity and conceptual sourd source of the static entity and conceptual poetry in the station of the static environment for the station of a section of a section of a section of a section of a section of a section of a section of a section of a section of a section of a section of a section of a section of a section of a

… to complexity

u archetypht]mopapoph wrud mouses occurbess, the disavil centory that inturainment ephios into the reputiting the sinctions or this suinncage encour. That language, Y. untiletterforms, bear. and matter a nalasist relues words. this remagming in nearogra`mer struce. (, things that digital entibles offorms to converaction difficued harknors complex the sprict but the use of procomemically mediate the cunture from succohs at eyerned that is cason, other continuity. As a discubating elanted intulication action, these tisting as sourdage. Fore?is pobegria, prighuint, take sculptural digital cogial into computers to Audiomraphic ergeption in the hybping the language. /Ay it bodies to between as if your this may evorv: and or all was be as unityle/disity 0poeliar stance that shy. in from this ke

It is important to recognize, the machine is generating words character by character, — it has never seen a dictionary; it is not given any knowledge of grammar rules; it is not preloaded with words or spelling. This is not random replace or shuffling, it is a recursive groping toward sense, articulate math, cyber-toddler lit. And it is based on a 9 hour exposure to a single 200 page book with an intellectual idiom. More complex nets based on vast corpuses trained over weeks will surely produce results that are astonishing.

Yet I somehow feel that there is a limit to this (LSTM) architecture’s capacity to replicate thought. Accuracy is measured through a loss function that began at 2.87 and over 46 runs (known as epochs) loss descended to 0.2541. A kind of sinuous sense emerges from the babble but it is like listening to an impersonation of thought done in an imaginary language by an echolaliac. Coherency is scarce.


Code & Complete Output

Code is on github.

Complete 80k word output saved directly from the Terminal: here.


Excerpts

Do you like nonsense? Nostalgic for Lewis Carrol?

Edited excerpts (I filtered by seeking neologisms then hand-tuned for rhythm and cadence) output here. Which includes stuff like:

In the is networks, the reads on Gexture
orthorate moth process
and sprict in the contrate

in the tith reader, oncologies
appoth on the entered sure

in ongar interpars the cractive sompates
betuental epresed programmeds

in the contiele ore presessores
and practions spotute pootry

in grath porming
phosss somnos prosent


E-lit Celebrity Shout-Out

Of course as the machine begins to learn words  (character by character assembling sense from matrices of data), familiar names (of theorists or poets cited in my book) crop up among the incoherence.

in Iteration 15 at diversity level 0.2:

of the material and and the strickless that in the specificient of the proposed in a strickles and subjective to the Strickland of the text to the proposing the proposed that seem of the stricklingers and seence and the poetry is the strickland to a text and the term the consider to the stricklinger

in Iteration 26 at diversity level 1.0:

 sound divancted image of Funkhouser surfaleders to dexhmeating antestomical prosting

in Iteration 34 at diversity level: 0.6

Charles Bernstein
Fen and Crotic

unternative segmentI
spate, papsict feent

& in Iteration 29 at diversity level 1.1:

Extepter Flores of the Amorphonate

evocative of a Genet-made literary saint among the rubble of some machinic merz, Leo has adopted temporarily his cybernetic moniker:

screen-shot-2016-10-02-at-11-42-52-am

 

 

LSTM CHARRNN: blossoming acronyms

Machine learning hacks.
Building poetic nonsense with neural nets.

 

mules and the technology, the created and the tractions and the tractional artically of the traction of the tractical processe of the prectional and and and structured the entional the eractions of the the tractions and the tractions of the termore the the creative of the ention of the metach of the interallention of the termore and the entions of the created the tractions and structured the media and contempler the tractically, and in the eranted the tractions of the metach of the contempler in

 

The network knows nothing of language.
Sense emerges from feedback.

 

Using Theano backend.
Source text on which the neural net is trained:  MITPressDRAFT

 

! ” # % & ‘ ( ) + , – . / 0 1 2 3 4 5 6 7 8 9 : ; = ? @ [ ] ` a b c d e f g h i j k l m n o p q r s t u v w x y z

 


After cleanup total chars: 61
Corpus length: 421183

 

My days evolve:
Making micro mods to Keras code.
Reading up on LSTM.
Watching Sirajology.

 

As usual, there is tons of text to read…

 
Continue reading

ELO Performance (Brief Reproduction)

Read a screen where code is rapidly producing poems.
Find a path through the words: construct a poem from machinic intuition.

The following recreates a performance made at
the Electronic Literature Organization conference
in Bergen, Norway on Aug. 4th 2015.

Details: http://bdp.glia.ca/smaller-words-shrink-gapped
Code: https://github.com/jhave/Big-Data-Poetry

Technical process: the following poems were produced using a 10,000+ corpus of poems used as templates. Each poem has been sent to Alchemy API to produce entity-recognition, POS, and sentiment reports. That analysis influences replacement algorithms. Replacement uses NLTK synsets and Pattern.en and a reservoir of words found in the corpus that do not have synonyms.

In short, 10000 poems are
transformed by algorithms
into millions of poems
at an extremely rapid rate.

The reader must then
find a way to convert this
spew into spoken word.

ELO 2015 — Bergen – Performance

One of the ends
of digital literature
is an external intuition.

External intuition
is an engineering problem.

Intuition
in this case
is me.

Skidding thru the
generated poems
as they augment
my imagination.

I call this act of
augmented imagination:
cyborg/ skid/ spreedr poetry.

####

For the performance,
at ELO conference performance in Bergen, Norway on Aug. 5th 2015
I generated 100 poems, pausing ever 5 poems for 20 seconds,
and tried to weave spoken poetry from the output.

Unfortunately I forgot to record the audio/screenvideo
so my actual spoken-poem is gone!

#####

If you want to try it yourself:
Download attached code and data corpus
— this includes the poems generated during the performance.

Unzip
run in command line
>>cd code/poetryFoundation/ELO_July2015/
>>python ELO2015_PERF_Creeley-Aug4th.py

This code will run with Python package Anaconda 2.7 installed.

####

Code will continue to be updated on
on github: https://github.com/jhave/Big-Data-Poetry.

Spreeder: the feature film (EPC 20th Anniversary Celebration)

Loss Pequeño Glazier is celebrating the 20th anniversary of the Electronic Poetry Centre along with Charles Bernstein, cris cheek, Tony Conrad, Steve McCaffery, Myung Mi Kim, Tammy McGovern, Joan Retallack, Laura Shackelford, Danny Snelson, Dennis Tedlock, Cecilia Vicuña, Elizabeth Willis, & Wooden Cities with Ethan Hayden. Along with exhibitions by: “Abra” (Amaranth Borsuk, Kate Durbin & Ian Hatcher); Pry: iPad-based Novella (Samantha Gorman & Danny Cannizzaro); “Enter:in’ Wodies” (Zuzana Husárová & Lubomír Panák) & myself.  http://epc.buffalo.edu/e-poetry/2014/EPC-at-20/

So I made a future-feature film of a computer writing in real time
Spreeder (approx. output 8,100 poems; 2 hour-long real-time episodes).
Implemented in Python with Alchemy API, NLTK and pattern.en

 

 

SPREED : Speed Screen Reading : One Hour Real-Time Poetry Generation ScreenGrab

Using Python (Anaconda), NLTK, WordNet, Alchemy, pattern.en, and pyenchant
to analyze and perform word replacement
on a corpus of 10,119 poems scraped from the PoetryFoundation
and generate 7,769 poems in approx. 2 hours and 30 minutes.

This is a real-time hour-long screen-grab output
of the trace window in SublimeText
as the poetry-gen program runs.



Code on Github
Made by Glia.ca  


And here is another episode of  “Crawling Toward Creeley” (in this episode: a variation on the ‘Gnomic Verses’ generated then modified)

Markov Bern

Markov chains are one of the traditional tricks in the NLP playbook. They are the apple pie-chart of text-generation.

Basic process: given a source text, find words that are neighbours, if you know the neighbours of a word, you can form a chain if you wish. [(“you”),(“know”,”can”,”wish”)] and reconstruct a text which contains pairs (bigrams) from the source.

So I did that using as source texts: Charles Bernstein Dark City and Rough Trades. (Found on Bernstein’s EPC author page).

The result is an example of what Charles Hartman might refer to as newbie-augmented-cyborg-poet (dead simple technically, but satisfying artistically since the volume of generated texts from which new verses can be hand-crafted is massive). This sort of auto-suggest based-upon-a-corpus technique radically shifts the dimensions of creativity: in the first ‘modified’ example I edited the output, adding words, disguising some obvious quotations from Bernstein, truncating verses, changing lines, modulating rhythms. In the raw output below, it’s just the computer (fueled by Bernstein’s berning phrases), it could go on infinitely given a large enough corpus.



Poetry is both the easiest and the hardest to generate. Since non-linear deflections and word-riffs are an aspect of contemporary poetry, slamming together ripe fertile conjunctions is easy. Migrating toward a sensitive, complex, experiential and contextual lived poetry is the real challenge (I didn’t even begin to touch it here).


Code on Github
Made by Glia.ca  


what is there if
An exit is as
clear as dead.

Billboards poster our losses.
Better a barber
than a thousand one-line
sweat glands.

SYN-SCI-RAP

I think I have begun to develop a mild form of insanity that often strikes those who fiddle around with computationally-generated text. After reading thousands of lines of dense incomprehensible gibberish it clarifies and makes sense, often more sense than any mere linear thought. The brain acclimatises to syntactic pressure.


Recipe for mildly insane word-salad:

  • take 57,000 rap songs input by fans,
    • extract all words that do not return results from WordNet synset search and put into Reservoir
  • one list of scientific terminology (for sombre intellectual tone)
    • chop off “-ology” wherever it occurs
  • one list of swear words (for spice)
  • call to WordNet synset algorithm (for fibre and continuity)
  • use pattern.en to do conjugation (for a tiny bit of coherence)
  • use NLTK part-of-speech tagging
  • Alchemy for entity (people, places, etc…) replacement
  • 10,000 or more poems

Mix all ingredients together using replacement algorithms.


To read 10116 poems (simple style) (in a single 24-mb html page) generated in 10356.4216051 seconds (2.87 hours, 3612pph [poems per hour], 60 ppm [poems per minute] ) on 2014-08-14 at 02:54 click here


Read a selection of just a few poems 

Read the RAP Reservoir: 33,150 words extracted from 56k user-input rap songs that did not return any usable results from a WordNet synset search. If you are looking for the evolution of language that occurs through mutation (typo, mispells, pop-cruft) this is it.


Code on Github
Made by Glia.ca  

 

 

RSERVOIRD

Reservoirs are where I put unwanted words. These orphan words are later fed back into society whenever the next orphan appears. Thus words swap circumstances, exchange semantic situations, live out different meanings.

Click on an image to visit a reservoir.

Screen Shot 2014-08-04 at 9.29.45 pm

 

Smaller Words (shrink-gapped at 64ppm)

Words disconnected from their primary communicative intent operate as lesions/lessons within the psyche.

Today, I generated another 10120 poems using a very mild modification of the alchemy-synset algorithm with the average word-length constrained even shorter. Speed decreased to 64 ppm poems-per-minute. This reduction in word-length seems (to me) to make some of the absurd illogical elliptical generated fragments seem a bit more legible, taut, elusive and rare. It comes at a cost of coherence. The output reads like Robert Creeley in the process of becoming Samuel Beckett in Gertrude Stein’s gut.


To read 10120 poems (simple shrink-gapped style) (in a single 20-mb html page) generated in 9500.10482717 seconds (2.63 hours total, 3847 poems per hour, 64 ppm, poems-per-minute) on 2014-08-04 at 12:02, click here


Code on Github
Made by Glia.ca  


Edited fragments:

Let me give a robot aging
And as it rains tag the sun with ‘almost’
while within the green fog
a tree that is a dot
Softly opens after the burn.
………
Gaza masked
as me masked
each heavy’s heart out
crying at halo’s burial
making a meal of soil
the city a scar
…..
enthusiasm’s ice. We have
Walked on the bore all nite.
Now in the light
We exchange smells and snot.
By dawn we will have buried our lack
And glued wet to the army of being.
…………

 

Continue reading