David Cope and Emily

reprinted from the observer sunday 11 June

David Cope: 'You pushed the button and out came hundreds and thousands of sonatas'

Composer David Cope has spent the last 30 years teaching computers to create classical music

David Cope at work in his California home. Photograph: Catherine Karnow for the Observer

 

Where does music come from? If pressed on this question, many of us would say it comes from the "soul", or from the "heart" of the person who composed it. That music is the clearest expression of human emotion, one person to another; that certain chords, certain melodies seem to communicate a whole language of feeling. When we listen to a Beethoven symphony or a Chopin sonata, we are hearing, we might say, the authentic expression of the composer's inner harmonies and discords, carried magically across the centuries. Could we ever be so moved by a piece of music written by a computer? We'd probably like to think not. David Cope, emeritus professor of music at the University of California, Santa Cruz, would beg to differ. "The question," Cope tells me, "isn't if computers possess a soul, but if we possess one."
Cope, now 69, has devoted the past 30 years of his life to what amounts to an obsessive examination of that particular question. He began, almost by default, back in 1980, with a severe case of writer's block. One of America's most acclaimed young composers, whose music had been performed at Carnegie Hall, and won great critical praise, Cope had been commissioned to write an opera. For weeks and months he sat at his piano, or stared at a blank piece of sheet music; nothing came. He had a wife and four children to support. In desperation he started playing with a computer.
What he found there changed his life and, perhaps, the course of musical history. Cope had long held the belief that all music was essentially inspired plagiarism. The great composers absorbed the music that had gone before them and their brains "recombined" melodies and phrases in distinctive, sometimes traceable, ways. We all have an internal database of musical reference; composers were those with the ability to manipulate it in new patterns. With the aid of an early computer, he realised he could put this to the test.
His first experiments with artificial musical intelligence were clunky, synthesised pieces, pastiches of easily identifiable work; but slowly, programming and reprogramming, inputting vast amounts of coded reference, he came to see how he might begin to shape a musical memory. The Eureka moment came one afternoon in 1983 after he had been working for a while trying to take apart and put back together chorales (four-part vocal hymns) in the style of JS Bach. He had a rules-based program, complicated and code-heavy, but it never produced anything approaching life or surprise.
That afternoon, on the way to the local store, he came to realise that Bach didn't exist in his predictability, but in the minute, multiple places where he broke his own rules, where he defied expectation of a particular progression. Cope developed "a little analytical engine" that could insert some randomness within the predictability. He began to analyse Bach's music not just mathematically but with a sense of narrative tension and surprise, weighting different components according to his own feel for the music's "storytelling" power. His program, at this point, seemed to develop a personality of its own; "Experiments in Musical Intelligence" became Emmy. When fed with enough of a composer's work, Emmy could deconstruct it, identify signature elements, and recombine them in new ways. One day Cope pushed a button on Emmy, went out to get a sandwich and when he returned his workaholic creation had produced 5,000 original Bach chorales. In 1993, Cope released an album, Bach by Design, and waited for the response.
When you listen to that album now – and those that followed, including Virtual Mozart and, triumphantly, Virtual Rachmaninoff, you are discomfited and surprised in equal measure. Cope's work is far more than copying, it carries the recognisable DNA of the original style and fashions it into something recognisable but entirely new. The musical establishment reacted at first with alarm, and then with vitriol. Cope found it difficult to get any serious musicians to play Emmy's work, though it made many of the same demands as the "real thing". Critics convinced themselves that they heard no authentic humanity in it, no depth of feeling, Cope was characterised as a composer without a heart; his recent memoir is called Tin Man.
One of the problems that the music highlighted was the fact that in Cope's terms, the music of Mozart, say, was endless in its possibilities. As he suggested when I spoke to him last week: "Because my program was continuing to pump out music like a spigot, it became a problem of: 'Why play this sonata and not that one?'" Cope has no doubt that Mozart in particular, with his structural genius, would – if he'd had the means – have utilised computerised intelligence in exactly the same way. When you remove the "human" element of the work, however, Cope recognised, you also lose a great deal of its urgency. "When you had the database figured out it was really a one-stroke deal: you pushed the button and out came hundreds and thousands of sonatas or whatever."
He realised that what made a composer properly understandable, properly "affecting", was in part the fact of mortality. Composers had to die, and the ending made sense of what had gone before. With this in mind, Cope unplugged Emmy six years ago; her work – which he limited to 11,000 chosen pieces, was done. Emmy – housed on an ancient Power Mac 7500 (discontinued in 1996) now sits idle in the corner of his office. Cope has subsequently been at work, nurturing Emmy's "daughter", Emily Howell, (the first name from her mother, the second from the Christian name of Cope's own father) with whom he has a far more "equal" relationship.
Emily Howell has a compendious memory that involves an intimate understanding of the works of 36 composers "starting with Palestrina, [an Italian court musician of the 16th century] and ending with David Cope in the 21st century". Their output is far more collaborative than that of Emmy. Cope will ask Emily a musical question, feeding in a phrase. Emily will respond with her own understanding of what happens next. Cope either accepts or declines the formula, much in the way he would if he was composing "conventionally".
"It is," he says, "a bit like dealing with a small child; the program is an empty pot and I dribble small bits of music into it, and it responds to what I have put in… it's a process of carrots and sticks, really. I think it is producing good results but it takes a lot of time."
Cope's ambitions remain exactly what they were, when, as an asthmatic child in Phoenix, Arizona, he was moved to wonder by Tchaikovsky's Romeo and Juliet, and knew he had to be a composer himself. "My two goals are an original style and to create something I love," he says. "The program is a cat not a dog, it keeps itself to itself, you can't take it for walks. I can only generally pick it up and point it in the direction that I want it to go…"
Emily Howell's first album, From Darkness, Light, composed in six movements and performed on two pianos, was released earlier this year. It was met largely with silence; the critics who were moved to respond did so with the usual sniffy constructs about "an absence of genuine humanity". Cope remains undaunted.
"People tell me they don't hear soul in the music," he says. "When they do that, I pull out a page of notes and ask them to show me where the soul is. We like to think that what we hear is soul, but I think audience members put themselves down a lot in that respect. The feelings that we get from listening to music are something we produce, it's not there in the notes. It comes from emotional insight in each of us, the music is just the trigger."
Emily is still a work in progress for Cope. He thinks she is getting towards a mature style. "Five years from now I believe she will really be somewhere," he says.
It must be a curious process, like watching an external mind working, I suggest. What has it taught him about himself? "Two things. That the mechanisms of the brain are incredibly simple, but that its ability to create extraordinary complexity should constantly amaze us."
Will Emily survive him? "She needs a provocateur," Cope says, "but then so do humans. You cannot create music without reference to other music. Like us, she needs to be turned on to something."
He can't imagine the possibility of going back to writing with just his own intuition and a pen and paper. "The programs are just extensions of me. And why would I want to spend six months or a year to get to a solution that I can find in a morning? I have spent nearly 60 years of my life composing, half of it in traditional ways and half of it using technology. To go back would be like trying to dig a hole with your fingers after the shovel has been made, or walking to Phoenix when you can use a car."
When he began, though he was confident that what he was embarking on represented the future, Cope felt all the loneliness of the pioneer. Now, he suggests there is a growing interest in the possibilities. Not least the commercial ones. He was recently approached by a headline pop band – he won't say which – to see whether Emily could be persuaded to produce some hits. Though "recombining" elements of popular music is a court case in the making, there has, he suggests, not surprisingly been an enormous interest in creating music for ringtones and for games. "In the next 10 years," Cope says, "what I call algorithmic music will be a mainstay of our lives."
The perception that we might identify the particular musical combinations that stir our individual souls suggests many other potential applications. Until now online music stores have based recommendations for future purchases on what a customer has bought before, but Cope's kind of musical analysis suggests a more intimate understanding of our particular desert island discs might be possible. It was reported last month that separate teams of researchers at universities in San Diego and in São Paulo are refining different ways of analysing musical genres and rhythms, to enable predictions of what we are likely to buy to be far more precise (see panel below).
If music can be reduced to formulae and equations, does it begin to undermine notions of what music might mean to us? Douglas Hofstadter, author of the key book on the fundamentals of cognition, Gödel, Escher, Bach, has long lectured on the implications of Cope's work in understanding how the mind – and music – works. "In 20 years of working in artificial intelligence," he says, he has encountered "nothing more thought-provoking than David Cope's experiments." Hofstadter has distilled his thoughts on Cope's work into a full-length lecture performed in rhyme that begins with a question that might prove fundamental to future understanding of composition:
Is music a craft
Or is it an art?
Does it come from mere training
or spring form the heart?
Did the études of Chopin
reveal his soul's mood?
Or was Frédéric Chopin
Just some slick "pattern dude"?
Hofstadter is very fond of Cope's remark that "Good artists borrow, great artists steal," though he is troubled that some of the mysteries of the creative process might be lost along the way, and with them a part of our understanding of what it means to be human. Cope, for his part, retains all of his sense of wonder at the composers – geniuses of recombination – who have gone before. Does he still dream of creating a masterpiece? I wonder.
He says he has no idea what that word means.
Have Emmy and Emily at least short-circuited the angst and musical block that led him to create them in the first place?
"No," he says. "Not at all. I still get anxious and despairing. It never turns out as well as I hope it will. Every morning I wake up with the notion that I have failed at everything and I have to create some reason to exist." He and Emily then get back to work.

an example can be heard here

But is it art? Our classical music critic gives her verdict

So far, my acquaintance with the works of Emily Howell is limited. It's unlikely to get further. Listening to YouTube clips, I had to force my finger away from the "cancel" button. The repetition of silken arpeggiated chords, set against a simple melody, had the uneasy feel of dining with waxworks: all the key ingredients without the breath of humanity.
Computers have been used for musical ends since the 1950s, first on the Ferranti Mark 1, the IBM 704, the Moog and so on. Boulez built his entire Ircam research lab in Paris to harness music to technological innovation. But there the intention has been to enable composers of genius to venture into new, unimagined worlds of sound. Emily's software, instead, is enabling bad composers to imitate familiar idioms for their own pleasure. One day, it may do more. It's not a crime, but nor is it art. Fiona Maddocks

The beat goes on: programs that analyse our musical tastes

Would you trust the musical tastes of a computer? If you're shopping on Amazon, it's one thing to know that "customers who bought this item also bought..." whatever that might be. Internet radio station Last.fm and similar outfits such as mflow work by building profiles of their users' tastes and then cross-referencing that data: they're social networking services for music lovers.
Pandora Radio works on yet a different principle. This US-only music recommendation service was dreamed up by trained jazz pianist Tim Westergreen, who started what he calls the Music Genome Project a decade ago. This involves an attempt to "capture the essence of music at the fundamental level" through the identification of 400 musical attributes that describe any song.
The project's analysts study individual songs (a process that takes up to half an hour) and note their characteristics: for instance, perhaps the vocal is a bit squeaky, a la Lady Gaga (left), or perhaps it's more the sort of tenor growl employed by Tom Jones. Pandora can then serve up something with similar qualities; it now has 700,000 songs in its library.
Such a system still relies on human opinion, of course. Scientists at the University of California, San Diego, think such services could be improved by actually teaching computers the rudiments of music, according to a recent report in New Scientist.
Luke Barrington, a researcher in artificial intelligence, is building software that can analyse a piece of music and distil information about it that may be useful for software trying to compile a playlist. Drawing on this information and using tricks employed in speech recognition technology, the software can then assign the music a genre or even give it more subjective descriptions, such as whether or not a track is "funky", Barrington says. Caspar Llewellyn Smith

The characters and neural networks

The characters society is based, on part on neural networks. Neural networks exists in the brain, its how the brain cells connect with each other, how the mechanism of thinking work.
These can of be modeled in computers, not exactly, but close enough to give similar results. When used on a computer neural networks are called activation units.

Traditional computers work serially,no matter how fast they work, the process they compute still happen one after another. For instance the sum (4+5) + (6+8) is worked out as a three step process, first 4+5 are added,then 6+8, and finally both the results are added together.
Neural networks work in parallel, imagine them as loads of independent people working together rather than one person working in steps. In a parallel system 4+5 and 6+8 are worked out at the same time,then the results are added, effectively a two stage process.
If you imagine a computer as a room with a single bloke in it. When information comes in, for instance a picture of a horse, the bloke in the room looks up in a reference book,

step one Animal, vegetable or mineral?
step two Mammal,reptile,insect,bird?
step three how many legs?
step four how big is it.......and so on......

Eventually after all the search criteria has been followed the bloke reduces the information down to being a horse and sends the information out.
Alternatively a neural network could be imagined as a room with loads of blokes in it hanging around. When information comes in, the picture of a horse, something different happens. Each one of the blokes is an expert in a different field. One is really good at identifying colours, one is really into animals, one is really into the number four, one recognises sizes. A bit like when people can identify trains by their sound,or remembers all the different types of car. All at once the fellows recognise different aspects of the horse and come to a consensus weather the legs belong to a horse rather than a chair, as the animal guy knows its a horse, and the colour guy knows its brown, and so on. The knowledge is shared out to different points and processed.

A bit like how knowledge is shared by many different people over the (idealized) internet,no one is an expert but using knowledge as a collective force we can pool our resources and find out more stuff.
I can see parallels here on how we organise our lives, shops sell one type of product, if you need specialist advice to go to a specialist shop, not the supermarket,where it may be available but doesn't have the knowledge base. If specialist shops get to diversified the specific knowledge about one thing gets diluted. Thats why polymaths are not all that common, if people specialise in two subjects they are generally related.

Train spotters, English civil war enthusiast, real ale experts all reflect knowledge nodes, that when seen in isolation as sad enthusiasts but seen together as a solid knowledge database of the world as a whole.
We are a parallel species father than a serial species, we need the farmer, the farmer needs the industrialist.

The characters could be seen as very idiosyncratic neural network nodes. They have a knowledge base but either the knowledge base is wonky (the character that just wants to collect all the objects in the world, without knowing what they are) or what they do with it is wonky (such as the shopper character who looks up products and relates them together by weight rather than type of product). We follow the characters as they negotiate through life through using their knowledge base

Doug's game of life





Doug is a character based on Conways Game Of Life . Doug genrates a celluar pattern from an initial state and uses this to create music. Doug is modeled on a Biological algorithm and uses this to navigate life, Being a "biological algorithm" requires Doug to a sleep for 8 hours a day, which makes it difficult for him to actually be used. Ella the artist spends quite a lot of the time trying to wake him up.
Biological algorithms also produce other side effects, doug could die, requiring a doug volume 2, or a doug volume 3 and so on.
 The gaps in Dougs processing reflects the way we deal with the world, after an 8 hour sleep Doug carries on with what he was doing before in a very linear pattern. Although we can be get so angry about a situation,causing humans to go on to the streets to riot  our basic biological programing eventually kicks in requiring us to sleep, in the morning the anger returns.


As dougs cells evolve they trigger behaviours that change the composition. Each cell is given and attribute which is triggered when the cell is activated.
The cells act as individuals adding their influence to the composition. The cells have a variety of different roles.
The cells have two different roles providing memes which the music utilises, or to provide feed back on the memes changing them slighty producing a composition of constant flow of evolving  textures.
When doug stops evolving, or loops his state , doug dies ,and the music stops. One of the main set backs modelling something biologically is that they are always doomed to died.
 Doug can produce a confused mass of screaming voices or a choir of angels singing in the praise of syncro-no-city, the cells role are to create trends and copy, no matter how beautiful or ugly they may be. Dougs purpose is just to produce constant change, to add to the noise . What his cells copy is irrelevant to them, they are only interested in following the latest meme and improvising on it. monkey see monkey do.
Doug doesn't see the point of this, endlessly producing stuff just to produce stuff. He is tired and depressed about his situation, all he wants to do is sleep, at least he gets a break from being worked to death. The start of doug is the end of doug. Even death isn't an escape. A new doug is resurrected different from the last, but by still using the same algorithms, is doomed to repeat the same mistakes of the previous Doug. like Escher's never ending stairs Doug is forever working his way out of a looped maze . Doug is very aware of this, thats another problem using biologically modelled programs, developing the illusion of free will leads to depression,


heres doug doing his thing




Doug's audio engine consists of the following specification.

Doug use 6 fm synthesisers, and eight channels of feedback.
the fm synths have a variety of sounds , the sounds can be bought up in volume and stereo effects applied. Also the sound can pass through a variety of audio effects

They produce patterns by having a bag of notes that are disturbed in a controled random manner with a random percentage of spaces and a random quantise applied.
a controlled random octave, at which the notes play, and a controlled random number of sequenced steps controls how the riff meme is developed. The audio engine's tempo can also be controlled by doug. The synthesiser can be started and stopped also with conrolled random conditions.
These notes then can be modulated by a chord appreciator, using a variety of chords.
Each one of the six synths can be applied a pattern which can be unique or be a varient/same as one of the others.
The synth sounds are then effected by the feed back engine. The feed back engine is a granular synthesiser. An audio buffer, (which can or may not be related to the bpm, causing dissonance) , is split up into an equal number of pieces and the individual pieces, played back in order, can have a behaviour applied to them, as it's an audio sample its pitch can be changed, it can be turned backwards,have volume or stereo effects applied to it, repeated, passed through effects units ,or the order can be randomised .
The resulting effect is that which ever "meme" is being operated on a new variant is produced.
There are also controls to halt the changing of controls.

each cell can control one or more of these variables, the community really gets involved.   


Examples of Dougs compositions can be found here