Entries in simulation (3)

Sunday
Feb282010

Yamaha Vocaloid robot singer (with robot piano)

The embedded video shows a robot powered by the Vocaloid “voice synthesis” software. The movements and expressions were pre-programmed, but the generation of the singing voice from lyrics and score can be done in real-time apparently.

The piano is a Yamaha Disklavier of course, being controlled by the same program that is controlling the Vocaloid robot. This is a good example of state-of-the-art synthesis technology, and sounds natural and human to my ears. 

The inputs for Vocaloid are: 

  • phonemes (lyrics)
  • vocal line (music) 
  • expression (stresses, vibrato, etc.)

The inputs for Disklavier are:

  • precise key presses
  • precise pedal controls

Other inputs for the robot shown in the video are for mouth, face and head movements.

Wikipedia: Vocaloid (Crypton Future Media)

The Character Vocal Series is a computer music program that synthesizes singing in Japanese. Developed by Crypton Future Media, it utilizes Yamaha’s Vocaloid 2 technology with specially recorded vocals of voice actors. To create a song, the user must input the melody and lyrics. A piano roll type interface is used to input the melody and the lyrics can be entered on each note. The software can change the stress of the pronunciations, add effects such as vibrato, or change the dynamics and tone of the voice.

The series is intended for professional musicians as well as light computer music users. The programmed vocals are designed to sound like an idol singer from the future. According to Crypton, because professional singers refused to provide singing data, in fear that the software might create their singing voice’s clones, Crypton changed their focus from imitating certain singers to creating characteristic vocals. This change of focus led to sampling vocals of voice actors.[11]

Each Japanese Vocaloid is given an anime-type character with specifications on age, height, weight, and musical strengths (genre, pitch range and ideal tempos). The characters of the first three installments of the series are created by illustrator Kei.

Any rights or obligations arising from the vocals created by the software belong to the software user. Just like any music synthesizer, the software is treated as a musical instrument and the vocals as sound. Under the term of license, the Character Vocal Series software can be used to create vocals for commercial or non-commercial use as long as the vocals do not offend public policy. In other words, the user is bound under the term of license with Crypton not to synthesize derogatory or disturbing lyrics. On the other hand, copyrights to the mascot image and name belong to Crypton. Under the term of license, a user cannot commercially distribute a vocal as a song sung by the character, nor use the mascot image on commercial products, without Crypton’s consent.

It is encouraging to read that “professional singers refused to provide singing data” because, as you were warned in The Performing Pianist (a pocket primer, 2002), page 27 (Digital vs. Analog):

Pianists are for the most part analog creatures. They may be transformed into digital entities by such things as the glorified player piano, the Yamaha Disklavier. This hybrid machine digitally and precisely captures the manipulations of the keys and pedals, also known as performance capture, which can then be reproduced exactly, ad infinitum, on the analog piano. A lazy individual could purchase and not play this instrument, thus avoiding the need to hire (or bribe) musicians, take lessons, or even go to concerts. This is not in the best interests of the art form. You should consider carefully the degree to which you agree to be digitized…  

RELATED: Fauxharmonic Orchestra - The Robot Philharmonic


Wednesday
Oct072009

Undead Music: Zenph Studios (Performance Re-creation) (2008)

From the Archives: November 2008



In his TED presentation, you will see John Q. Walker demonstrate his “performance re-creation” technology to an uncritical, naïve audience.

“Imagine hearing great, departed pianists play again today, just as they would in person. At the 2007 EG conference, John Q. Walker shows how recordings from the likes of Glenn Gould and Art Tatum can be analyzed for precise keystrokes and pedal motions, then played back on computer-controlled grand pianos.”

Imagine… Undead Music.

This is not about the music or the musicians. It is about a fetish for dead, perfect things reproduced with ultimate fidelity for your greedy, selfish pleasure.  Making music is a thing of the past, and your music needs are fulfilled by corporations who pander to this fetish for profit. You faithfully purchase the regurgitated masterworks, and listen to them on your hi-fidelity entertainment system.

Meanwhile, Glenn Gould and Art Tatum haven’t given their permission for this use. That is to say, owning the rights means one doesn’t need,  their permission. You pay a premium to the rights holders and they can continue to hold the monopoly forever. Pay close attention to what John Q Walker says during the TED talk. Not many people are going to fork over $50,000 to buy a Yamaha Disklavier to listen to piano music.

Copyright is forever. Public Domain be damned. Lock the vaults, add minor value to the works, and presto you have another monopoly for another century. This is from the Zenph Studios site, where they address “Labels and Studios” (emphasis mine):

The Diversity of Copyright Laws

The USA has strong copyright laws; sound recordings essentially don’t go into the public domain until well into the 21st century. But, in the European Union (EU), for example, recordings go into the public domain 50 years after their first release. Small recording companies in the EU already re-issue CDs of historical mono recordings in volume. That’s been a small concern to the labels, but in 2006 the situation gets troubling. 1956 was the start of early stereo, which is how we still listen nowadays. Starting in 2006, the “good stuff” from 1956 forward starts going into the public domain. Year by year, labels will lose European rights to the most prized, profitable recordings in their archives. With global retailing, CDs made in the EU are readily available anywhere.

The way around this is to create new, highly-desirable music recordings, which establish a new copyright. A modern re-recording can be a premium product, protected with the latest Digital Rights Management (DRM). For a modern re-recording to be acceptable to discerning jazz, classical, and pop listeners, it must be faithful, note-perfect, and identical to the original performance. That’s our business.


Where to begin? The fact that copyright isn’t forever is troubling to these people. These people are not musicians as much as they are businessmen, audio and software engineers and technologists.

“Faithful, note-perfect and identical” doesn’t describe most live performances. Perhaps he is referring to re-recording of studio performances.  Either way, he’s uniquely positioned to make sure that the “public domain” of mid-century works never happens, those who resist are turned into “pirates.”
 
Musicians not needed. Live performance not needed. Perfection is the standard, and very few humans make the grade. iPod. iGod. Don’t you feel special “owning” all this music? Yes, it’s yours! At least until the device breaks, the DRM expires, or the format is no longer supported. Then you can look forward to the next trendy offerings from the corporations. Perhaps smell-o-vision and 3d moving imagery to accompany the music.

Wednesday
Oct072009

Fauxharmonic Orchestra - The Robot Philharmonic

I’ve been following developments like GigaStudio (see below) for years. It was only a matter of time before a service such as this became available.

I wonder how much effort is spent tailoring each simulated performance, and whether the tailoring is captured and modeled as “training the robot conductor?”

An intelligent robot conductor gets better and better at performing unfamiliar music, as you train it to better perform specific music.

This requires modeling composers. This clip (the second part) is the result of teaching a robot conductor to better perform Beethoven. Note: the human in the video only sets the tempo and cues starts and stops. All of the difficult robot training must be done beforehand.

You don’t want its Beethoven training to make all its performances more Beethoven-like. You want to break down the Beethoven training as training in the Classical and Romantic styles. This way the training common to the Romantic style will improve its performance of unfamiliar Romantic music. What is unique to Beethoven, such as sudden muscular, peasant-like sections for example, blend in with the general training. Given the same music, if you don’t say Beethoven those sections aren’t quite so muscular or peasant-like.

Then you can play with scenarios like “pretend the conductor has been lied to and thinks piece X was written by composer Y in style Z.” All of these questions are moot, since I’m quite sure the process and results are heavily guarded secrets. Proprietary technology with a combination of “intelligent training” and manual labor.

Film music rates – per minute (five minute minimum)

25+ instruments: inquire
15-25 instruments: $400
5-14 instruments: $200
1-4 instruments: $100


TV spot rates
(:30, :60 or :90)

25+ instruments: inquire
15-25 instruments: $700
5-14 instruments: $350
1-4 instruments: $175


Concert music rates
– per measure (20 measure min.)

25+ instruments: inquire
15-25 instruments: $5
5-14 instruments: $3
1-4 instruments: $2

The prices are high, but not unreasonable for such quality. They do give estimates, which is where they may share savings when the “intelligent training” reduces the manual labor for a given piece. Conversely, if you hand them an intellectual exercise in opaque experimental notation you will no doubt pay for it.

They also have a call for scores, which they may decide to program at a future concert. They don’t say whether accepting your score for a concert gives you anything other than exposure. The “performance right” of your work is their valuable (and presumably costly) intellectual property. They do say they won’t do anything without your permission — so if you are that rarest of breeds, the symphonic composer, it doesn’t hurt to send them a copy of your symphony.

Fauxharmonic Orchestra 20090520 Beethoven Symph. no 1 mvt. 1

Today you can have excellent sampled sound driven by your music production software — and use the Fauxharmonic service to record your masterpiece with the money you earn selling that commercial jingle or karaoke pop version of whatever aria Andrea Bocelli is singing lately.

 

Giga is joining Garritan. We’re excited about this and proud to own some of the best sampling technologies on the planet. One of our goals is to provide the best music-making tools possible to our users, and acquiring the Giga technology helps us to achieve this…

We are evaluating how to incorporate Giga technologies in our future products. We will extensively develop the Giga technologies and take them to the next level. The Giga software product line ended in the summer of 2008, but we plan to continue the legacy of these products in new forms…

We realize that many Giga users have a great deal invested in Giga libraries. It is our hope that those who invested in Giga libraries will be able to continue to use them in our future products. Owning the Giga format puts us in the ideal position to provide native solutions for Giga files…

Garritan will continue to offer its libraries based on ARIA and the open SFZ format. For the foreseeable future, many of the real-time performance tools from Giga can fit into ARIA’s modular architecture.

 

Fauxharmonic Orchestra 20090520 Beethoven Symph. no 1 mvt. 1