Freesound Forums

Hi ~ Just found this incredible FREESOUND site while doing some research on a project that has been plaguing me for a few years. You have most likely seen images of Photomosaics (by Silvers) on the cover of TIME and other major zines. Software was designed to break an image into groups of pixels, and then drawing from a library of other images, 'make up' the primary photo from the other pics, 'mapping' them in color, etc. to each group of pixels. I didn't explain that too well, so if you are not familiar with the process (and the result), go to www.photomosaic.com and have a look.

What I am wanting to do is to do the similiar in sound. I would like to draw from a sound library and match indidvidual sounds to particular notes in a musical piece. The effect I am looking for is that the musical piece would be instantly recognizable..... but, it would also be notably different, maybe in a hauntingly organic way. A screeching tire might become a violin note, a keyboard 'click' part of the percussions, etc.

My question is whether there is software on the market that can do this, either automatically or manually. I talked with a couple of programmers over the years but couldn't really get anyone interested in the project. I think it would have to be object based, but admittedly so, I am NOT too tech savvy.

I really have a desire to see this through, even if it is only for my own satisfaction, but not knowing the proper terminology, research has been difficult. Any advice is greatly appreciated. Thanks

Hello idattu,

You should talk to FS contributor Pitx, he is a past master at the sort of thing you describe.

http://freesound.iua.upf.edu/usersViewSingle.php?id=40665

Have a listen to

http://www.soundlift.com/band/music.php?song_id=215054

all from sounds obtained from a balloon.

Mike

the closest "pre-made" software I can think of is melodyne

http://www.celemony.com/melodyne/index.html

which allows you to work with audio data as if it was MIDI. Never tried it myself, but even there you'd have to pick the sounds by hand.

But best approach could be some sort of audio programming language like PD (http://puredata.info/) or MAX/MSP (http://www.cycling74.com/products/maxmsp), and program it yourself...

good luck

idattu
Hi ~ Just found this incredible FREESOUND site while doing some research on a project a

I work with a program called Reaktor, it can do almost anything with sound and mathematics and it's especially made for doing studio based synthesisers and sequencers and samplers.

first of all you have to be able to analyse a piece of music accurately into all the different musical parts and turn them into notation information. This would already take a programmer perhaps a year to do in a way that you would be satisfied with.

The best thing would be to take a MIDI file and use your own mosaic of samples on the notation it supplies.

Working with sound is much more complicated than working with colours, I would say that a really good programer would take perhaps three years to program what you mention in a way that would be satisfying.

Hello freesound,

I've a feeling this may be my first post... I signed up to post some thoughts on the zoom h4, but haven't got 'round to it yet...

zeroline
I work with a program called Reaktor, it can do almost anything with sound and mathematics and it's especially made for doing studio based synthesisers and sequencers and samplers.

first of all you have to be able to analyse a piece of music accurately into all the different musical parts and turn them into notation information. This would already take a programmer perhaps a year to do in a way that you would be satisfied with.

Hmm. I don't think you'd necessarily need to analyse the piece of music in terms of the different parts and notation... think of how it works in the photo version; they don't try to first process the image to work out what all the separate objects are. I guess what they do is chop the image into lots of little squares, run a frequency analysis on them (so far so similar to JPEG etc), and then search the database of images they intend to use for the mosaic for a close spectral match for each segment...

In terms of sound, this is somewhat similar to work being done by Michael Casey at Goldsmiths where I'm at uni, as well as related to some of Bram and other's work, too, it seems.

Working with sound is much more complicated than working with colours, I would say that a really good programer would take perhaps three years to program what you mention in a way that would be satisfying.

This would be considerably less than three years work for a good programmer, I think. As long as they don't try to implement it in reaktor although it could possibly be done, I suppose.
It's a nice idea, too. Good luck finding someone willing to work on it.