AirDeck mentioned in Electronic Gaming Monthly!

My AirDeck virtual theremin application was recently mentioned in Electronic Gaming Monthly and I was briefly quoted. Here is the excerpt from the article:

I am also posting a couple of photos that I sent to EGM to use in the article. EGM did not end up using them, but they are really great shots taken by my friend Omar Ramirez of Public Works Collective from my sit-down with them last year. Public Works Collective is a group putting together a film documentary chronicling the history of the Los Angeles electronic dance music scene. Here are the shots:

Summer!

Summer is here and along with it an opportunity to recharge, refresh and get working on some stuff that the extra time available from not having school will afford me. That includes more frequent posts to my blog here; I know I’ve lagged somewhat but this last quarter was pretty brutal for me.

Things I’d like to get working on this summer:

1. Make a couple of new mixes. Discogs is now a media partner with Juno, so they are an excellent source of digital media along with Beatport which I have also used in the past. I love vinyl, but there is no denying the ease of use and efficiency of time and cost that mixing with digital media offers. Reading some of the comments in various forums there seems to be some animosity by some DJs and vinyl heads towards the digital media and it’s a debate I’d like to engage in, but that is for another post… Feel free to post your thoughts in the comments if you have an opinion.

2. Work on a couple of remixes. There are a number of remix competitions out there, some of which offer some pretty cool prizes. I’ve done one so far, but I definitely want to get cracking on some more and continue polishing and refining my production efforts. The findremix blog is a great way to keep track of all of these competitions.

3. Do some more work with TouchOSC and PureData.

4. I downloaded the VST Software Development Kit from Steinberg a while back, which allows you to create your own virtual synth instruments. I just haven’t had the time to work with any of it. So I definitely want to take a more serious look at that.

5. Start reviewing some more music/DJ based apps for the Ipod/Iphone/Ipad here (Yes, I want an Ipad, dammit!). Like the vinyl versus digital format debates, there seems to be some apathy by DJs and producers towards these emerging tools. Again, I see no problem with looking for new tools to work with and technology is merely about providing new tools to work with. Again, another debate to post about in depth another time. But since I have an interest in developing apps myself I guess you could call me biased on these issues. Main thing preventing me from serious app development is Apple’s requirement that apps be developed on a Mac… which I neither have nor can afford any time soon. So I might have to look at starting with droid app development.

6. There are still a number of other music development platforms I have been exposed to recently including Max, CSound, Super Collider and something called the Synth Toolkit. Obviously, I can’t delve too deep into all of them, especially with all the other stuff I am trying to get familiar with, but definitely want to play around a bit with each of these. That’s always been one of my problems, overly ambitious with too broad a focus. Something that’s hindered me in the past, but something I am working to improve as I get older (and school has definitely helped with that!)

7. Last but not least, I would like to add a section to this blog that focuses on the history of electronic music. I’ve been fortunate to be exposed to a lot of interesting information over the years; stuff that I think would be interesting to share with other folks who are into synthesis, electronic music of all styles, music production, DJing, music technology and everything that goes along with all that. There are so many interesting precursors and influences to today’s electronic music that I believe are important to share. So maybe something like a wiki or just a “spotlight of the week” type thing.

MouseDeck

For the past few months I have had a number of people asking me if and when I plan on releasing the code to my AirDeck project. This is something I have been thinking about, but I have a number of concerns. I am not intending to make any money off this project (not even sure if I could), but I know that I would not someone else to run with the code and make money off of it either. Also, because I am using several APIs that were developed by other people, I am not sure what the licensing issues are. So, I am still deciding on how to proceed.

In the meantime, however, I have decided to post an early iteration of this project in applet form. This is a mouse based version, not using the Wii remote, as this is how I initially began coding and testing the project. I will be the first to admit this is somewhat buggy. I am also interested in seeing how this runs as a client applet on other machines over the internet. This is using a beta version of the JSYN api that is supposed to work without any plugin software. It works fine on my machine, but please leave me a note in the comments describing any issues you may encounter. Thanks and enjoy!

Also, this is a java applet, so java must be installed on your machine. Here is a link to the applet page:

MouseDeck

AirDeck… it lives!!!

Here is some video I shot shortly after my project presentation which demonstrates the AirDeck virtual theremin application I designed and explains some of the features. It uses the Wii remote as an input mechanism by tracking motion with Infrared LEDs. The AirDeck is written in Java with the WiiUseJ API for handling Wii remote events and the JSyn API for internal synthesis. It can control MIDI out as well as offering a simple DJ scratch interface for real-time manipulation of sound samples similar to a DJ scratching with vinyl records. After about a year of working on this project, I am very relieved that it is finished. I’ve learned a lot and am looking forward to possibly working on other similar concepts in the not too distant future.

Converting MP3s in Java

I was surprised to find that there was very little information on the internets on decoding MP3s for manipulation in Java, which is one of the things I would like to add to my project. After spending almost two weeks on this problem, I have finally cobbled together a solution, that while not perfect, gets close to a resolution. I would like to share my findings in the interest of helping any other struggling souls out there who are stuck. At this point, the main problem I am having is that certain large files are not being played in their entirety – only snippets are being played. I feel this is probably occurring somewhere in the logic of my array assignment; perhaps another pair of eyes could can spot something and post a tip in comments.

In any event, in order to get started, one needs a decoder class. There are two APIs that provide such classes, JLayer and Tritonus, which offers a plugin that can work with Java Sound. I used JLayer with MP3SPI as they have some documentation that serves as a good starting point.

Here is some code that exemplifies the process. This method takes a file name string in as a parameter and after defining an output format it is read through the decoder object and the resultant bytes are assigned to a ByteArrayOutputStream. After this, I send the stream as a byte array on return for further manipulation.

public byte[] testPlay(String filename) throws UnsupportedAudioFileException, IOException {
ByteArrayOutputStream f = new ByteArrayOutputStream();
File file = new File(filename);
AudioInputStream in = AudioSystem.getAudioInputStream(file);
AudioFormat baseFormat = in.getFormat();
AudioFormat decodedFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED,
    baseFormat.getSampleRate(), 16,
    baseFormat.getChannels(), baseFormat.getChannels() * 2,
    baseFormat.getSampleRate(), true);
DecodedMpegAudioInputStream decoder = new DecodedMpegAudioInputStream(decodedFormat, in);
try {
    byte[] byteData = new byte[4];
    int nBytesRead = 0;
    int offset = 0;
    while (nBytesRead != -1) {
        nBytesRead = decoder.read(byteData, offset, byteData.length);
        if (nBytesRead != -1) {
            int numShorts = nBytesRead >> 1;
            for (int j = 0; j < numShorts; j++) {
                f.write(byteData[j]);
            }
        }
    }
} catch (SynthException e) {
    e.printStackTrace();
} catch (IOException e) {
    e.printStackTrace();
}
byte[] buffer = new byte[f.size()];
buffer = f.toByteArray();
f.close();
return buffer;
}

The above method is called like so and I proceed to assign the byte array as a short array so that I can feed it into the format that synthesis API I use, JSyn, prefers.


stream = (InputStream) (new FileInputStream(fileName));
String ext = fileName.substring(fileName.lastIndexOf('.')+1, fileName.length());
if (ext.equalsIgnoreCase("mp3") ) {
    byte[] buffer = testPlay(fileName);
    int j=0;
    short shrtData[] = new short[buffer.length];
    for (int i = 0; i < buffer.length ; i++) {
        int sampled = ((int)(buffer[i])) & 0x00FF;
        sampled += ((int)(buffer[i++])) << 8;
        shrtData[j++] = (short) sampled;
}
if( shrtData != null ){
    sampleTable.allocate( shrtData.length );
    sampleTable.write( shrtData );
}
Hope this is helpful to someone out there…

Project Presentation Day Selected

I will be presenting my project on December 4th. I have been furiously trying to wrap things up. I still have a few bugs here and there to resolve as well as trying to add a few of the last features that I wanted to get in. For example, I am still trying to get the scratch function more accurate. I am also working on trying to decode MP3 files for real time manipulation. In addition to WAV or AIFF files.

Most of the features on the Synth side have been completed. I still have to finish the Infra-Red gloves. Expect some video within the next week or so of where things are at.