Lee Goddard .net — another boring personal homepage

Plonk — Like Plink, in two and a half mornings

Sunday

When looking for something to impplement using HTML5 Web Sockets, other than a chat server, I cam across , by the Swedish compnay, DinahMoe, and so ripped-off the idea.

When watching Plink, it seemed obvious to implement it as ... a chat server, in as much a number of users are linked to a persistent-state server, that relays each users messages to all other users, in real time. The interesting thing about Plink is that the messages are musical sounds.

Monday: Relay Server

The first task was to write or find a Web Socket relay server. Node.js seemed the obvious choice, but support for Web Sockets in Node.js is patchy, and as I wanted to play with the latest version of the specification, I had only the web-socket library to look at. Unfortunately, this failed to build on OS X, and the support response was 'your need Xcode,' which I have had since buying the Mac.

I thought of trolling through the Maven repositories for something suitable, but on my way there checked what Perl had to offer. After getting side-tracked by the usual Perl half-baked or badly documented implementations, I found that Mojolicious has a simple WS daemon that can run out of the box. It took literally five minutes to install the package and have the relay up and running – a fraction of the time it took to find the library.

#!/usr/bin/perl 
use strict;
use warnings;

use Mojolicious::Lite;
# use Data::Dumper;

my $clients_tx = {};
my $clients_cursors = {};

websocket '/' => sub {
	my $self = shift;
	my $client_id = sprintf "%s",$self->tx;
	$clients_tx->{$client_id} = { tx =>$self->tx };

	$self->on(message => sub {
		my ($self, $msg) = @_;
		# warn $msg;
		my ($x, $y, $pitch) = split/,/, $msg;
		if (defined $y){
			$clients_cursors->{ $client_id }->{xy} = [$x+0, $y+0];
			$clients_cursors->{ $client_id }->{pitch} = $pitch ne ''? $pitch+0 : undef;
			# warn Dumper( $clients_cursors );
			for my $i (keys %$clients_tx) {
				$clients_tx->{$i}->{tx}->send(
					Mojo::JSON->new->encode({ 
						cursors => $clients_cursors
					})
				);
			}
		}
	});
	
	$self->on(finish => sub {
		delete $clients_tx->{$client_id};
		delete $clients_cursors->{$client_id};
	});
};
app->start; 

__END__

The code simply receives a CSV of three numbers representing the cursor position and selected pitch, which it stores for each connected client. Every time a client sends this information, the server responds with the latest copy of the information for all clients.

Prototype 1

Screenshot
It's not easy to take a screenshot whilst playing with two mice.

The first stage of the prototype was boiler plate – setting up an Apache virtual server, creating directories to hold JS and CSS, and a basic HTML5 document to pull in Mootools, Modernizer, the blank application CSS file, and a fresh Mootools class definition, the latter linked to an element in the body of the page into which the app could insert itelf.

Capturing mouse movement events is straight forward, and once their offsets are removed, I was ready to send the co-ordinates to the server. When jotting down the server code, above, I had already decided to send both co-ordinates and pitch information, as I wasn't quite sure how I would implement the functionality of the app, and wanted to leave my options open. I've been a victim of premature optimisation in the past.

Next I had to find the Web Socket API. There are many examples of this lying around the net, and it took only a minute to implement the necessary callbacks, and have the server logging connections and messages, and the client console logging what it was sending and receiving.

After adding to my websocket .message callback some code to render a circle at the co-ordinates specified in the message, I could see a mess under my cursor whenever I moved it.

Next came my only implementation error – at least, the only one I have noticed.

Plink shows the current note the user is playing as a circle on the screen, but also shows past notes as scrolling horizontally away from the cursor: very pretty. I remember implementing horizontal scrolling for games on a Vic-20 in the early 1980s, and the fastest way of doing it then was to shift a block of memory by the number of bytes to be scrolled, an operation easily and quickly done even then, by a ten-year old.

HTML5 is not so simple, and the canvas element still lacks a built-in scrolling function. This left four choices: two involved dropping the canvas, in favour of either SVG or WebGL; I've played with both, and found basic operation in both to be similarly straight-forward: in both cases, I would for every 'note' create an object that could be rendered. The other option was to do this for the canvas, and wipe it between every rendering, or to attempt to copy the canvas element, wipe the original, and render the copy one pixel to the left.

My thoughts were that the latter would be the simplest, and closest to the pattern I had learnt as a kid.

scroll: function(){

var destinationCanvas = this.canvas.clone()

destinationCanvas.cloneEvents( this.canvas, 'mousemove');

var destCtx = destinationCanvas.getContext('2d');

destCtx.drawImage(

this.canvas,

(this.options.scrollPx)*-1,

0

);

destinationCanvas.replaces( this.canvas );

this.canvas.destroy();

this.canvas = destinationCanvas;

this.ctx = destCtx;

},

I hooked the above method up to a timer, via Mootools .periodial method, and was mildly pleased to see a trail from cursor, disappearing off the edge of my screen.

Prototype 2

Before the day ended, I wanted to hear sounds, and as I browsed for API notes and implementation examples, I noticed my fan buzzing, and Mozilla getting sluggish. I flicked, slowly, through my tabs, and found the Plink rip-off crawling. I killed the server, closed the tab, and everything was fine. I still don't know exactly what the problem was: I tried minimising the rate messages were sent to the server, and the frequency of canvas renderings, but in the end went to have tea, and gave up for the day.

Tuesday

For some reason I felt an attachment to the canvas copying method, but dropped it all the same, and had the .message handler push the latest batch of JSON objects onto a stack. I then rewrote the periodical scroll method to render everything on the stack, making the latest object appear on the right, decrementing everything by one cursor width, and dropping from the stack anything that would fall the screen. This removed the performance problem, and reinforced my desire for a native canvas.scroll method.

Web Audio API

In my experiments writing an audio sequencer for SoundCloud files, I learn the limitations of the HTML5 audio element – it is to the Audio API as the video element is to the Web Graphics Library. So, I found a decent introduction to the Audio API, on HTML5Doctor, and had sound within a few minutes, hooked up in a new periodical method. I had the .send method calculate pitch by computing within which of a number zones the vertical position of the mouse fell, and then spent an hour find sounds in Logic, and exporting individual notes as wav files, loaded into buffers stored in arrays, whose indexes could be accessed by the 'pitch' index sent to the server.

Plink has a percussion track, so I extended the sound-firing method to play the drums as appropriate:

options: {

percussion: {

kick: [[4,1]],

snare: [[8,1], [16,2]],

hat_closed: []

}

}



self.percNames.each( function(instrument){

self.options.percussion[instrument].each( function(i){

console.log( instrument +' '+ self.pulseNumber +' '+ i[0] +' '+i[1]);

if (self.pulseNumber % i[0] == i[1]){

self.playNow( self.percBuffers[instrument] );

self.scaleCursor += self.options.cursorScaleIncrement;

}

});

});

Notice that routine also changes the cursor size, as in Plink, so the visual echo of the sound reflects the unerlying rhythm.

My children then took over the development machine for beta testing, and I joined in on the old media centre PC, on which I had to install Chrome, which seems to be the only limitation of the project, other than my inability to find a free host for my web-socket relay code: my current ISP is too cheap to provide this.

I was quite pleased with the effort, and dropped a line to the address Dinahmoe publicised for job applicants, but no word yet: perhaps they realise Plink isn't that hard to do. Whilst I was there I had a quite look at the Plink source code, and was only mildly horrified, mainly by the amount of hard-coding, lack of framework, and lack of white-space.

Wednesday

I then added the ability to change patches, as in Plink, as well as some options to have the horizontal cursor position equate to volume and panning – two options the children found far from intuitive, and quite intrusive into their play.

Also added the ability for patches to only sound if pre-required, much in the manner of the percussion track.

Node.js

Node.js is beautiful: clearer and more natural than Perl, faster to write and install than Java, and performs as well as both. The following Node.js websocket server took ten minutes to write, based of the stub code that came with the module. The only change made to the Plond.js client was to add a sub-protocol argument to the WebSocket instantiation.

#!/usr/bin/env node
var WebSocketServer = require('websocket').server;
var http = require('http');

var cursors = {};

var server = http.createServer(function(request, response) {
    console.log((new Date()) + ' Received request for ' + request.url);
    response.writeHead(404);
    response.end();
});
server.listen(3000, function() {
    console.log((new Date()) + ' Server is listening on port 3000');
});

wsServer = new WebSocketServer({
    httpServer: server,
    autoAcceptConnections: false,
    maxReceivedFrameSize: 50
});

function originIsAllowed(origin) {
  // put logic here to detect whether the specified origin is allowed.
  return true;
}

wsServer.on('request', function(request) {
    if (!originIsAllowed(request.origin)) {
      // Make sure we only accept requests from an allowed origin
      request.reject();
      console.log((new Date()) + ' Connection from origin ' + request.origin + ' rejected.');
      return;
    }

    var connection = request.accept('sec-websocket-protocol', request.origin);
    console.log((new Date()) + ' Connection accepted.');

	var id =  connection.socket._peername.family
		+':'+connection.socket._peername.address
		+':'+connection.socket._peername.port;
		
    connection.on('message', function(message) {
    		if (message.type === 'utf8') {
            // console.log('Received Message: ' + message.utf8Data);

            var csv = message.utf8Data.split(',');
            cursors[id] = {
            		userId: 		 csv[0],
            		xy: 			 [ parseInt(csv[1]), parseInt(csv[2])],
            		scaleCursor: parseInt(csv[3]),
            		gain:		 parseInt(csv[4]),
            		pain:		 parseInt(csv[5]),
            		patch:		 parseInt(csv[6]),
            		pitch:		 parseInt(csv[7]),
            };
            connection.sendUTF(JSON.stringify({ cursors: cursors} ));
        }
        else if (message.type === 'binary') {
            console.warn('Ignoring received Binary Message of ' + message.binaryData.length + ' bytes');
            connection.close();
        }
    });
    
    connection.on('close', function(reasonCode, description) {
        console.log((new Date()) + ' Peer ' + connection.remoteAddress + ' disconnected.');
    		delete cursors[id];
	});
});

Finally, I moved the patches into objects, along with their individual pulses, for ease of legibility and maintenance.

In hindisght, the cross-platform features of MooTools were unncessary, as the code will only run where Web Audio is available, which seems to be only Safari and Chrome. Still, I think it does allow for more readable code, and has no performance impact (unless it was responsible for that canvas copying slow-down).

Next

Links