May 2012 Archives

May 31, 2012

MIDI Synth example built using Web Audio API + midibridge (Java applet)

Midiwasynth

Seeing as MIDI was the recent topic of discussion, it seems only fitting to show off this cool implementation of a synthesizer, which was built using the Web Audio API and implements MIDI via midibridge, a Java applet.

I was able to setup a Network MIDI connection on my Mac and play the synth -- albeit with considerable latency and sluggishness -- using a free MIDI controller on the iPad called Pad MIDI (App Store link) and with the $1.99 AK-7 Core MIDI Keyboard (App Store link) app. I don't know if the latency was due to the controller software app on the iPad, WiFi-based interface, or Java, but just the fact that there is a solution for building MIDI-capable instrument in a web browser today is pretty exciting!

May 30, 2012

MIDI API Editor's Draft posted

W3C logo

Recently, the W3C Audio Working Group agreed that MIDI functionality should be included as part of HTML5. Today, the Working Group has posted the first Editor's Draft of the new MIDI API. Comments regarding the draft or implementation of the MIDI API (or the Web Audio API) can be emailed to public-audio@w3.org.

May 29, 2012

Angry Birds Chrome uses Web Audio API

Angry Birds

Stumbled across the unexpected news that Angry Birds Chrome shipped in late January with Web Audio API support. The sound for Angry Birds isn't exactly what I would deem complex usage, but certainly it couldn't have been done easily or reliably with the <audio> tag and it's indeed exciting to see a shipping product build using these new tools. Even more interesting is that the user can try playing using different sound engines, as noted in the article here:

The responsibility of choosing the appropriate audio API for the game's sound is (mostly) left up to gwt-voices, which chooses the audio API that will give the best experience. If you'd like to hear how other audio APIs perform, you can ask gwt-voices to try to use the Web Audio API, Flash, HTML5 Audio, or even native audio. Your mileage will vary by browser and platform and which plugins you have installed. Also, gwt-voices will select the best available fallback, if the desired audio API is not going to work at all in your environment.

May 28, 2012

Audio APIs - A Discussion

Jens Nockert posted an interesting (and low-level) discussion of the two competing audio specs that are currently being used to build the new W3C official audio API. It seems to me that Nockert prefers the Audio Data API approach, since the Web Audio API's functionality could be built within the lower level structure and not vice versa.

Would Audio Data API be better? I don't know. From the perspective of a game audio guy, Web Audio API appears to provide what is necessary to build a decent audio engine. But from the perspective of experimental music composers, Audio Data API is likely to present far more flexibility. It's a good question and one worth discussing, since work done in the next two years is likely to stick with us for quite some time and not dramatically expanded in the next decade.

Timbre: A JavaScript Library for Objective Sound Programming

Just received this neat example of a sound programming library that works in both Chrome and Firefox. There's a bit of example code, although the page doesn't really explain how the back end works. Also, it seems to work a little more smoothly in Firefox than it does in Chrome, although I've noticed that Chrome tends to introduce some noise in all Web Audio API playback.

Thanks to Kevin Ryan for the contribution!

May 23, 2012

A tone generator built using the Web Audio API

Just stumbled across this excellent little example of a simple tone generator built using the Web Audio API. It works only in Chrome and supposedly in Safari, although I still haven't succeeded in getting any Web Audio API code to work even in a nightly of WebKit.

The article includes a thorough walkthrough of how to do the math necessary to generate tones, work with sample rates, and a lot of what you might need to create interesting results.

Nicely enough, the entire code is available on GitHub!

Google's Moog Doodle falls back to Flash

Moog doodle

As reported yesterday, Google created a doodle in honor of Bob Moog's 78th birthday.  The doodle caused a sensation throughout the Internet as Google users enjoyed a fully capable synthesizer embedded directly within their web browser.

What intrigued me, however, was the fact that an initial look at Google's JavaScript code indicated they were utilizing the new Web Audio API, which has yet to be supported anywhere other than Google's own Chrome browser.  Yet, reports came in that the doodle works great pretty much everywhere.  (It doesn't work on my iPad, though.  I would have to assume that Android devices also are similarly mute, although I'd love to hear confirmation of this from Ice Cream Sandwich users who have the Android version of Chrome.)

So the big question is how did Google make this work on browsers that don't yet support the Web Audio API?  After all, the <audio> tag is not functional enough to do anything even approaching the level of sophistication being demonstrated here.

The answer is simple:  the doodle employs a Flash fallback.  Google tests to see which browser you're using, as well as which version of Flash is present, and utilizes the appropriate option.  Hence the reason why I was unable to use the doodle in Safari or Firefox, as Click2Flash and FlashBlock, respectively, were preventing the invisible Flash object from launching.

If the browser supports webkitAudioContext(), it is used.  If an error is generated, an <iframe> is created and an HTML page containing an embedded Flash swf is written into that <iframe>.

You can see the beautified code here:

var Td = function (a) {
		this.b = [];
		this.k = a;
		if (this.e = "function" === typeof webkitAudioContext) a: {
			try {
				var b = new webkitAudioContext
			} catch (c) {
				break a
			}
			for (a = 0; 4 > a; a++) {
				var d = [new Bb(b, 0.46, 4, 4, 0, i, l, l, 0, i, 0.05, 0, 0.4, 1),
						new Bb(b, 0.82, 4, 16, 0, i, l, l, 0, i, 0.05, 0, 0.4, 1),
						new Bb(b, 0.46, 4, 2, 0, i, l, i, 0.6, i, 0.05, 0, 0.4, 1)],
					e = new xb(b, 2100, 7, 0, 0.8, 0);
				this.b.push(new Ac(b, d, e, 0.82))
			}
			a = new Rd(b, 20, 2E4);
			this.c = new yb(b, this.b, a);
			Sd(this)
		} else if (qb = u(this.l, this), mb && 0 <= na(nb, "10.0.0.0")) {
			D = document.createElement("iframe");
			D.name = "doodle-moog";
			D.style.position = "absolute";
			D.style.top = "-150px";
			D.style.border = 0;
			D.style.width = "1px";
			D.style.height = "1px";
			document.getElementById("hplogo").appendChild(D);
			E = D.contentDocument;
			if (E == g || E == j) E = D.contentWindow.document;
			E.open();
			E.write('<html><head></head><body><object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=9,0,0,0" width="0" height="0" id="doodle-moog-flash-1" type="application/x-shockwave-flash"><param name="movie" value="/logos/swf/moog12-2.swf"><param name="allowScriptAccess" value="always"><object id="doodle-moog-flash-2" type="application/x-shockwave-flash" data="/logos/swf/moog12-2.swf" width="0" height="0"><param name="allowScriptAccess" value="always"></object></object></body></html>');
			E.close()
		}
	};

So, there you have it. Mystery solved. If you'd like to peruse the code yourself, you can skip all the tracing and just go straight to this page.

May 22, 2012

New Google Doodle uses Web Audio API

Moog doodle

Today, Google posted a Moog synthesizer doodle that can be played by the user in the browser. The doodle utilizes webkitAudioContext(), part of the still-under-consideration Web Audio API, and therefore only works inside Google's Chrome browser.

The doodle also includes a 4-track recorder, which demonstrates pretty effectively what will be possible when audio functionality in HTML5 becomes truly usable.

Update: It appears that the Google Australia version works in Firefox, too.

Update 2: You can find the JavaScript code here. Paste the code into jsbeautifier.org to make it a bit more readable.

Update 3: I've received several emails today informing me that the doodle works fine in all browsers. When I have a chance, I'll delve into how the code does this, since only Chrome truly supports Web Audio API at this time.

Update 4: I have replaced the link with the archival version.

May 14, 2012

<audio> playback in Android

Android

According to a recently linked-to article on gameindustry.biz, audio support in HTML5 is merely a little annoying, but "works well in most browsers". What the article fails to mention is that Android devices fail to support the <audio> tag in any meaningful way. While the Froyo and Gingerbread versions of Android's browser do indeed support the <audio> tag, there are no codecs included, and therefore the browser is unable to play back any audio whatsoever.

While the Ice Cream Sandwich version of Android does appear to finally bring audio capability to Android, that doesn't really help developers a whole lot. According to this chart, Ice Cream Sandwich makes up fewer than 4% of Android devices, while Gingerbread and and earlier versions account for more than 90%!

Broken HTML5 audio labeled "an annoyance"

This article from last week calls out some of the issues developers face using HTML5 and the hurdles to achieving wider adoption.  The article has this to say about audio in HTML5:

Games without sound are usually no fun; therefore HTML5 must be able to handle all kinds of audio correctly and allow developers to work their magic. HTML5 sound works well in most browsers, but we have found limitations, especially where it comes to looping music. Another problem is that sound within an HTML5 application cannot be muted by the hardware volume buttons on the iPhone, forcing players to rely on onscreen software buttons. This will almost certainly be fixed in the near future, but for now it is an annoyance that can break user engagement with a game.

Anyone who has attempted to implement audio in HTML5 projects has certainly found that web audio is far more than "an annoyance".  It does not work well in most browsers as the article claims.  All major browsers do not support one audio codec among them (as noted in this previously linked post), playback & manipulation is absolutely broken and barely functional, iOS devices can play only one sound at a time, Android cannot play sound at all due to having no included codecs!  (Android users with the Ice Cream Sandwich version can now get Chrome for Android, which does play sound, but given that covers fewer than 4% of the active user base, it's not much help to developers.)  Unless all you want to do is make a simple music or single-shot sound player, HTML5 audio is a non-starter.

If you need to trigger multiple sounds in-sync with on-screen or user-controlled actions, you are likely to encounter both a big headache and severe limitations. Those limitations will probably leave you running back to Flash for its far more functional audio support, leaving mobile devices utterly stranded.

The article continues on though, discussing the problems developers must address when it comes to visual presentation:

While audio is certainly an issue, addressing different screen sizes and pixel shapes is much more pressing.

Imagine for a moment that you're a developer, working on the next awesome game that will set the world on fire. You've begun developing your game in HTML5 for desktop browsers because you've been told that adapting it to mobile is easy. Your code can quickly be taken to a mobile device by simply scaling down your graphics with little to no hassle. Turns out this is mostly wishful thinking.

By going this route, you'll end up with a game that looks terrible and requires a lot of extra work in order to make it run correctly.

Somehow, "a lot of extra work" is a terrible thing for visual elements, but "utterly broken audio" is nowhere near as severe.  Additionally, "almost certainly be fixed in the near future" — the article's assertion about the fixes necessary to make web audio more engaging — means waiting what will likely be a minimum of 12-24 months for adoption of the Web Audio API, browser updates, and feature parity across all platforms.

Given that the Web Audio API isn't even a completed nor adopted proposal, the possibility that functional audio support will exist in the next two years seems extremely unlikely, if not downright impossible.

Thanks to Julian Kwasneski from Bay Area Sound for submitting the gameindustry.biz article.

HTML5 Video Guide provides tutorials on audio, plus testing code

html5videoguide.net, an excellent resource for both video and audio on the web, has several great pages worth perusing, including a page that tests codec compatibility and information about both Web Audio API and the Mozilla Audio Data API.

May 13, 2012

Web Audio API in Safari, plus examples

So you want to try out the Web Audio API, right?  And it only works in Google Chrome, right?  Wrong!  It can work in a nightly build of Safari WebKit, but you must do a few things first.  Thankfully Google has a page with instructions on how to configure the nightly builds of WebKit to work.  The same page also links to a bunch of demos of Web Audio API in action!

Web Audio FAQ

In January, Boris Smus provided an FAQ to address common questions in implementing the new Web Audio API.  It discusses playback, microphone input, and more.  Check it out!

May 11, 2012

HTML5 Audio and Flash (In a Tree)

Scott Schiller, creator of SoundManager 2, posted an interactive slideshow covering the state of HTML5 audio as of February 2012, as well as the history of how we got here.

Use the left and right arrow keys to navigate the slideshow.

Emberwind: Flash-free audio

Erik Möller wrote up an outstanding article about developing Flash-free audio in HTML5 using audio sprites in JSON.  The article gives some great examples and even demonstrates multiple channels of audio playback in their running prototype!

SoundManager 2 HTML5 audio with Flash fallback

Given the (currently) very limited capabilities of HTML5 audio, one might decide it is necessary to fall back to Adobe Flash in order to create a rich experience.  SoundManager 2 is a JavaScript-based solution that provides all that and more.

Variable-length audio sprites

Chris Heilmann at Mozilla has a tutorial and code samples discussing how to implement variable-length audio sprites.  Worth a glance, at the very least for the links to existing JavaScript solutions such as SoundManager 2.

HTML5 Audio APIs - How Low can we Go?

Mark Boas has an excellent piece discussing the differences between the Web Audio API and Mozilla's Audio Data API.  The upshot is that while Web Audio API provides something developers can begin to tinker with now, Audio Data is a much more low-level API that would really benefit middleware providers and those wishing to truly manipulate audio.

Using audio sprites in HTML5

Since the Web Audio API isn't terribly useful at this time, yet implementation of audio is still necessary, it seems the only somewhat elegant method of providing sounds to play in HTML5 applications is by creating the audio equivalent of sprites.  This article covers the concept and an implementation with sample code.

Mixing Positional Audio and WebGL

For a truly rich interactive audio experience in games, an audio engine must be capable of playing sound in relation to objects and their movement around a three-dimensional space.  Enter positional audio, which using the Web Audio API, can be implemented in HTML5 in the few browsers that support the not-yet-adopted API.  Here's how it works.

Developing Game Audio with the Web Audio API

Another outstanding Web Audio API tutorial from Boris Smus, this time covering many features necessary to create a rich audio experience in HTML5 games.  The tutorial covers music, 3D positional sound, sound effects playback, and more.

Getting Started with Web Audio API

Boris Smus has written an outstanding tutorial demonstrating how to use the new (and not yet adopted, barely supported) Web Audio API.  The API utilizes webkitAudioContext() to build a very functional audio playback system that could be used to build interactive audio engines.

Unfortunately, support for the new Web Audio API is limited mainly to Google Chrome.  The article indicates that Safari can support the API, too, but I've not yet found a way to make that happen.

jPlayer's Inspector shows when JavaScript events are triggered

One of the more difficult things to evaluate when troubleshooting HTML5 audio playback is which events are firing and when.  This jPlayer example has an Inspector included (be sure to display it by click "Show".  Then, when you work with the player controls, you'll see exactly what's happening!

Browsers and format compatibility, charted

Wondering which audio formats you can use on which browsers?  This set of charts illustrates and describes support quite clearly.

Simple audio player in HTML5

Lots of audio-related things don't work in HTML5, but a basic music player is something HTML5 can do.  Here's an excellent tutorial describing precisely how to create your own.

May 10, 2012

No streaming in HTML5 Audio?

Internet radio folks probably are already aware, but another place where HTML5 audio support falls flat is for streaming of audio.  As this article discusses, radio websites are having to continue relying upon Adobe Flash in order to provide necessary functionality.

HTML5 Audio rant from March 2011

If you have delicate sensibilities, this State of HTML5 Audio article may not be for you, but no one else on the web has put so clearly how screwed up audio implementation is in HTML5.

Among the topics are browser format support, playback issues (such as noise introduced during playback), complete lack of audio codecs in some browsers (cough, Android, cough), and more.

Interestingly, the article complains quite loudly about Google Chrome, saying:

Google's Chrome has the worst HTML5 Audio support of all the good Desktop Browsers - that is every Browser but IE. I'm not an audio engineer, but before Browser vendors took their shot at it, my impression of digital audio was that it is a solved problem. I'm amazed that after so many iterations HTML5 Audio is still that broken.

As of this writing (May 2012) Google Chrome now has what might be the best audio functionality of any browser, supporting the new Web Audio API.

The State of HTML5 Audio...as of November 2010

Scott Schiller provides an outstanding look at how limited HTML5 Audio is...18 months ago.  He discusses the format wars, browser's political games in terms of audio format support, "upcoming" APIs, and current (as of November 2010) state of audio abilities on various browsers and mobile devices.

Time-based audio transport automation

Want to play sound and have the playback stop at a particular time?  This sample code provides a look at how to use setInterval() to check current playback time and trigger audio.pause() when the time reaches a certain point.

Beginner's guide to using <audio> and <video>.

Simon Pieters, a developer at Opera, published an excellent primer on using basic audio and video functionality in HTML5.  If you've done some web development in JavaScript in the past, you should find these examples a good starting point.

Welcome!

Welcome to the HTML5 Audio blog.

You may have noticed that HTML5 has become a fantastic place to develop applications, games, and other cross-platform solutions. One can do amazing things with images, text, layout, and more. But audio tools are sadly lacking and in need of some attention.

This site intends to help bring attention to the current state of HTML5 Audio and encourage radical improvements.