• Skip to main content
  • Select language
  • Skip to search
MDN Web Docs
  • Technologies
    • HTML
    • CSS
    • JavaScript
    • Graphics
    • HTTP
    • APIs / DOM
    • WebExtensions
    • MathML
  • References & Guides
    • Learn web development
    • Tutorials
    • References
    • Developer Guides
    • Accessibility
    • Game development
    • ...more docs
Archive of obsolete content
  1. MDN
  2. Archive of obsolete content
  3. Archived Mozilla and build documentation
  4. Introducing the Audio API extension

Introducing the Audio API extension

In This Article
  1. Reading audio streams
    1. The loadedmetadata event
    2. The MozAudioAvailable event
  2. Creating an audio stream
  3. Processing an audio stream
  4. See also

The Audio Data API extension extends the HTML5 specification of the <audio> and <video> media elements by exposing audio metadata and raw audio data. This enables users to visualize audio data, to process this audio data and to create new audio data.

Obsolete since Gecko 28 (Firefox 28 / Thunderbird 28 / SeaMonkey 2.25)
This feature is obsolete. Although it may still work in some browsers, its use is discouraged since it could be removed at any time. Try to avoid using it.

Deprecated since Gecko 22 (Firefox 22 / Thunderbird 22 / SeaMonkey 2.19)
This feature has been removed from the Web. Though some browsers may still support it, it is in the process of being dropped. Do not use it in old or new projects. Pages or Web apps using it may break at any time.

Please note that this document describes a non-standard experimental API. This API has been deprecated since Gecko 22, disabled since Gecko 28, and removed from Gecko 31. You should use the Web Audio API instead.

Reading audio streams

The loadedmetadata event

When the metadata of the media element is available, it triggers a loadedmetadata event. This event has the following attributes:

  • mozChannels: Number of channels
  • mozSampleRate: Sample rate per second
  • mozFrameBufferLength: Number of samples collected in all channels

This information is needed later to decode the audio data stream. The following example extracts the data from an audio element:

<!DOCTYPE html>
<html>
  <head>
    <title>JavaScript Metadata Example</title>
  </head>
  <body>
    <audio id="audio-element"
           src="song.ogg"
           controls="true"
           style="width: 512px;">
    </audio>
    <script> 
      function loadedMetadata() {
        channels          = audio.mozChannels;
        rate              = audio.mozSampleRate;
        frameBufferLength = audio.mozFrameBufferLength;      
      } 
      var audio = document.getElementById('audio-element'); 
      audio.addEventListener('loadedmetadata', loadedMetadata, false);
    </script>
  </body>
</html>

The MozAudioAvailable event

As the audio is played, sample data is made available to the audio layer and the audio buffer (size defined in mozFrameBufferLength) gets filled with those samples. Once the buffer is full, the event MozAudioAvailable is triggered. This event therefore contains the raw samples of a period of time. Those samples may or may not have been played yet at the time of the event and have not been adjusted for mute or volume settings on the media element. Playing, pausing, and seeking the audio also affect the streaming of this raw audio data.

The MozAudioAvailable event has 2 attributes:

  • frameBuffer: Framebuffer (i.e., an array) containing decoded audio sample data (i.e., floats)
  • time: Timestamp for these samples measured from the start in seconds

The framebuffer contains an array of audio samples. It's important to note that the samples are not separated by channels; they are all delivered together. For example, for a two-channel signal: Channel1-Sample1 Channel2-Sample1  Channel1-Sample2 Channel2-Sample2 Channel1-Sample3 Channel2-Sample3.

We can extend the previous example to visualize the timestamp and the first two samples in a <div> element:

<!DOCTYPE html>
<html>
  <head>
    <title>JavaScript Visualization Example</title>
  </head>
  <body>
    <audio id="audio-element"
           src="revolve.ogg"
           controls="true"
           style="width: 512px;">
    </audio>
	<pre id="raw">hello</pre>
    <script> 
      function loadedMetadata() {
        channels          = audio.mozChannels;
        rate              = audio.mozSampleRate;
        frameBufferLength = audio.mozFrameBufferLength; 		
      } 
      function audioAvailable(event) {
        var frameBuffer = event.frameBuffer;
        var t = event.time;
        var text = "Samples at: " + t + "\n";
        text += frameBuffer[0] + "  " + frameBuffer[1];
        raw.innerHTML = text;
      }
      var raw = document.getElementById('raw');
      var audio = document.getElementById('audio-element'); 
      audio.addEventListener('MozAudioAvailable', audioAvailable, false); 
      audio.addEventListener('loadedmetadata', loadedMetadata, false);
    </script>
  </body>
</html>

Creating an audio stream

It is also possible to create and setup an <audio> element for raw writing from script (i.e., without a src attribute). Content scripts can specify the audio stream's characteristics, then write audio samples. Users must create an audio object and then use the mozSetup() function to specify the number of channels and the frequency (in Hz).  For example:

// Create a new audio element
var audioOutput = new Audio();
// Set up audio element with 2 channel, 44.1KHz audio stream.
audioOutput.mozSetup(2, 44100);

Once this is done, the samples need to be created. Those samples have the same format as the ones in the mozAudioAvailable event. Then the samples are written in the audio stream with the function mozWriteAudio(). It's important to note that not all the samples might get written in the stream. The function returns the number of samples written, which is useful for the next writing. You can see an example below:

// Write samples using a JS Array
var samples = [0.242, 0.127, 0.0, -0.058, -0.242, ...];
var numberSamplesWritten = audioOutput.mozWriteAudio(samples);
// Write samples using a Typed Array
var samples = new Float32Array([0.242, 0.127, 0.0, -0.058, -0.242, ...]);
var numberSamplesWritten = audioOutput.mozWriteAudio(samples);

In the following example, we create an audio pulse:

<!doctype html>
<html>
  <head>
     <title>Generating audio in real time</title>   <script type="text/javascript">
     function playTone() {
       var output = new Audio();
      output.mozSetup(1, 44100);
       var samples = new Float32Array(22050);
       for (var i = 0; i < samples.length ; i++) {
         samples[i] = Math.sin( i / 20 );
       }
              output.mozWriteAudio(samples);
     }
   </script>
 </head>
 <body>
   <p>This demo plays a one second tone when you click the button below.</p>
   <button onclick="playTone();">Play</button>
 </body>
 </html>

The mozCurrentSampleOffset() method gives the audible position of the audio stream, meaning the position of the last heard sample.

// Get current audible position of the underlying audio stream, measured in samples.
var currentSampleOffset = audioOutput.mozCurrentSampleOffset();

Audio data written using the mozWriteAudio() method needs to be written at a regular interval in equal portions, in order to keep a little ahead of the current sample offset (the sample offset that is currently being played by the hardware can be obtained with mozCurrentSampleOffset()), where "a little" means something on the order of 500 ms of samples. For example, if working with two channels at 44100 samples per second, a writing interval of 100 ms, and a pre-buffer equal to 500 ms, one would write an array of (2 * 44100 / 10) = 8820 samples, and a total of (currentSampleOffset + 2 * 44100 / 2).

It's also possible to auto-detect the minimal duration of the pre-buffer, such that the sound is played without interruptions, and lag between writing and playback is minimal. To do this start writing the data in small portions and wait for the value returned by mozCurrentSampleOffset() to be greater than 0.

var prebufferSize = sampleRate * 0.020; // Initial buffer is 20 ms
var autoLatency = true, started = new Date().valueOf();
...
// Auto latency detection
if (autoLatency) {
  prebufferSize = Math.floor(sampleRate * (new Date().valueOf() - started) / 1000);
  if (audio.mozCurrentSampleOffset()) { // Play position moved?
    autoLatency = false; 
}

Processing an audio stream

Since the MozAudioAvailable event and the mozWriteAudio() method both use Float32Array values, it is possible to take the output of one audio stream and pass it directly (or process first and then pass) to a second. The first audio stream needs to be muted so that only the second audio element is heard.

<audio id="a1" 
       src="song.ogg"
       controls>
</audio>
<script>
var a1 = document.getElementById('a1'),
    a2 = new Audio(),
    buffers = [];
function loadedMetadata() {
  // Mute a1 audio.
  a1.volume = 0;
  // Setup a2 to be identical to a1, and play through there.
  a2.mozSetup(a1.mozChannels, a1.mozSampleRate);
}
function audioAvailable(event) {
  // Write the current framebuffer
  var frameBuffer = event.frameBuffer;
  writeAudio(frameBuffer);
}
a1.addEventListener('MozAudioAvailable', audioAvailable, false);
a1.addEventListener('loadedmetadata', loadedMetadata, false);
function writeAudio(audio) {
  buffers.push(audio);
  // If there's buffered data, write that
  while(buffers.length > 0) {
    var buffer = buffers.shift();
    var written = a2.mozWriteAudio(buffer);
    // // If all data wasn't written, keep it in the buffers:
    if(written < buffer.length) {
      buffers.unshift(buffer.slice(written));
      return;
    }
  }
}
</script>

See also

  • Creating a Web based tone generator
  • Visualizing an audio spectrum
  • Displaying a graphic with audio samples
  • Creating a simple synth (using existing libraries)
  • Wikipedia article on digital audio

Document Tags and Contributors

Tags: 
  • Deprecated
 Contributors to this page: teoli, kohei.yoshino, rillian, Ehsan, NickDesaulniers, MMTLukas, malachai, Kinetik, furrykef, jswisher, Bzbarsky, Kennyluck, quinnirill, sebmozilla
 Last updated by: teoli, May 26, 2014, 11:32:42 PM

  1. .htaccess ( hypertext access )
  2. <input> archive
  3. Add-ons
    1. Add-ons
    2. Firefox addons developer guide
    3. Interaction between privileged and non-privileged pages
    4. Tabbed browser
    5. bookmarks.export()
    6. bookmarks.import()
  4. Adding preferences to an extension
  5. An Interview With Douglas Bowman of Wired News
  6. Apps
    1. Apps
    2. App Development API Reference
    3. Designing Open Web Apps
    4. Graphics and UX
    5. Open web app architecture
    6. Tools and frameworks
    7. Validating web apps with the App Validator
  7. Archived Mozilla and build documentation
    1. Archived Mozilla and build documentation
    2. ActiveX Control for Hosting Netscape Plug-ins in IE
    3. Archived SpiderMonkey docs
    4. Autodial for Windows NT
    5. Automated testing tips and tricks
    6. Automatic Mozilla Configurator
    7. Automatically Handle Failed Asserts in Debug Builds
    8. BlackConnect
    9. Blackwood
    10. Bonsai
    11. Bookmark Keywords
    12. Building TransforMiiX standalone
    13. Chromeless
    14. Creating a Firefox sidebar extension
    15. Creating a Microsummary
    16. Creating a Mozilla Extension
    17. Creating a Release Tag
    18. Creating a Skin for Firefox/Getting Started
    19. Creating a Skin for Mozilla
    20. Creating a Skin for SeaMonkey 2.x
    21. Creating a hybrid CD
    22. Creating regular expressions for a microsummary generator
    23. DTrace
    24. Dehydra
    25. Developing New Mozilla Features
    26. Devmo 1.0 Launch Roadmap
    27. Download Manager improvements in Firefox 3
    28. Download Manager preferences
    29. Drag and Drop
    30. Embedding FAQ
    31. Embedding Mozilla in a Java Application using JavaXPCOM
    32. Error Console
    33. Exception logging in JavaScript
    34. Existing Content
    35. Extension Frequently Asked Questions
    36. Fighting Junk Mail with Netscape 7.1
    37. Firefox Sync
    38. Force RTL
    39. GRE
    40. Gecko Coding Help Wanted
    41. HTTP Class Overview
    42. Hacking wiki
    43. Help Viewer
    44. Helper Apps (and a bit of Save As)
    45. Hidden prefs
    46. How to Write and Land Nanojit Patches
    47. Introducing the Audio API extension
    48. Java in Firefox Extensions
    49. JavaScript crypto
    50. Jetpack
    51. Litmus tests
    52. Makefile.mozextension.2
    53. Microsummary topics
    54. Migrate apps from Internet Explorer to Mozilla
    55. Monitoring downloads
    56. Mozilla Application Framework
    57. Mozilla Crypto FAQ
    58. Mozilla Modules and Module Ownership
    59. Mozprocess
    60. Mozprofile
    61. Mozrunner
    62. Nanojit
    63. New Skin Notes
    64. Persona
    65. Plug-n-Hack
    66. Plugin Architecture
    67. Porting NSPR to Unix Platforms
    68. Priority Content
    69. Prism
    70. Proxy UI
    71. Remote XUL
    72. SXSW 2007 presentations
    73. Space Manager Detailed Design
    74. Space Manager High Level Design
    75. Standalone XPCOM
    76. Stress testing
    77. Structure of an installable bundle
    78. Supporting private browsing mode
    79. Table Cellmap
    80. Table Cellmap - Border Collapse
    81. Table Layout Regression Tests
    82. Table Layout Strategy
    83. Tamarin
    84. The Download Manager schema
    85. The life of an HTML HTTP request
    86. The new nsString class implementation (1999)
    87. TraceVis
    88. Treehydra
    89. URIScheme
    90. URIs and URLs
    91. Using Monotone With Mozilla CVS
    92. Using SVK With Mozilla CVS
    93. Using addresses of stack variables with NSPR threads on win16
    94. Venkman
    95. Video presentations
    96. Why Embed Gecko
    97. XML in Mozilla
    98. XPInstall
    99. XPJS Components Proposal
    100. XRE
    101. XTech 2005 Presentations
    102. XTech 2006 Presentations
    103. XUL Explorer
    104. XULRunner
    105. ant script to assemble an extension
    106. calICalendarView
    107. calICalendarViewController
    108. calIFileType
    109. xbDesignMode.js
  8. Archived open Web documentation
    1. Archived open Web documentation
    2. Browser Detection and Cross Browser Support
    3. Browser Feature Detection
    4. Displaying notifications (deprecated)
    5. E4X
    6. E4X Tutorial
    7. LiveConnect
    8. MSX Emulator (jsMSX)
    9. Old Proxy API
    10. Properly Using CSS and JavaScript in XHTML Documents
    11. Reference
    12. Scope Cheatsheet
    13. Server-Side JavaScript
    14. Sharp variables in JavaScript
    15. Standards-Compliant Authoring Tools
    16. Using JavaScript Generators in Firefox
    17. Window.importDialog()
    18. Writing JavaScript for XHTML
    19. XForms
    20. background-size
    21. forEach
  9. B2G OS
    1. B2G OS
    2. Automated Testing of B2G OS
    3. B2G OS APIs
    4. B2G OS add-ons
    5. B2G OS architecture
    6. B2G OS build prerequisites
    7. B2G OS phone guide
    8. Building B2G OS
    9. Building and installing B2G OS
    10. Building the B2G OS Simulator
    11. Choosing how to run Gaia or B2G
    12. Customization with the .userconfig file
    13. Debugging on Firefox OS
    14. Developer Mode
    15. Developing Firefox OS
    16. Firefox OS Simulator
    17. Firefox OS apps
    18. Firefox OS board guide
    19. Firefox OS developer release notes
    20. Firefox OS security
    21. Firefox OS usage tips
    22. Gaia
    23. Installing B2G OS on a mobile device
    24. Introduction to Firefox OS
    25. Mulet
    26. Open web apps quickstart
    27. Pandaboard
    28. PasscodeHelper Internals
    29. Porting B2G OS
    30. Preparing for your first B2G build
    31. Resources
    32. Running tests on Firefox OS: A guide for developers
    33. The B2G OS platform
    34. Troubleshooting B2G OS
    35. Using the App Manager
    36. Using the B2G emulators
    37. Web Bluetooth API (Firefox OS)
    38. Web Telephony API
    39. Web applications
  10. Beginner tutorials
    1. Beginner tutorials
    2. Creating reusable content with CSS and XBL
    3. Underscores in class and ID Names
    4. XML data
    5. XUL user interfaces
  11. Case Sensitivity in class and id Names
  12. Creating a dynamic status bar extension
  13. Creating a status bar extension
  14. Gecko Compatibility Handbook
  15. Getting the page URL in NPAPI plugin
  16. Index
  17. Inner-browsing extending the browser navigation paradigm
  18. Install.js
  19. JXON
  20. List of Former Mozilla-Based Applications
  21. List of Mozilla-Based Applications
  22. Localizing an extension
  23. MDN
    1. MDN
    2. Content kits
  24. MDN "meta-documentation" archive
    1. MDN "meta-documentation" archive
    2. Article page layout guide
    3. Blog posts to integrate into documentation
    4. Current events
    5. Custom CSS classes for MDN
    6. Design Document
    7. DevEdge
    8. Developer documentation process
    9. Disambiguation
    10. Documentation Wishlist
    11. Documentation planning and tracking
    12. Editing MDN pages
    13. Examples
    14. Existing Content/DOM in Mozilla
    15. External Redirects
    16. Finding the right place to document bugs
    17. Getting started as a new MDN contributor
    18. Landing page layout guide
    19. MDN content on WebPlatform.org
    20. MDN page layout guide
    21. MDN subproject list
    22. Needs Redirect
    23. Page types
    24. RecRoom documentation plan
    25. Remove in-content iframes
    26. Team status board
    27. Trello
    28. Using the Mozilla Developer Center
    29. Welcome to the Mozilla Developer Network
    30. Writing chrome code documentation plan
    31. Writing content
  25. MMgc
  26. Makefile - .mk files
  27. Marketplace
    1. Marketplace
    2. API
    3. Monetization
    4. Options
    5. Publishing
  28. Mozilla release FAQ
  29. Newsgroup summaries
    1. Newsgroup summaries
    2. Format
    3. Mozilla.dev.apps.firefox-2006-09-29
    4. Mozilla.dev.apps.firefox-2006-10-06
    5. mozilla-dev-accessibility
    6. mozilla-dev-apps-calendar
    7. mozilla-dev-apps-firefox
    8. mozilla-dev-apps-thunderbird
    9. mozilla-dev-builds
    10. mozilla-dev-embedding
    11. mozilla-dev-extensions
    12. mozilla-dev-i18n
    13. mozilla-dev-l10n
    14. mozilla-dev-planning
    15. mozilla-dev-platform
    16. mozilla-dev-quality
    17. mozilla-dev-security
    18. mozilla-dev-tech-js-engine
    19. mozilla-dev-tech-layout
    20. mozilla-dev-tech-xpcom
    21. mozilla-dev-tech-xul
    22. mozilla.dev.apps.calendar
    23. mozilla.dev.tech.js-engine
  30. Obsolete: XPCOM-based scripting for NPAPI plugins
  31. Plugins
    1. Plugins
    2. Adobe Flash
    3. External resources for plugin creation
    4. Logging Multi-Process Plugins
    5. Monitoring plugins
    6. Multi-process plugin architecture
    7. NPAPI plugin developer guide
    8. NPAPI plugin reference
    9. Samples and Test Cases
    10. Shipping a plugin as a Toolkit bundle
    11. Supporting private browsing in plugins
    12. The First Install Problem
    13. Writing a plugin for Mac OS X
    14. XEmbed Extension for Mozilla Plugins
  32. SAX
  33. Security
    1. Security
    2. Digital Signatures
    3. Encryption and Decryption
    4. Introduction to Public-Key Cryptography
    5. Introduction to SSL
    6. NSPR Release Engineering Guide
    7. SSL and TLS
  34. Solaris 10 Build Prerequisites
  35. Sunbird Theme Tutorial
  36. Table Reflow Internals
  37. Tamarin Tracing Build Documentation
  38. The Basics of Web Services
  39. Themes
    1. Themes
    2. Building a Theme
    3. Common Firefox theme issues and solutions
    4. Creating a Skin for Firefox
    5. Making sure your theme works with RTL locales
    6. Theme changes in Firefox 2
    7. Theme changes in Firefox 3
    8. Theme changes in Firefox 3.5
    9. Theme changes in Firefox 4
  40. Updating an extension to support multiple Mozilla applications
  41. Using IO Timeout And Interrupt On NT
  42. Using SSH to connect to CVS
  43. Using workers in extensions
  44. WebVR
    1. WebVR
    2. WebVR environment setup
  45. XQuery
  46. XUL Booster
  47. XUL Parser in Python