Is it possible to stream audio and video independently within the same video file?

<video id="video1" width="320" height="176" controls="controls">
  <source src="mov_bbb.mp4" type="video/mp4">
  
  <source src="mov_bbb.m4a" type="video/m4a">
  
</video>

I am looking to enhance this video by incorporating an audio file into its playback.

Answer №1

The MediaSource API enables us to achieve exactly this.

It's important to note that in order for this to function properly, your media assets must be prepared in a way that is compatible with the API. Take some time to thoroughly understand the process by reading through this informative MDN article.

Once you have your assets ready, the rest of the steps are quite straightforward:

(async() => {

  const fetching = Promise.all( [
    // getting the video "only" file
    fetchData( "https://dl.dropboxusercontent.com/s/u9ycdfwy8fig4dl/bbb_video.mp4" ),
    // acquiring the audio "only" file
    fetchData( "https://dl.dropboxusercontent.com/s/rj4dh32vxwi1iv5/bbb_audio.mp4" )
  ] );

  const video_mime = "video/mp4; codecs=avc1.64000c";
  const audio_mime = "audio/mp4; codecs=mp4a.40.2";
  if(
    !MediaSource.isTypeSupported( video_mime ) ||
    !MediaSource.isTypeSupported( audio_mime )
  ) {
    throw "unsupported codecs";
  }
  
  const source = new MediaSource();
  document.querySelector( "video" ).src = URL.createObjectURL( source );
  await waitForEvent( source, "sourceopen" );

  const video_buffer = source.addSourceBuffer( video_mime );
  const audio_buffer = source.addSourceBuffer( audio_mime );
  video_buffer.mode = audio_buffer.mode = "sequence";

  const [ video_data, audio_data ] = await fetching;
  
  const chunk_size = 10 * 1024 * 1024;
  let i = 0;
  while (
    i < video_data.length &&
    i < audio_data.length
  ) {
    const next_i = i + chunk_size;
    const events = Promise.all( [
      waitForEvent( video_buffer, "updateend" ),
      waitForEvent( audio_buffer, "updateend" )
    ] );
    video_buffer.appendBuffer( video_data.subarray( i, next_i ) );
    audio_buffer.appendBuffer( audio_data.subarray( i, next_i ) );
    await events;
    i = next_i;
  }
  
  source.endOfStream();

})().catch( console.error );

function fetchData( url ) {
  return fetch( url )
    .then( (resp) => resp.ok && resp.arrayBuffer() )
    .then( (buf) => new Uint8Array( buf ) );
}
function waitForEvent( target, event_name ) {
  return new Promise( (res) => {
    target.addEventListener( event_name, res, { once: true } );
  } );
}
<video controls></video>

Answer №2

My solution to this problem is:

<meta charset="utf-8">
<video controls id="v" height="480" src="file.webm"></video>
<audio id="a" src="file.weba"></audio>

<script>
let v = document.getElementById('v');
let a = document.getElementById('a');
v.onpause = () => a.pause();
v.onplay = () => a.play();
v.onseeked = () => a.currentTime = v.currentTime;
</script>

In this setup, the video element has controls and the audio element responds to play, pause, and seek actions of the video. The volume control feature is not included here, so you have the option to implement it separately or utilize the system volume control instead.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Deciphering an ApplePay Token using JavaScript

Looking to decrypt an ApplePay token using NPM packages in JavaScript. Although there are repositories available for decrypting in Ruby like https://github.com/spreedly/gala, I am interested in porting it to JavaScript. However, I am uncertain about the Ja ...

Guide to utilizing variables with Pattern.compile in Java for selenium RC

I am facing a challenge where I need to check if the date in a field corresponds to the current day. In Selenium IDE, this is easily achieved: <tr> <td>storeExpression</td> <td>javascript{var date = new Date ...

Prevent typing on input fields for numbers exceeding 3 digits

How can I prevent users from entering a number with more than 3 digits? For example, allowing entries like 150 but not accepting numbers like 1601. The keypress should be disabled in such cases. The keypress event must be disabled. <template> < ...

Oops! Looks like there's an issue with reading the property 'value' of undefined in React-typeahead

Having some issues with setting the state for user selection in a dropdown menu using the react-bootstrap-typeahead component. Any guidance or resources on this topic would be highly appreciated. Details can be found here. The function handleAddTask is su ...

"Mongo server is rejecting the connection, and the reason is unclear to me

I created a MongoDB model with the following structure:- var mongoose = require('mongoose'); const itemsModel = new mongoose.Schema({ _id: { type: String, }, userName: { type: String, required: true }, ...

What are some possible causes for an iframe failing to load?

When it comes to iframes failing to load their content, there could be various reasons causing this issue. I recently encountered a situation where an iframe on my site's "thank you" page was not displaying any content when inspected using Chrome dev ...

Turn off the background color box with a single click

Whenever I click on a header menu item, the background changes along with it. Is there a way to prevent this from happening? https://i.stack.imgur.com/p6bJ9.png Take a look here ...

Having trouble with jQuery displaying the first child element properly

Hi there! I am currently utilizing jQuery to hide certain elements on my webpage, and it seems to be functioning well. However, I am encountering an issue where I want the first child to be shown by default upon loading the page, but this feature does not ...

The error states that the type '() => string | JSX.Element' cannot be assigned to the type 'FC<{}>'

Can someone help me with this error I'm encountering? I am fairly new to typescript, so I assume it has something to do with that. Below is the code snippet in question: Any guidance would be greatly appreciated. const Pizzas: React.FC = () => { ...

Bringing in data from <script> to use in <script setup>

To ensure unique ids for instances of a Vue component, I am using a simple generator to enumerate them. Typically, I would initialize the generator outside of the setup function like this: const idGen = generateIds("component:my-component"); export defaul ...

Position the div at the bottom of the menu container, ensuring it is not displayed on mobile devices

When I click the button on my mobile sidenav menu, it appears and disappears. I'm trying to figure out how to position <div class="mtsMenu_footer"> at the bottom of the menu. I attempted to use position: absolute; and bottom: 0; in the ...

Leveraging an array to store data within highcharts

When I input the data directly in high charts, it works perfectly: $(function () { $('#container').highcharts({ title: { text: 'Monthly Average Temperature', x: -20 //center }, subtit ...

AngularJS - Calculate multiple currencies

I need to calculate the product of a value and months. For the input value, I am utilizing a Jquery Plugin to apply a currency mask. Unfortunately, the calculations are not functioning properly with this plugin. My goal is to multiply the value, includin ...

Markdown in Vue.js filters

I found a helpful example by Evan You on how to convert HTML to markdown - check it out here. However, when trying to install the marked package via npm and implementing it, I encountered an error stating that 'marked' is not defined. After inc ...

considering multiple website addresses as one collective entity for the Facebook like button

Currently, I am tackling an issue on a website that employs a custom CMS which automatically generates URLs based on the titles of articles. The main problem faced by our contributors is that when they update the title of an article, it creates a new URL ...

Insert, delete, and modify rows within the table

I'm struggling with a JavaScript issue and could use some help. How can I add a new row for all columns with the same properties as the old rows, including a "remove" button for the new row? Is there a way to prevent editing cells that contain b ...

Guide on linking Influxdb information in a Vue application using node.js?

I have successfully connected my InfluxDB database to my Vue app and can log data in the terminal using the code below: // index.js import express from "express"; // These lines make "require" available import { createRequire ...

Error in Chart.jsx: Unable to retrieve the length property of an undefined object in the COVID-19 Tracker App

INQUIRY Greetings, I am in need of assistance to identify an error that is perplexing me. The source of this code can be traced back to a tutorial on creating a covid tracker available on YouTube. While attempting to implement the chart feature, I encounte ...

Steps for obtaining images using lazy loading: <img loading="lazy"

After successfully utilizing the JavaScript-executer to locate the ImageElement, I encountered an error when attempting to extract the URL for downloading the image.svg. The error message reads "NoSuchElementException." Below is a snippet of my code: ((J ...

State of Active Scroll group

How can I dynamically add the "active" class to a link based on which section it has scrolled to? I want the active state to be applied without a hover effect, only when the corresponding section is currently in view. #pagemenu{ display: block; positio ...