Adaptive streaming video with dash.js in react

I was recently instructed to create videos that had to be played smoothly in a slow network or on low -productive devices. I started with a native html5.<Video>The tag, but quickly encounters the obstacle - it simply does not work when the connection is slow or the devices have insufficient power.

After some studies I found thatAdaptive flow transmission- This is the solution I needed. But here is what is upset: it was very difficult to find a comprehensive guide for beginners. Resources on MDN and other websites were useful, but they lacked the comprehensive guide I was looking for.

That's why I write this article: to give you a step -by -step guide, which I am sorry I have not found. I will combine FFMPEG scripts, video file encoding and video player implementation compatible with Dash (Dash.js), using code examples you can use.

Exit outside the native html5<Video>Tag

You can ask why you can't just rely on html<Video>element. There is a good reason for this. Let's compare the difference between native<Video>Item and adaptive streaming video in browsers.

Progressive load

During progressive download, your browser linearly loads the video file from the server via HTTP and starts playback if buffering. It's a default behavior.<Video>element.

<Video SRC = "Rabbit320.mp4" />

While playing video, check the network tab in your browser and you will see a few requests with206 Partial ContentThe state code.

If the server or browser does not support the range queries, the entire video file will be loaded with one request by returning200 OKThe state code. In this case, video playback can only start after the entire file is completed.

Problems?If you have a slow connection when trying to watch high resolution videos, you will have to wait a long time before playing.

Adaptive flow transmission

Instead of playing one video file,Streaming with adaptive bitraite (ABR)divides a video into several segments with different bitrack and resolution. When playing, the ABR algorithm automatically selects the highest quality segment that can be downloaded in a timely manner for smooth playback, depending on your network connection, bandwidth and other device capabilities. It is constantly tuned to adapt to changing conditions.

This magic occurs with two key browser technologies:

• Expanding media source (MSE)
  This allows you to transfer the MediaSource object toSRCAttribute y<Video>that allows you to send several Sourcebffer objects that represent video segments.

<Video SRC = "BLOB: https: //Example.com/6e31Fe2a-a0a8-43f9-b415-73dc02985892"/>

• API media can be possible
  It provides information about the decoding and encoding of the video with your device, which allows ABR to make informed decisions as to what resolution.

Together, they provide the basic functionality of the ABR, maintaining videos optimized for restrictions on your device, in real time.

Stream Transmission Protocols: MPEG-DASH v. HLS

As mentioned above, for adaptive streaming of media files, the video is divided into fragments with different levels of quality at different times. We need to simplify the process of adaptive switching between these segments in real time. To achieve this, the ABR transmission is based on certain protocols. Two most common ABR protocols:

• Mpeg-dash,
• Straight HTTP (HLS) broadcast.

Both of these protocols use HTTP to send video files. So they are compatible with the HTTP web servers.

This article is focused on MPEG-DASH. However, it is worth noting that Dash is not supported by Apple's devices or browsers, as mentioned in the MUX article.

Mpeg-dash

MPEG-DASH provides adaptive flow transmission through:

• Media Presentation Description File (MPD)
  This XML File of the Manifesto contains information on how to choose and control streams based on adaptive rules.
• Segmented media
  Video and audio files are divided into segments with different resolutions and duration with codecs and formats compatible with MPEG-DASH.

On the client's side, DASH video player reads the MPD file and constantly monitors the network bandwidth. Based on the available bandwidth, the player chooses the appropriate bitraite and asks the appropriate video. This process is repeated throughout the reproduction, providing smooth and optimal quality.

Now that you understand the basics, let's create our adaptive video player!

Steps to create an adaptive bitraite player streaming video

Here's a plan:

1. Transfer the MP4 video to audio and video versions with different resolution and bitraite using FFMPEG.
2. Generate the MPD file using FFMPEG.
3. Service output files from the server.
4. Create a video player compatible with Dash to play video.

FFMPEG installation

For MacOS users, install FFMPEG using Brew by executing the following command in your terminal:

Brew Install FFMPEG

Generation of audio formation

Next, start the following script to pull out the audio track and encode it in Webm format for compatibility with Dash:

ffmpeg -i "input_video.mp4" -vn -acodec libVorbis -ab 128k "Audio.webm"

• -i "Input_video.mp4": Identifies the input video file.
• -vn: Turn off the video flow (only audio exit).
• -acodec libvorbisUsesLibVorbis codecfor audio encoding.
• -ab 128k: Sets a bitraite audio on128 kbps .
• “Audio.webm”: Identifies the Webm Output Audio file.

Generation of video visualization

Run this script to create three video versions with different resolution and bitrack. The highest resolution should match the size of the input file. For example, if an incoming video has a resolution576 × 1024With a frequency of 30 frames per second (FPS), the script generates versions optimized for vertical video playback.

ffmpeg -i "input_video.mp4" -c: v libvpx -vp9 -keyint_min 150 -g 150
-tile -Columns 4 -frame -Parallel 1 -F Webm \
-an -vf Scale = 576: 1024 -b: V 1500k "Input_video_576x1024_1500k.webm"
-an -vf Scale = 480: 854 -b: V 1000k "Input_video_480x854_1000k.webm"
-an -vf Scale = 360: 640 -b: V 750k "Input_video_360x640_750k.webm"

• -C: V Libvpx-VP9Useslibvpx-VP9as VP9 video coder for Webm.
• -Keyint_min 150and-g 150: SetThe key frame interval of 150 frames(approximately every 5 seconds at 30 frames/s). This allows you to switch the bitraite every 5 seconds.
• -tile-Columns 4and-frame-parallel 1Optimize coding performance with parallel processing.
• -F Webm: Defines the Webm Output format.

In each performance:

• -an: Excludes audio (output only video).
• -vf Scale = 576: 1024: Scales video to resolution576x1024Pixels.
• -B: V 1500k: Set up the video video on1500 kbps .

Webm is selected as the original format because they are smaller in size and optimized, but widely compatible with most webbrowers.

To generate the MPD Manifesto File

Combine videos and audio traveling into a MPD Manifesto File, DASH compatible with the following script:

ffmpeg;
  -F Webm_dash_manifest -i "Input_video_576x1024_1500k.webm"
  -F Webm_dash_manifest -i "Input_video_480x854_1000k.webm"
  -F Webm_dash_manifest -i "Input_video_360x640_750k.webm"
  -f webm_dash_manifest -i "Audio.webm"
  -c Copy
  -Map 0 -Map 1 -Map 2 -Map 3
  -f webm_dash_manifest
  -adaptation_sets "id = 0, streams = 0.1.2 id = 1, streams = 3"
  “INPUT_VIDEO_MANIFEST.MPD”

• -F Webm_dash_manifest -I "...": Identifies the inputs between which the ASH video player will be dynamically switch depending on the network condition.
• -Map 0 -Map 1 -Map 2 -Map 3Includes all videos (0, 1, 2) and audio (3) in the final manifesto.
• -adaptation_SetsGroups flows into adaptation sets:
  • ID = 0, Streams = 0.1.2: Group video versions into one set of adaptations.
  • ID = 1, Streams = 3: Appoints an audio track with a separate set of adaptation.

MPD file received (input_video_manifest.mpd) describes the flows and provides adaptive bitraite transmission in MPEG-DASH.

<? XML Version = "1.0" Encoding = "UTF-8"?>
<Mpd
  xmlns: xsi = "http://www.w3.org/2001/xmlschema-instance"
  XMLNS = "URN: MPEG: DASH: SCHEMA: MPD: 2011"
  XSI: Schemalocation = "URN: MPEG: DASH: Schema: MPD: 2011"
  Type = "STATIC"
  Mediapresentationdration = "PT81.166s"
  Minbuffertime = "PT1S"
  Profiles = "URN: MPEG: Dash: Profile: Webm-on-Demand: 2012">

  <Period Id = "0" Start = "pt0s" duration = "pt81.166s">
    <AdptationSet
      id = "0"
      Mimetype = "Video/Webm"
      Codecs = "VP9"
      lang = "eng"
      Bitstreamswitching = "TRUE"
      subsegmentalignment = "false"
      subsegmentstartswithsap = "1">
      
      <Representation ID = "0" Bandwidth = "1647920" Width = "576" Height = "1024">>
        <Baseurl> Input_video_576x1024_1500k.webm </baseurl>
        <SegmentBase Indexrange = "16931581-16931910">
          <Initialization Range = "0-645" />
        </ SegmentBase>
      </ARPRESENTATION>
      
      <Representation ID = "1" Bandwidth = "1126977" Width = "480" Height = "854">>
        <Baseurl> Input_video_480x854_1000k.webm </baseurl>
        <SegmentBase Indexrange = "11583599-11583986">
          <Initialization Range = "0-645" />
        </ SegmentBase>
      </ARPRESENTATION>
      
      <Representation ID = "2" Bandwidth = "843267" Width = "360" Height = "640">>
        <Baseurl> Input_video_360x640_750k.webm </baseurl>
        <SegmentBase Indexrange = "8668326-8668713">
          <Initialization Range = "0-645" />
        </ SegmentBase>
      </ARPRESENTATION>
      
    </ AdaptationSet>
    
    <AdptationSet
      id = "1"
      Mimtype = "Audio/Webm"
      Codecs = "VORBIS"
      lang = "eng"
      AudiosamplingLINGRATE = "44100"
      Bitstreamswitching = "TRUE"
      subsegmentalignment = "true"
      subsegmentstartswithsap = "1">
      
      <Representation ID = "3" Bandwidth = "89219">
        <baseurl> audio.webm </baseurl>
        <SegmentBase Indexrange = "921727-922055">
          <Initialization Range = "0-4889" />
        </ SegmentBase>
      </ARPRESENTATION>
      
    </ AdaptationSet>
  </period>
</mpd>

After performing these steps you will have:

1. Three video versions (576x1024, 480x854, 360x640)
2. One audio traveling, and
3. MPD Manifesto File.

input_video.mp4
audio.webm
Input_video_576x1024_1500k.webm
Input_video_480x854_1000k.webm
Input_video_360x640_750k.webm
input_video_manifest.mpd

Original videoinput_video.mp4You should also save to be used as a backup source of video in the future.

Serve output files

These output files can now be downloaded to a cloud repository (eg, AWS S3 or Cloudflare R2) to play. Although they can be serviced directly from the local folder, I strongly recommend storing them in a cloud storage and using CDN for causing resources for better performance. Both AWS and Cloudflare support the HTTP band requests immediately after installation.

Creating a video player compatible with Dash, in React

Nothing helps you understand how everything works as a real example. There are different ways of implementing a video player compatible with Dash, but I will focus on the approach using React.

NPM I DASHJS

Next, create a component called<Dashvideooplayer />and initialize an instance of Dash Mediaplayer, pointing it to the MPD file while mounting the component.

The Ref Call function is performed after mounting the component and inside the reverse function,PlayerrefIt will be sent to the actual copy of the Dash Mediaplayer and will be related to listeners. We also include the original MP4 URL in<Source>Item as a backup option if the browser does not support the MPEG-DASH.

If you useNext.js programs router, don't forget to add'Use Client'Customer's side of hydration on the client's side, as the video player is initialized only on the client's side.

Here's the full example:

import dashjs from 'dashjs'
Import {USECALLBACK, UserEF} from 'React'

Export const dashvideooplayer = () => {
  const Playerref = Useref ()

  const Callbackref = USECALLBACK ((Node) => {
    if (node! == null) {  
      Playerref.current = dashjs.mediaplayer (). Create ()

      Playerref.current.initialize (node, "https://example.com/uri/to/input_video_manifest.mpd", false)
  
      Playerref.current.on ('canplay', () => {
        // Upon Video Is Playable
      })
  
      Playerref.current.on ('Error', (E) => {
        // Handle Error
      })
  
      Playerref.current.on ('PlaybackStarted', () => {
        // Handle Playback Started
      })
  
      Playerref.current.on ('Playbackpaused', () => {
        // Handle Playback Paused
      })
  
      Playerref.current.on ('Playbackwaiting', () => {
        // Handle Playback Buffering
      })
    }
  }, [])

  return (
    <Video Ref = {Callbackref} width = {310} height = {548} Controls>
      <Source SRC = "https://example.com/uri/to/input_video.mp4" Type = "Video/MP4"/>
      Your Browser Does Not Support the Video Tag.
    </ Video>
  )
}

Result

Watch a video file when the network connection changes with the Fast 4G by 3G using Chrome Devtools. The resolution switches from 480p to 360p, showing how to optimize reproduction for greater or lesser accessible capacity.

Conclusion

That's it! We have just implemented a working video player compatible with DASH in React to adjust the streaming video transmission with adaptive bitrack. Again, the benefits of this are related toproductivity. When we use the ABR streaming, we ask the video smaller fragments, which allows you to play it faster than if we had to fully download the video file at first. And we did this in such a way that it supports several versions of the same video that allows us to display the best format for the user's device.