Check out Optimize, our new AI-powered product!

Callback for End-User WebRTC Media Quality and Status
By: , ,
24 August 2015

One of the most critical functions of a user interface is to show system status. Users need to know what is going on in a system to be able to use it in an effective manner. The 3.4 version of callstats.js will introduce a new feature that tracks the conference media quality of each participant as well as their connection status and periodically sends that information to the application in a callback function.

The new feature reports three things:

  • whether a conference participant is online or offline (Internet connectivity)
  • whether a conference participant is connected to the other participants
  • the media quality of each remote participants

Connection disruptions and quality issues are the most common but also most annoying problems that occur in conferences. They are also sometimes difficult to detect by the user because some web pages work normally when the content is in the browser cache. This new feature gives WebRTC developers an easy way to show if the connection to the conference is cut off. Media quality can also be shown to conference participants to further highlight if someone is having trouble hearing or seeing the other conference participants.

How to implement with your WebRTC application

Step 1: Extending initialize()

The initialize() API authenticates the javascript WebRTC application with the callstats.io backend, and sets up a trusted relationship with it. See the integration section on the API page for details. Today, we extended the API by adding a csStatsCallback parameter. The callback parameter is OPTIONAL.

The csStatsCallback() will be called by the callstats.js for each peerconnection independently at regular intervals. By default the interval is set as 10 seconds to make sure we do not overwhelm the app with too many messages. The data structure is discussed in the following section, however it is the responsibility of the application developer to visualise the metrics as they see fit in their application.

Step 2: Parsing Data in the csStatsCallback

The statistics object returns a JSON file, which contains the connectionState, the fabricState, and the metrics per SSRC. The connectionState represents the connectivity to the web, i.e., is the web application online or offline. The fabricState corresponds to the peerconnection state and it may be initialising, established, or disrupted. The disrupted state typically occurs when an established peerconnection loses connectivity. The reason to decouple connection and fabric state was done for deployments where peers may be accessible within their network but may be disconnected from the Internet (e.g., shopping malls, hospitals, etc.). Lastly, the callback shows the interval received bit rate, jitter, fractional loss for the inbound media and throughput, round trip time (RTT) for outbound media.

For inbound media, we map the media and network metrics to a very basic quality model, and report this in a form of excellent, fair and bad. The model is based on boundary conditions, for example, if RTT is higher than 500ms for audio, it will return bad. Or if jitter varies by too much, too often — it will be bad. Below is the JSON description of the metrics structure.

"stats": {
  "connectionState":, // one of: online, offline
  "fabricState":, // one of: initialising, established, disrupted.

  // New csStatsCallback format

  "mediaStreamTracks": [
    [0]: {
      "remoteUserID":, // associated during the addNewFabric() API
      "reportType":,   // either inbound or outbound

      // data here is parsed from the local and remote SDP
      "cname":,      // CNAME associated with the SSRC
      "msid":,       // mediastreamID associated with the SSRC
      "mediaType":,  // either video or audio

      // provided in associateMstWithUserID() API
      "usageLabel":,         //usage information
      "associatedVideoTag":, // video tag that renders the media


      "bitrate":,             // interval bitrate in kbps
      "packetRate":,          // packets per second
      "fractionLoss":,        // interval fractional loss
      "packetLossPercentage":,// Packet loss in percentage   
      "jitter":,              // interval jitter in ms
      "averageJitter":,       // Average jitter
      "rtt":,                 // round-trip time in ms.
      "averageRTT":,          // Average round-trip time in ms
      "quality":,             // interval quality as per boundary conditions.
      "audioInputLevel":,     // audio input level
      "audioOutputLevel":,    // audio output level

    },
  ...
  ]

  // **** streams are Deprecated *************************

  "streams": [
    [ActualSSRC]: {
      "remoteUserID":, // associated during the addNewFabric() API
        "reportType":,   // either inbound or outbound

        // data here is parsed from the local and remote SDP
        "cname":,      // CNAME associated with the SSRC
        "msid":,       // mediastreamID associated with the SSRC
        "mediaType":,  // either video or audio

        // provided in associateMstWithUserID() API
        "usageLabel":,         //usage information
        "associatedVideoTag":, // video tag that renders the media

        // currently available only on inbound media
        "bitrate":,         // interval bitrate in kbps
        "fractionLoss":,    // interval fractional loss
        "quality":,         // interval quality as per boundary conditions.
        "audioInputLevel":, // audio input level

        // currently available only on outbound media
        "rtt":,             // round-trip time in ms.
        "audioOutputLevel": // audio output level
    },
  ...
  ]
};

The example below shows the structure above filled up

"stats": {
  "connectionState": "online",
  "fabricStats": "established",
  "streams": [
    "387846137": {
      "remoteUserID": "3624"
      "reportType": "inbound"
      "mediaType": "audio"
      "associatedVideoTag": "videoTag1",
      "usageLabel": "mic"
      "cname": "A1gQ3iL/Zz5jJthG"
      "msid": "xHkeNqQdzOA1fDgw f0d8b8e6-a058092a8afb"
      "bitrate": 41.0,
      "fractionLoss": 1
      "quality": "excellent"
    }
    "853229723": {
      ...
      }
    ]
}

The code example shows a list of SSRCs, if they are inbound or outbound. For outbound shows the RTT, and for inbound shows the average loss rate.


var reportType = {
  inbound: 'inbound',
  outbound: 'outbound'
};

// callback function to receive the stats
var csStatsCallback = function (stats) {
  var ssrc;
  for (ssrc in stats.streams) {
    console.log("SSRC is: ", ssrc);
    var dataSsrc = stats.streams[ssrc];
    console.log("SSRC Type ", dataSsrc.reportType);
    if (dataSsrc.reportType === reportType.outbound) {
      console.log("RTT is: ", dataSsrc.rtt);
    } else if (dataSsrc.reportType === reportType.inbound) {
      console.log("Inbound loss rate is: ", dataSsrc.fractionLoss);
    }
  }
}

// initialize the callstats js API
var callStats = new callstats();
callStats.initialize(appId, appSecret, myUserId, csInitCallback, csStatsCallback);

For an example implementation check out our Github repository.

This new feature helps any WebRTC developer to show media quality inside their app to end-users. In addition, the traffic lights are an easy way for developers to visually see what kind of quality end-users see in their application. If you have any comments or questions regarding the csStatsCallback function or want to expose a particular metric which is not currently reported, contact us support-at-callstats.io.

comments powered by Disqus