Tuesday, June 3, 2014

Introduction of audio pipeline in ChromiumOS

1. System architecture

Chromium OS is an open source project, based on Linux kernel with Chrome browser as the main application interface.
The communication mechanism between Chromium OS and its browser currently is via D-bus.

2. Chrome OS audio server (CRAS)

2.1 Introduction

  • CRAS sits between Chrome and ALSA 
    • Output: Chrome connects to this audio server, sends audio data to it, then audio server renders this to the selected ALSA device through alsa-lib. 
    • Input: the server will get audio samples from alsa and forward them to Chrome. 
  • Three main requirements: 
    • Low Latency: 20ms latency and waking up every 10ms 
    • Low CPU usage: Less than two percent of the CPU of a CR48 is used while playing back audio 
    • Dynamic audio routing

2.2 Code structure

  • server - the source for the sound server 
  • libcras - client library for interacting with cras 
  • common - files common to both the server and library 
  • tests - tests for cras and libcras
  • source code location: chromiumos/src/third_party/adhd/cras

2.3 Block diagram

2.4 Run-time behavior

2.4.1 Initialization

2.4.2 Audio IO thread

Audio IO thread is responsible for reading data from each stream, mixing and sending to output device. Note: Digital signal processing(DSP) is applied post-mix, i.e. global effects.

2.4.3 DSP thread

There is a dedicated thread to handle DSP related requests.

2.4.4 server main thread

2.4.5 Audio routing

2.4.5.1 Jack plugged in

2.4.5.2 Switch to Bluetooth

3. Chromium Media Pipeline

3.1 Introduction

Pull-based pipeline, consists of data source, demuxers, decoders and renders.

3.2 Pipeline integration into Webkit

3.3 Pipeline state machine

3.4 Pipeline decomposition

3.4.1 class diagram

3.4.2 Audio Sink to CRAS

3.4.3 Example of initiating a new playback stream

3.5 Multi-channel processing

Currently IO devices enumerated with CRAS server only support up to stereo output. If a client requests to add a stream which isn’t compatible with iodev’s audio format, then CRAS client will perform format conversion automatically.

4. Interactions between UI and CRAS

Because all applications run in browser tabs, Chromium has defined an extension API currently for audio to control CRAS parameters, such as setting volume, etc.

References