Audio Effect

This document mainly introduces the relevant content of Audio Effect, including its architecture, operating principle with the Application layer, and implementation.

As shown in the figure below, the Audio Effect is a high-level function module of the Audio Subsystem in parallel with the Stream, Notification, Signal and Volume modules. The Audio Effect module includes EQ, NREC, WDRC and Sidetone. The Effect modules act on the Stream modules or the Signal modules, generating actual effects through the algorithm engine inside the DSP.

../../../_images/audio_subsystem_architecture.png — Audio Subsystem Architecture

Architecture Overview

The following diagram provides a simple description of the Audio Effect architecture. The Application invokes API from Audio Effect submodules to enable, disable, or update specific effects. On the other hand, the Application invokes API from Stream submodules to start, stop or restart specific data stream. If the Application wishes to apply a certain effect to a data stream, it should actively invoke the API from the Stream submodules to attach it. The underlying AudioPath module will deliver the specific effect information attached to the data stream to the DSP at an appropriate time.

../../../_images/audio_effect_architecture.png — Audio Effect Architecture

Operating Principle

This section mainly describes the effect related interaction flow between Application and Audio Subsystem, and Audio Subsystem internal principles.

Interaction with Application

The supported effects include EQ, NREC, WDRC and Sidetone. The interaction process with Application is similar for these effects, and EQ will be used as an example for explanation.

As shown in the figure below, the Application calls eq_create() to create an EQ and calls eq_enable() to enable it. Then, the Application calls audio_track_effect_attach() to attach the EQ to the target playback data stream. In addition, during the runtime of the playback data stream, the Application can invoke eq_set() to update the EQ coefficients, invoke eq_disable() to disable the EQ effect temporarily, or invoke eq_release() to release the EQ effect.

Note

For more detailed information about EQ, such as EQ types and configuration steps, please refer to EQ.

../../../_images/eq_interaction_with_app.png — EQ Interaction with APP

Internal Principle

In the Audio Subsystem, the internal principles of the supported effects are similar. Here, we will use EQ combined with Audio Track as an example to explain.

The figure below illustrates the internal interaction for EQ. When the API provided by the EQ module is called, it triggers a series of internal events within the Audio Effect and Audio Track. These events are ultimately translated into commands for the DSP.

../../../_images/eq_interaction_internal.png — EQ Internal Interaction

Implementation

This section introduces the inheritance relationships derived from the abstract module Audio Effect and the internal details of binding an Audio Effect to a data stream.

Effect Inheritance Hierarchy

The diagram below illustrates the inheritance hierarchy of the effect-related software modules in the Audio Subsystem. As shown in the diagram, the Audio Effect serves as the base module, providing a set of general interfaces. The derived modules, such as EQ, NREC, WDRC, and Sidetone, extend the general interfaces provided by the base module according to their respective needs. For example, eq_enable() is an extension of audio_effect_enable, indicating a desire to activate the EQ; eq_set() is an extension of audio_effect_update, representing an update to the EQ parameters.

../../../_images/inheritance_hierarchy.png — Effect Inheritance Hierachy

Binding to Data Streams

The Audio Subsystem supports dynamic binding of effects and data streams. As shown in the figure below, any specific effect can be bound to any kind of data stream. Furthermore, the binding and unbinding of data streams and effects can be dynamically managed at runtime, allowing Applications to control effects more flexibly. This design uses an abstract Audio Effect as a mediator to decouple the data stream model from specific effects, facilitating the independent expansion of the Audio Subsystem along the dimensions of data streams and effects.

../../../_images/binding_to_data_stream.png — Effect Binding to Data Streams

Taking Audio Track as an example, each Audio Track instance can aggregate one or more Audio Effect instances, thereby binding to specific Audio Effect instances. The Application explicitly calls the following set of APIs to bind and unbind Audio Effects to an Audio Track instance.

bool audio_track_effect_attach(T_AUDIO_TRACK_HANDLE    handle,
                               T_AUDIO_EFFECT_INSTANCE instance);

bool audio_track_effect_detach(T_AUDIO_TRACK_HANDLE    handle,
                               T_AUDIO_EFFECT_INSTANCE instance);

Note

A data stream can be attached to multiple Audio Effect instances simultaneously, but an Audio Effect instance can only be bound to one data stream at a time. This is because each Audio Effect instance is uniquely associated with the attached data stream instance. As shown in the figure below, an Audio Effect instance internally records the specific Audio Track instance it is truly bound to.

../../../_images/unique_effect_owner.png — Unique Effect Owner at A Time