Audio Plugin

The Audio Plugin is a key module within the Audio Subsystem, designed to enable seamless integration of external audio devices. The schematic below illustrates a simplified software architecture of the audio application. This architecture is composed of two main components: the Application and the Audio Subsystem. The Application handles user interaction logic and manages the input/output data streams, while the Audio Subsystem provides a variety of functional modules designed for different types of data streams.

As illustrated in the figure above, the Audio Plugin module interfaces with both the External Device module and the Audio Path module to ensure seamless integration. The External Device module, located in the application layer, is responsible for managing external devices such as external amplifiers and DSPs. In contrast, the Audio Path module handles the control of various data streams and manages internal devices within the SoC, including hardware codecs, gateways and DSPs.

The purpose of the Audio Plugin module is to facilitate collaboration between customized external devices and internal devices within the SoC.

When dealing with SoC internal devices, certain limitations must be considered. For instance, the internal hardware codec may produce a slight popping noise during its shut-down process. If this noise is transmitted to a working amplifier, it will be amplified and perceived by the end-user. To mitigate this, the amplifier controller must ensure that the amplifier is turned off before initiating the shut-down procedure of the internal hardware codec. Since the amplifier controller, with its specific customized control details, is typically located in the application layer, it cannot directly interact with the internal hardware codec. To address this constraint, the amplifier participates in several handshake procedures initiated by the Audio Plugin module. This ensures that the amplifier is properly synchronized with the SoC internal devices throughout the lifecycle of an Audio Path instance.

The concept of the Audio Plugin involves treating the External Device module as a component plugged into the Audio Path instance. It performs specific actions during particular occasions, as detected by the Audio Path module. Once an action is completed, the External Device module must notify the Audio Plugin module, which in turn notifies the Audio Path module. The overall interaction flow is depicted as below.

Policy

Before exploring the utilization of Audio Plugin APIs, it is important to introduce the notion of policies. A policy functions as a contract between the External Device module and the Audio Plugin module. It details the specific actions that must be performed at designated occasions within an Audio Path instance of a particular category. The occasions and action handlers are predefined by the Audio Plugin module, while the category is a universally shared definition among all components within the Audio Subsystem. Consequently, a policy can be understood as a combination of a category, an occasion, and an action handler.

typedef struct t_audio_plugin_policy
{
    T_AUDIO_CATEGORY              category;
    T_AUDIO_PLUGIN_OCCASION       occasion;
    P_AUDIO_PLUGIN_ACTION_HANDLER handler;
} T_AUDIO_PLUGIN_POLICY;

Occasion

An occasion refers to the various stages within the lifecycle of an Audio Path instance. Typically, an Audio Path instance includes both an analog part and a digital part. The name of each occasion indicates the state of these internal components. The occasions are defined as follows:

typedef enum t_audio_plugin_occasion
{
    AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON,
    AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_OFF,
    AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON,
    AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF,
    AUDIO_PLUGIN_OCCASION_NUM,
} T_AUDIO_PLUGIN_OCCASION;

The figure below illustrates the relationship between exposed occasions and the internal device states. From the perspective of an Audio Path instance, the starting and stopping procedures are performed sequentially. When starting, a typical Audio Path instance sequentially enables the internal hardware codec, gateway, and DSP. Conversely, when stopping, it sequentially disables the internal DSP, gateway, and hardware codec. To seamlessly integrate with the internal control flow, the Audio Plugin instance must perform the appropriate actions at the designated times.

Note

The AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON is mandatory.
Since the data stream will arrive after the AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON occasion, the External Device module must ensure that it is ready to process the data stream before acknowledging the AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON.

Action Handler

An action handler is provided by the External Device module and registered with the Audio Plugin module during the creation of an Audio Plugin instance. The following is the prototype of the action handler.

typedef struct t_audio_plugin_param
{
    uint32_t dac_sample_rate;
    uint32_t adc_sample_rate;
} T_AUDIO_PLUGIN_PARAM;

typedef void (*P_AUDIO_PLUGIN_ACTION_HANDLER)(T_AUDIO_PLUGIN_HANDLE handle,
                                              T_AUDIO_PLUGIN_PARAM  param,
                                              void                 *context);

The action handler, together with the policy, is designed and implemented on a per Audio Path instance basis to facilitate the transfer of specific information from the Audio Path instance to the External Device module. Currently, the Audio Plugin module provides the sample rates for both the decoding and encoding directions of an Audio Path instance as parameters. The External Device moudle can utilize these parameters if they are applicable to its custom operations.

The context parameter of the action handler is used for establishing a handshake between the Audio Plugin module and the External Device module. It tracks the current policy and its associated Audio Path instance. The application layer must reserve the context parameter and pass it back to the Audio Plugin module to indicate that the action specified in the current policy has been completed.

API Usage

Below are the APIs provided by the Audio Plugin module, along with a brief introduction to their usage.

To create an Audio Plugin instance for the External Device module, register a policy table.

T_AUDIO_PLUGIN_HANDLE audio_plugin_create(const T_AUDIO_PLUGIN_POLICY *policies, uint32_t count)

It is important to successfully invoke this API in order to obtain a handle for further operations.

Note

Please note that the policies parameter cannot be an empty array.

Below is a sample code snippet:

bool app_amp_init(void)
{
    //...
    app_amp_db.plugin_handle = audio_plugin_create(app_amp_policies,
                                                   sizeof(app_amp_policies) / sizeof(app_amp_policies[0]));
    if (app_amp_db.plugin_handle == NULL)
    {
        return false;
    }
    //...
    return true;
}

A handshake should be completed once the registered action has been successfully executed.

void audio_plugin_msg_send(T_AUDIO_PLUGIN_HANDLE handle, void *context)

When an occasion is detected, the internal control flow of the Audio Path instance will be blocked while waiting for the Audio Plugin instance to perform its registered action. Once the action is completed, the audio_plugin_msg_send API should be invoked to resume the internal control flow of the Audio Path instance.

Note

To successfully complete the handshake, the value of the context parameter must match the one passed to the registered P_AUDIO_PLUGIN_ACTION_HANDLER callback.

Below is a sample code snippet:

static void app_amp_poweron(T_AUDIO_PLUGIN_HANDLE handle, T_AUDIO_PLUGIN_PARAM param, void *context)
{
    app_amp_db.ref_cnt++;
    if (app_amp_db.state == APP_AMP_STATE_ON)
    {
        // already powered on, send the context back immediately
        audio_plugin_msg_send(app_amp_db.plugin_handle, context);
    }
    else
    {
        // save the context parameter
        app_amp_db.poweron_context = context;

        // start the customized AMP power on flow
        power_on_step_1();
        power_on_step_2();
        ...
    }
}

void app_amp_power_on_finished_callback()
{
    // send the context back when the power on flow is finished
    audio_plugin_msg_send(app_amp_db.plugin_handle, app_amp_db.poweron_context);
}

Please release the plugin instance when the External Device module no longer needs it.
```
void audio_plugin_release(T_AUDIO_PLUGIN_HANDLE handle)
```

Sample

The sdk\src\sample\rws\app_amp.c file implements the power-on and power-off procedures for an analog AMP using the Audio Plugin module. The analog AMP can be powered on and off using a GPIO switch. It must be powered on before playing any audio stream and powered off when there is no audio stream to play.

Due to certain hardware restrictions, a delay is required during the power-on and power-off procedures. In the software, this delay is implemented using two timers for the power-on procedure and one timer for the power-off procedure.

AMP Policy

static const T_AUDIO_PLUGIN_POLICY app_amp_policies[] =
{
    /* category */          /* occasion */                           /* action handler */
    { AUDIO_CATEGORY_AUDIO, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_VOICE, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_ANALOG, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_TONE, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_VP, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_LLAPT, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },
    { AUDIO_CATEGORY_ANC, AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON, app_amp_poweron },

    { AUDIO_CATEGORY_AUDIO, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_VOICE, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_ANALOG, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_TONE, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_VP, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_LLAPT, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },
    { AUDIO_CATEGORY_ANC, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON, app_amp_run },

    { AUDIO_CATEGORY_AUDIO, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_VOICE, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_ANALOG, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_TONE, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_VP, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_LLAPT, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
    { AUDIO_CATEGORY_ANC, AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF, app_amp_poweroff },
};

static void app_amp_run(T_AUDIO_PLUGIN_HANDLE handle, T_AUDIO_PLUGIN_PARAM param, void *context)
{
    audio_plugin_msg_send(handle, context);
}

As shown in the code snippet, the AMP should be powered on during the AUDIO_PLUGIN_OCCASION_ANALOG_DOMAIN_ON occasion and powered off during the AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_OFF occasion. This rule applies to all the categories listed from AUDIO_CATEGORY_AUDIO to AUDIO_CATEGORY_ANC.

The AUDIO_PLUGIN_OCCASION_DIGITAL_DOMAIN_ON occasion is mandatory, although the only task is to complete the handshake.

AMP Power On Procedure

This figure illustrates the power-on procedure for the AMP. Please note that when the AMP receives a power-on request, if it is not already powered on (meaning no Audio Path instances are currently using this AMP), it should first save the context parameter of P_AUDIO_PLUGIN_ACTION_HANDLER and then initiate its power-on process.

AMP Power Off Procedure

This figure illustrates the power-off procedure for the AMP. Please note that when the AMP receives a power-off request, if the reference count reaches 0 (meaning no Audio Path instances are currently using this AMP), it should first save the context parameter of P_AUDIO_PLUGIN_ACTION_HANDLER and then initiate its power-off process.