Notification

This document mainly introduces the relevant content of Notification, including its architecture, operating principle with the Application layer, and implementation.

As shown in the figure below, Notification is a high-level function module of the Audio Subsystem in parallel with the Stream, Signal, Effect, and Volume modules. The Notification module includes Ringtone, Voice Prompt, and TTS. Ringtone is generated by the Frequency Modulation (FM) synthesizer. Voice Prompt is pre-recorded voice interaction data. TTS is text speech generated by the speech synthesizer.

Architecture Overview

The following diagram provides a simple description of the Notification architecture. The Application invokes Notification API to play, stop, cancel, or flush notifications. These activities are then serialized into a queue by the Notification Activity Manager. The Notification Scheduler receives requests from the Notification Activity Manager and controls the lifecycle of Notification instances by performing the following tasks:

1. Manipulating the Notification Buffers attached to Notification instances.

2. Manipulating the Audio Paths attached to Notification instances.

Operating Principle

The operating principle part mainly describes the detailed interaction flow between Notification and Application, and Notification module internal interactions.

Ringtone

As shown in the figure below, the Application calls ringtone_play() to play a Ringtone, and receives the AUDIO_EVENT_RINGTONE_STARTED event when the Ringtone is started. Once the Ringtone playback is completed, the Application will receive the AUDIO_EVENT_RINGTONE_STOPPED event. In addition, the Application can invoke ringtone_flush() to flush all ringtones in the queue, invoke ringtone_cancel() to cancel the specific Ringtone in the queue, and invoke ringtone_stop() to stop the playing Ringtone.

The figure below illustrates the internal interaction for Ringtone. The Notification module loads Ringtone Coefficient from flash and sets the timestamp to DSP, and receives the data complete event from DSP.

Voice Prompt

For Voice Prompt, the interaction flow with the Application is the same as Ringtone. The Application calls play, cancel, flush, and stop API for Voice Prompt.

The internal flow for Voice Prompt is different from Ringtone. After the Voice Prompt is started, the Notification module loads Voice Prompt data from flash, and sends the first Voice Prompt data and timestamp to DSP. When the Notification module receives a request from DSP, it sends the next Voice Prompt data. The last Voice Prompt data payload length is 0, DSP receives the last data and replies with the data complete event.

TTS

For TTS, the Application calls tts_create() to create a TTS instance. Then the Application calls tts_start() to start the TTS and will receive the AUDIO_EVENT_TTS_STARTED event. The Application calls tts_play() to send TTS data to the Notification module. The Notification module sends events to the Application indicating Notification Buffer level state. The chapter Notification Buffer Level Control shows more details. The Application can call tts_stop() to stop the TTS. After the Application receives the AUDIO_EVENT_TTS_STOPPED event, the TTS instance will be released through the tts_destroy() interface.

The internal interaction flow of TTS is similar to the Voice Prompt, but TTS data is from the Application.

Notification Start and Stop Flow

The pictures below show the details of Notification start and stop flow. The Application invokes the API to trigger the Notification start flow. Once the Notification completes playing, the audio path will send AUDIO_PATH_EVT_DATA_EMPTY to the Notification module and trigger the stop flow.

Implementation

This chapter introduces the Ringtone Coefficient, Notification Buffer level control, synchronization flow, and the detailed process of the DSP module.

Ringtone Coefficient

Here is an example of a Ringtone composed of four tone notes. The Ringtone coefficient layout is as below.

uint8_t coefficient_vector = [0, 4,
                              25, 67, 110,
                              20, 69, 100,
                              15, 71, 90,
                              20, 72, 75];

Ringtone Coefficient

Coef	Value	Note
Coef[0]	0	Timbre type, val = 0~5
Coef[1]	4	Number of Tones, val=1~24
Coef[2]	25	Tone Length, val=0~127 (*10mSec)
Coef[3]	67	Tone Note, val=21~127
Coef[4]	110	Velocity, Val = 0~127

MCU will load Ringtone coefficient from APP CFG image, then send them to DSP. The Frequency Modulation (FM) synthesizer of DSP will generate the Ringtone according to the coefficient. APP CFG image is generated from MCUConfig Tool, users can configure Ringtone and customize own Ringtone by following these steps.

1. Users can see the ringtones available for selection in ‘Available Ringtone’. Click [Tone Config] to enter the Ringtone configuration page.

2. The tool provides 45 non-editable ringtones, and it also supports Ringtone customization. Click [Add more by customer] to add a new Ringtone. Click [Value] to configure the Ringtone coefficient.

3. The picture below shows the Ringtone coefficient, which is the parameter written into the APP CFG image.

Notification Buffer Level Control

The Notification Buffer is attached to the Notification instance for Notification data storage and management. Before using the Notification Buffer, its buffer size, prequeue threshold, lower threshold, and upper threshold need to be configured.

There are two types of data management for the Notification Buffer: stream type and frame type. Currently, Voice Prompt and TTS use the stream and frame types respectively. When the water level of the Notification Buffer changes and reaches the threshold, its status will be notified to the Notification module through an event in T_NOTIFICATION_BUFFER_EVENT for flow control.

typedef enum t_notification_buffer_type
{
   NOTIFICATION_BUFFER_TYPE_FRAME      = 0x00, /* buffer in frame type */
   NOTIFICATION_BUFFER_TYPE_STREAM     = 0x01, /* buffer in stream type */
} T_NOTIFICATION_BUFFER_TYPE;

typedef enum t_notification_buffer_event
{
   NOTIFICATION_BUFFER_EVENT_PREQUEUE      = 0x00, /* buffer under prequeue */
   NOTIFICATION_BUFFER_EVENT_PLAY          = 0x01, /* buffer prequeue completed and ready to play */
   NOTIFICATION_BUFFER_EVENT_LOW           = 0x02, /* buffer watermark under the low threshold */
   NOTIFICATION_BUFFER_EVENT_HIGH          = 0x03, /* buffer watermark above the upper threshold */
   NOTIFICATION_BUFFER_EVENT_F_PLAY        = 0x04, /* buffer watermark under the upper threshold */
   NOTIFICATION_BUFFER_EVENT_EMPTY         = 0x05, /* buffer empty */
   NOTIFICATION_BUFFER_EVENT_E_PLAY        = 0x06, /* buffer watermark above the low threshold */
   NOTIFICATION_BUFFER_EVENT_FULL          = 0x07, /* buffer full */
} T_NOTIFICATION_BUFFER_EVENT;

The diagram below shows the flow control interaction process for the Notification Buffer of Voice Prompt. In the case of Voice Prompt, the Notification Buffer is filled to capacity each time, and when the buffer is empty, the remaining data continues to be loaded from the flash until the Voice Prompt playback is completed.

For TTS, the status of the water level is further notified to the Application layer, which handles data flow control based on the event.

Notification Synchronization Flow

To achieve synchronous Notification playback in both buds, it is necessary to synchronize the status of both buds through three handshakes.

Note

Apart from TTS, both Ringtone and Voice Prompt support synchronous playback for both buds and share the same synchronization flow. The following synchronization flow uses Voice Prompt as an example.

Both the primary and secondary buds can call the API voice_prompt_play() to play synchronized Voice Prompt. However, the secondary needs to first notify the primary of the Voice Prompt information via NOTIFICATION_REMOTE_PUSH_RELAY, then the primary bud will initiate the synchronization process.

The first poll-ack handshake aims to inquire about the secondary state and synchronously start the audio paths for both buds. After the primary and secondary audio paths both enter the running state, in the secondary handshake, the primary informs the secondary the synchronous timestamp to play.

The third handshake is that the primary and secondary DSP open SPORT at the synchronous timestamp.

DSP Module Process

The following diagram illustrates the processing path of Voice Prompt data in the stereo scenario, taking the example where both the Voice Prompt and audio are set to output on SPORT 0.

1. After the MCU sends the Voice Prompt data to the DSP, the data is first decoded into 1-channel PCM data at the Decode Terminal.

2. In a stereo scenario, the Channel Reorder module duplicates the 1-channel Voice Prompt data, outputting it as 2-channel data. The D0 and D1 represent the data of the two channels respectively.

3. After that, the Signal Process module is applied. Currently, the DSP does not perform any specific signal processing for the Voice Prompt.

4. Then data enters the Post Mixer module. In the audio mixing Voice Prompt scenario, the audio data and Voice Prompt data are mixed in the Post Mixer module. After that, it goes through the Gateway Process module and is ultimately output to the Codec via SPORT 0.

The following diagram illustrates the processing path of Voice Prompt data in the TWS scenario, taking the example where the Voice Prompt is set to output on SPORT 2 and audio is set to output on SPORT 0.

The differences with the stereo scenario:

1. The Channel Reorder module for the Voice Prompt does not apply duplication, directly outputting 1-channel PCM data. The audio data will be converted from 2-channel data to 1-channel data by the Channel Reorder module.

2. The audio data and Voice Prompt data are output to the Codec via SPORT 0 and SPORT 2 respectively, and mixed in the Codec module.

Notification Mode

typedef enum t_ringtone_mode
{
  RINGTONE_MODE_AUDIBLE   = 0x00, /**< Ringtone is enabled and audible. */
  RINGTONE_MODE_SILENT    = 0x01, /**< Ringtone is disabled and silent. */
  RINGTONE_MODE_FIXED     = 0x02, /**< Ringtone is enabled, but volume is fixed and not adjustable. */
} T_RINGTONE_MODE;

typedef enum t_voice_prompt_mode
{
  VOICE_PROMPT_MODE_AUDIBLE   = 0x00, /**< Voice Prompt is enabled and audible. */
  VOICE_PROMPT_MODE_SILENT    = 0x01, /**< Voice Prompt is disabled and silent. */
  VOICE_PROMPT_MODE_FIXED     = 0x02, /**< Voice Prompt is enabled, but volume is fixed and not adjustable. */
} T_VOICE_PROMPT_MODE;

Ringtone and Voice Prompt have three modes, which are audible mode, silent mode, and volume fixed mode. Audible mode is the default mode where the Ringtone and Voice Prompt can be played and stopped, and volume can be adjusted, muted, and unmuted. Voice Prompt and Ringtone cannot be played, and their volume cannot be adjusted in silent mode. Fixed mode will make Voice Prompt and Ringtone volume unadjustable.

Modes can be set and retrieved by calling the following APIs:

T_RINGTONE_MODE ringtone_mode_get(void);
bool ringtone_mode_set(T_RINGTONE_MODE mode);

T_VOICE_PROMPT_MODE voice_prompt_mode_get(void);
bool voice_prompt_mode_set(T_VOICE_PROMPT_MODE mode);