processing audio input

Pixilang programming language
Post Reply
User avatar
leondustar
Posts: 98
Joined: Tue Feb 28, 2017 12:40 am
Contact:

processing audio input

Post by leondustar » Fri May 07, 2021 4:49 pm

Hi wonderful pixilang users,

What is the best approach to process audio (line-in/microphone) input?
There's some examples using audio input, and some audio-processing (examples/sound/16bit_sine.pixi) and some sunvox-player examples.
As far as I know:
  • pixilang can access audio-in
  • pixilang can play/process audio on a lowlevel (examples/sound/16bit_sine.pixi)
  • pixilang can play sunvox files (and sunvox support audio processing)
  • pixilang cannot feed audio into the 'input'-module or any module inside a sunvox module
So that leaves me to process audio input using the native `audio_callback`-approach.
Is this approach fast enough? Or does pixilang interpreter introduce too much latency for serious audio processing?

hope all is well

User avatar
NightRadio
Site Admin
Posts: 3517
Joined: Fri Jan 23, 2004 12:28 am
Location: Ekaterinburg. Russia
Contact:

Re: processing audio input

Post by NightRadio » Sat May 08, 2021 8:51 pm

Hi!
pixilang cannot feed audio into the 'input'-module or any module inside a sunvox module
It's possible too :)
See the file examples/c/test5.c (using SunVox as a filter for some user-generated signa) in the SunVox library archive.
Pixilang-version of this example will look very similar.
Unfortunately, I don't have time yet to port this example to Pixilang, but I will definitely do it soon.

So that leaves me to process audio input using the native `audio_callback`-approach.
Is this approach fast enough?
No and yes :)
It will not be very fast (with a lot of CPU usage, especially on the mobile devices) if you write signal processing in the "bare" Pixilang, using built-in operators. For example:

Code: Select all

$i = 0 while $i < $frames
{
  $out[ $i ] = $in[ $i ] * $volume
  $i + 1
}
Such code will be translated to the Pixilang bytecode, which will be executed in the Pixilang virtual machine.
Pixilang bytecode speed is much slower (from 10 to 150 times) the speed of the same code generated by the C/C++ compiler.
But...
You can use the block data processing functions, like op_cn, op_cc, op_ccn, sampler, fft, apply_filter, etc.
In this case, most of the load will fall on these built-in functions, which will execute at the native speed of machine code.
And the execution speed will be close to the speed of the same C/C++ code.
Example:

Code: Select all

copy( $out, $in, 0, 0, $frames )
op_cn( OP_MUL, $out, $volume, 0, $frames )
Block data processing functions:
https://warmplace.ru/soft/pixilang/manual.php#dataproc

List of data processing operations for op_cn(), op_cc() and op_ccn():
https://warmplace.ru/soft/pixilang/manual.php#cdproc

User avatar
leondustar
Posts: 98
Joined: Tue Feb 28, 2017 12:40 am
Contact:

Re: processing audio input

Post by leondustar » Tue May 11, 2021 10:50 am

Thanks for the extensive answer.
See the file examples/c/test5.c (using SunVox as a filter for some user-generated signal) in the SunVox library archive.
Pixilang-version of this example will look very similar.
Unfortunately, I don't have time yet to port this example to Pixilang, but I will definitely do it soon.
Im looking into test5.c as we speak.
Does this mean the Input-module actually works (with the default hardware audio-in device) when using the API?
Just to confirm, if I would run the pixilang-version of:

Code: Select all

int mod1 = sv_new_module( 0, "Input", "Input", 96, 0, 0 );
would it open the audio-in device on android?

User avatar
NightRadio
Site Admin
Posts: 3517
Joined: Fri Jan 23, 2004 12:28 am
Location: Ekaterinburg. Russia
Contact:

Re: processing audio input

Post by NightRadio » Wed May 12, 2021 8:01 am

would it open the audio-in device on android?
Yes. Here is a more complete code:

Code: Select all

int mod1 = sv_new_module( 0, "Input", "Input", 96, 0, 0 );
sv_update_input();
sv_update_input() - handle input ON/OFF requests to enable/disable input ports of the sound card (for example, after the Input module creation). Call it from the main thread only, where the SunVox sound stream is not locked.

User avatar
leondustar
Posts: 98
Joined: Tue Feb 28, 2017 12:40 am
Contact:

Re: processing audio input

Post by leondustar » Wed May 12, 2021 5:34 pm

THank you for answering.
sv_update_input() - handle input ON/OFF requests to enable/disable input ports of the sound card (for example, after the Input module creation).
understood, thanks.
Call it from the main thread only, where the SunVox sound stream is not locked.
Aha, so pixilang a multithreaded environment by using callbacks.

Post Reply