User talk:Jusa

(Questions)
m
 
(3 intermediate revisions not shown)
Line 1: Line 1:
-
Page 15 of this PDF http://linuxplumbersconf.org/2009/slides/Jyri-Sarha-audio_miniconf_slides.pdf shows a block diagram of the N900 audio system (or at least what I assume is the N900 audio system). Is there any chance you can expand on which algorithms (e.g. stw, xprot, eap, aep, drc, eq, iir, fir etc) match with the blocks marked "Mic EQ", "music processing", "recording processing", "transducer processing" and "call audio processing"?
+
As a preface, I may mix up things between Fremantle and Harmattan, given that it has already been quite a while when working with the former. But as a general rule, Fremantle and Harmattan are (from PulseAudio components point of view) pretty close to each other. For the algorithms, when you look at the meego modules, wherever there is algorithm processing in Harmattan side, there likely was pretty identical processing in Fremantle as well. Meego modules are an evolution of the modules written for Fremantle, and one of the goals was also to separate the closed source algorithms from the source tree. To achieve this, "algorithm-hook"s were introduced, which just execute the algorithms from separate modules (shipped with Harmattan).
 +
MeeGo != Harmattan, but the PulseAudio modules were named meego when open sourcing them. So in Harmattan you have source trees pulseaudio-modules-meego (containing open source code) and pulseaudio-modules-nokia (containing proprietary algorithms).
-
I have been examining the Meego code and the Harmattan code (both the open bits in pulseaudio-modules-meego and the closed bits in pulseaudio-nokia) with a view to possibly using the blobs or parts of them for the Neo900 audio work (since they are more open than the ones on the N900). Comparing what Meego has to what Fremantle has, it would appear as though the Meego and Harmattan blobs are missing functions labeled stw. Are you aware of any other pieces in the Meego or Harmattan blobs that are missing or that might have been built differently? (ignoring anything to do with different versions or compilers, PulseAudio, libc or other things, just talking about the actual algorithms and such in the pulseaudio-nokia blobs).
+
'''Page 15 of this PDF http://linuxplumbersconf.org/2009/slides/Jyri-Sarha-audio_miniconf_slides.pdf shows a block diagram of the N900 audio system (or at least what I assume is the N900 audio system). Is there any chance you can expand on which algorithms (e.g. stw, xprot, eap, aep, drc, eq, iir, fir etc) match with the blocks marked "Mic EQ", "music processing", "recording processing", "transducer processing" and "call audio processing"?'''
-
Or for that matter anything that is present in the Meego or Harmattan blobs but isn't used on Fremantle?
+
Mic EQ:
 +
* fir/iir
-
Other than the cmtspeech stuff, are you aware of anything in the nokia Fremantle Pulseaudio blobs that talks to or is dependent on the N900 cellular modem?
+
Music processing:
 +
* mudrc
 +
Recording processing:
 +
* mudrc
 +
* (I think there was agc/automatic gain control at some point... hm)
-
Which of these Nokia algorithms are necessary for safe system functioning and which are just going to give not-so-nice audio if we dont have them? Obviously xprot is something we probably need in order to protect the speakers/earpiece/etc from damage but what else? (the cellular modem we are using in the Neo900 already does all sorts of voice processing so identifying which bits we need and which bits we can leave to the cellular modem will save us a lot of time)
+
Transducer processing:
 +
* iir/fir
 +
* xprot
 +
Call audio processing:
 +
* aep
-
Are you aware of open-source implementations of algorithms similar to the proprietary Nokia ones?
+
'''I have been examining the Meego code and the Harmattan code (both the open bits in pulseaudio-modules-meego and the closed bits in pulseaudio-nokia) with a view to possibly using the blobs or parts of them for the Neo900 audio work (since they are more open than the ones on the N900). Comparing what Meego has to what Fremantle has, it would appear as though the Meego and Harmattan blobs are missing functions labeled stw. Are you aware of any other pieces in the Meego or Harmattan blobs that are missing or that might have been built differently? (ignoring anything to do with different versions or compilers, PulseAudio, libc or other things, just talking about the actual algorithms and such in the pulseaudio-nokia blobs).'''
-
Do you know if the stereo widening stuff is actually used on the N900 on Fremantle? All of the .parameters files that control the blobs set x-maemo.stereo-widening = "false" which I assume means "turn it off". If its in fact not used that means we can ignore it (it would also explain why its not even compiled in for Meego)
+
stw - Stereo Widening. Kind of useful with N900, since that has stereo speakers, dropped with N9 since N9 has only mono speaker. It might be that it was dropped from N900 as well, if it wasn't seen as enough improvement to the sound compared to the resource usage.
 +
Algorithms themselves are pretty much the same, although I think AEP saw quite big change. All the configuration parameter binary blobs in the parameters files were just fed to the algorithms as is. Tuning was done with proprietary plugin to Fast Trace (iirc).
 +
'''Or for that matter anything that is present in the Meego or Harmattan blobs but isn't used on Fremantle?'''
-
nb_eeq is "narrowband ear equalizer"
+
'''Other than the cmtspeech stuff, are you aware of anything in the nokia Fremantle Pulseaudio blobs that talks to or is dependent on the N900 cellular modem?'''
-
nb_meq is "narrowband mic equalizer"
+
-
wb_eeq is "wideband ear equalizer"
+
-
wb_meq is "wideband mic equalizer"
+
-
What is the difference between "narrowband" and "wideband" here?
+
-
What is xprot displacement?
+
 +
Not that I know of.
-
Can you tell me which of these sets of parameters will be tied to the specific details of the microphone in the N900 or of the FMTX in the N900? (bluetooth is digital so changing bluetooth chips shouldn't change the way the audio needs to be processed and we are using the same earpiece, speakers, headphone, lineout and tvout setup so those wont change)
+
'''Which of these Nokia algorithms are necessary for safe system functioning and which are just going to give not-so-nice audio if we dont have them? Obviously xprot is something we probably need in order to protect the speakers/earpiece/etc from damage but what else? (the cellular modem we are using in the Neo900 already does all sorts of voice processing so identifying which bits we need and which bits we can leave to the cellular modem will save us a lot of time)'''
 +
I don't think any of the algorithms are needed if the hardware is run conservatively enough. XProt helps with getting more loudness safer, but it would be just as well to run the speaker a bit quieter.
-
aep
+
From audio quality perspective, voice call algorithms are really crucial. It is impossible to get device certified without echo cancellation and friends, and almost impossible without EQs. I remember it was really hard, even with the super equipment the tuning guys had.
-
nb_eeq
+
-
nb_meq
+
-
wb_eeq
+
-
wb_meq
+
-
mumdrc.ul
+
-
mumdrc.dl
+
-
limiter.ul
+
-
limiter.dl
+
-
xprot
+
-
That will tell us which bits we need to dig deep into and retune/replace for our needs
+
-
retune/replace
+
 +
'''Are you aware of open-source implementations of algorithms similar to the proprietary Nokia ones?'''
-
oh and one more thing, if you know anything at all about alsaped, alsaped.conf or alsa-policy-enforcement, feel free to share :)
+
Not really. Maybe speex.
 +
 
 +
'''Do you know if the stereo widening stuff is actually used on the N900 on Fremantle? All of the .parameters files that control the blobs set x-maemo.stereo-widening = "false" which I assume means "turn it off". If its in fact not used that means we can ignore it (it would also explain why its not even compiled in for Meego)'''
 +
 
 +
Answered above.
 +
 
 +
'''nb_eeq is "narrowband ear equalizer" nb_meq is "narrowband mic equalizer" wb_eeq is "wideband ear equalizer" wb_meq is "wideband mic equalizer" What is the difference between "narrowband" and "wideband" here? What is xprot displacement?'''
 +
 
 +
Narrowband/wideband: Voice call bandwith. DocScrutinizer05 already wrote about this in IRC.
 +
 
 +
XProt displacement - don't remember. Something about the speaker coil safe traversal probably.
 +
 
 +
'''Can you tell me which of these sets of parameters will be tied to the specific details of the microphone in the N900 or of the FMTX in the N900? (bluetooth is digital so changing bluetooth chips shouldn't change the way the audio needs to be processed and we are using the same earpiece, speakers, headphone, lineout and tvout setup so those wont change)'''
 +
 
 +
'''aep nb_eeq nb_meq wb_eeq wb_meq mumdrc.ul mumdrc.dl limiter.ul limiter.dl xprot. That will tell us which bits we need to dig deep into and retune/replace for our needs
 +
retune/replace'''
 +
 
 +
* mumdrc.ul: Multiband DRC uplink (run in record module)
 +
* mumdrc.dl: Multiband DRC downlonk (run in music module)
 +
* limiter.ul: Limiter (don't remeber more specifically) uplink
 +
* limiter.dl: same with downlink
 +
* aep: Algorithm package especially meant for voice call processing, includes echo cancelling, comfort noise, whatnot.
 +
* I think eap is also mentioned somewhere, iirc that contains the algorithms for DRC etc.
 +
 
 +
I think the best way to verify these is to look at the different places in Harmattan where meego_algorithm_hook_fire() is called, like said above, those places pretty much match what's happening with Fremantle as well.
 +
 
 +
'''oh and one more thing, if you know anything at all about alsaped, alsaped.conf or alsa-policy-enforcement, feel free to share :)'''
 +
 
 +
IIRC alsaped and alsa-policy-enforcement were doing the same thing as alsa-mixer stuff in Harmattan. That is, set ALSA interface switches for achieving given output/input routing. Porting the alsaped configuration to recent PulseAudio alsa-mixer configuration should be doable (and might actually already be done for N900 Nemo port, not sure.)
 +
 
 +
I'd probably need to take a look how Harmattan stuff was laid out to better remember these details, but as I currently don't have that available I'm just running with my (bad) memory. I do have N9 though (probably even with stock image flashed) so I could take a look at Harmattan root filesystem at some point as well.

Latest revision as of 10:08, 26 October 2014

As a preface, I may mix up things between Fremantle and Harmattan, given that it has already been quite a while when working with the former. But as a general rule, Fremantle and Harmattan are (from PulseAudio components point of view) pretty close to each other. For the algorithms, when you look at the meego modules, wherever there is algorithm processing in Harmattan side, there likely was pretty identical processing in Fremantle as well. Meego modules are an evolution of the modules written for Fremantle, and one of the goals was also to separate the closed source algorithms from the source tree. To achieve this, "algorithm-hook"s were introduced, which just execute the algorithms from separate modules (shipped with Harmattan).

MeeGo != Harmattan, but the PulseAudio modules were named meego when open sourcing them. So in Harmattan you have source trees pulseaudio-modules-meego (containing open source code) and pulseaudio-modules-nokia (containing proprietary algorithms).

Page 15 of this PDF http://linuxplumbersconf.org/2009/slides/Jyri-Sarha-audio_miniconf_slides.pdf shows a block diagram of the N900 audio system (or at least what I assume is the N900 audio system). Is there any chance you can expand on which algorithms (e.g. stw, xprot, eap, aep, drc, eq, iir, fir etc) match with the blocks marked "Mic EQ", "music processing", "recording processing", "transducer processing" and "call audio processing"?

Mic EQ:

  • fir/iir

Music processing:

  • mudrc

Recording processing:

  • mudrc
  • (I think there was agc/automatic gain control at some point... hm)

Transducer processing:

  • iir/fir
  • xprot

Call audio processing:

  • aep

I have been examining the Meego code and the Harmattan code (both the open bits in pulseaudio-modules-meego and the closed bits in pulseaudio-nokia) with a view to possibly using the blobs or parts of them for the Neo900 audio work (since they are more open than the ones on the N900). Comparing what Meego has to what Fremantle has, it would appear as though the Meego and Harmattan blobs are missing functions labeled stw. Are you aware of any other pieces in the Meego or Harmattan blobs that are missing or that might have been built differently? (ignoring anything to do with different versions or compilers, PulseAudio, libc or other things, just talking about the actual algorithms and such in the pulseaudio-nokia blobs).

stw - Stereo Widening. Kind of useful with N900, since that has stereo speakers, dropped with N9 since N9 has only mono speaker. It might be that it was dropped from N900 as well, if it wasn't seen as enough improvement to the sound compared to the resource usage.

Algorithms themselves are pretty much the same, although I think AEP saw quite big change. All the configuration parameter binary blobs in the parameters files were just fed to the algorithms as is. Tuning was done with proprietary plugin to Fast Trace (iirc).

Or for that matter anything that is present in the Meego or Harmattan blobs but isn't used on Fremantle?

Other than the cmtspeech stuff, are you aware of anything in the nokia Fremantle Pulseaudio blobs that talks to or is dependent on the N900 cellular modem?

Not that I know of.

Which of these Nokia algorithms are necessary for safe system functioning and which are just going to give not-so-nice audio if we dont have them? Obviously xprot is something we probably need in order to protect the speakers/earpiece/etc from damage but what else? (the cellular modem we are using in the Neo900 already does all sorts of voice processing so identifying which bits we need and which bits we can leave to the cellular modem will save us a lot of time)

I don't think any of the algorithms are needed if the hardware is run conservatively enough. XProt helps with getting more loudness safer, but it would be just as well to run the speaker a bit quieter.

From audio quality perspective, voice call algorithms are really crucial. It is impossible to get device certified without echo cancellation and friends, and almost impossible without EQs. I remember it was really hard, even with the super equipment the tuning guys had.

Are you aware of open-source implementations of algorithms similar to the proprietary Nokia ones?

Not really. Maybe speex.

Do you know if the stereo widening stuff is actually used on the N900 on Fremantle? All of the .parameters files that control the blobs set x-maemo.stereo-widening = "false" which I assume means "turn it off". If its in fact not used that means we can ignore it (it would also explain why its not even compiled in for Meego)

Answered above.

nb_eeq is "narrowband ear equalizer" nb_meq is "narrowband mic equalizer" wb_eeq is "wideband ear equalizer" wb_meq is "wideband mic equalizer" What is the difference between "narrowband" and "wideband" here? What is xprot displacement?

Narrowband/wideband: Voice call bandwith. DocScrutinizer05 already wrote about this in IRC.

XProt displacement - don't remember. Something about the speaker coil safe traversal probably.

Can you tell me which of these sets of parameters will be tied to the specific details of the microphone in the N900 or of the FMTX in the N900? (bluetooth is digital so changing bluetooth chips shouldn't change the way the audio needs to be processed and we are using the same earpiece, speakers, headphone, lineout and tvout setup so those wont change)

aep nb_eeq nb_meq wb_eeq wb_meq mumdrc.ul mumdrc.dl limiter.ul limiter.dl xprot. That will tell us which bits we need to dig deep into and retune/replace for our needs retune/replace

  • mumdrc.ul: Multiband DRC uplink (run in record module)
  • mumdrc.dl: Multiband DRC downlonk (run in music module)
  • limiter.ul: Limiter (don't remeber more specifically) uplink
  • limiter.dl: same with downlink
  • aep: Algorithm package especially meant for voice call processing, includes echo cancelling, comfort noise, whatnot.
  • I think eap is also mentioned somewhere, iirc that contains the algorithms for DRC etc.

I think the best way to verify these is to look at the different places in Harmattan where meego_algorithm_hook_fire() is called, like said above, those places pretty much match what's happening with Fremantle as well.

oh and one more thing, if you know anything at all about alsaped, alsaped.conf or alsa-policy-enforcement, feel free to share :)

IIRC alsaped and alsa-policy-enforcement were doing the same thing as alsa-mixer stuff in Harmattan. That is, set ALSA interface switches for achieving given output/input routing. Porting the alsaped configuration to recent PulseAudio alsa-mixer configuration should be doable (and might actually already be done for N900 Nemo port, not sure.)

I'd probably need to take a look how Harmattan stuff was laid out to better remember these details, but as I currently don't have that available I'm just running with my (bad) memory. I do have N9 though (probably even with stock image flashed) so I could take a look at Harmattan root filesystem at some point as well.

Retrieved from "http://wiki.maemo.org/User_talk:Jusa"