Linux and OSX Support

[opcon]

Progress

• Add OpenAL as a sound output ( OpenAL Implementation #93 )
• Fix project not building on Linux/OSX due to filename case sensitivity ( 588bc9b )
• Add support for using Apple's CoreAudio apis on OSX to load compressed formats (*.mp3, *.aac, etc.)
  • Implement a CoreAudio IWaveSource
  • Add support for creating a CoreAudio IWaveSource from a Stream
  • Add compressed formats to Codec Factory
  • Complete IDispoable implementations for OSX CoreAudio classes
• Add support for encoding files using Core Audio on OSX (probably not as important?)
• Look into using ffmpeg on Linux to load compressed formats
• Resampler for Linux/OSX
• Support for channel-conversion based on a channel matrix

[filoe]

Using ffmpeg as decoder on non-windows platforms, was on my todo list for months. I've just had a quick look at it. Unfortunately it is quite hard to implement seeking. Also, we would have to check the license of ffmpeg. I'm not sure whether it is compatible with the ms-pl or mit license.

I've added a few extra points to your progress list above.

[opcon]

FFmpeg is licensed under LGPL which is compatible with MS-PL and MIT. If we don't distribute FFmpeg with CSCore and rely on a system installation of ffmpeg, then calling PInvoke on the system installation of FFmepg is the same as dynamically linking to it, which means the FFmpeg license shouldn't have any effect.

I think CSCore would fall under this section of the LGPL license (found here):

A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License.

I'm not a lawyer though, so I could be wrong.

Thanks for the added progress points.

[opcon]

I have made some progress with decoding compressed audio on OSX using MacCore, a project that binds Apple's Objective-C API to C#. I just need to clean up the code and implement an IWaveSource decoder.

The resulting MacCore binary is quite big (~8 MB), so to avoid redistributing the binary on every platform I thought it might be a good idea to have a separate OSX dll: CSCore.OSX.dll?

This way all the OSX dependent code could be kept separate from the main CSCore dll.

[filoe]

Would you still need a reference on the MacCore assemblies? Or would you put all OSX relevant code into the CSCore.OSX project?

[opcon]

At the moment the OSX specific code references the MacCore assemblies, however it should be possible to just pull the relevant code from MacCore directly into CSCore, since it only uses a small section of the library.

If that's possible than maybe there's no point having a separate CSCore.OSX.dll, and we can have it in the main project directly?

[filoe]

Well, if we could include it directly it would be perfect. But we could start with using MacCore assemblies as seperate files. What would you say?

[opcon]

I think that's a good idea.

I should have the OSX code working soon, I'll put it in a CSCore.OSX project for now, and we can always combine them later.

[opcon]

I have implemented a CoreAudio IWaveSource which works perfectly with the OpenAL output!

Video here

Note the audio is a little choppy because I'm running OSX in a VM.

[filoe]

Wow thats awesome!
I'm going to have a look at the linux build soon. For testing out OSX, I would have to set up a VM first(and learn how to deal with a OSX system ;)).

[filoe]

Just merged your openal pull request into the openal branch. I've fixed already a few bugs. I've also created a new linux branch, containing the CSCore.Linux project which contains links to the main CSCore project: http://fs5.directupload.net/images/160528/9brj96dh.png

[opcon]

I had a look at the linux branch, nice job! Thanks for catching those bugs in the OpenAL implementation

I also edited the checklist to reflect the OSX implementation status

[opcon]

Hi @filoe, just an update on the OSX implementation. I've written a post-build step that merges CSCore.OSX.dll with MonoMac.dll (the MacCore assembly) and removes unused code from MonoMac.dll, resulting in a self-contained CSCore.OSX.dll that is only ~40kb in size. I've been using this merged dll in my projects and have not run into any issues.

I feel like it's a good idea to keep CSCore.OSX as a separate project, similar to CSCore.FFmpeg, and it could have it's own nuget package.

What are your thoughts on merging my branch osx back into the main codebase? It contains some fixes to the OpenAL implementation to correctly flush buffers after stopping, as well as the OSX implementation.

I don't know if you'd prefer to merge your openal/linux branches into master first, or if I should merge/rebase directly into one of those?

[filoe]

Hello @opcon,

since I don't want to include linux and osx support in 1.2 release, I would prefer to not merge the extra branches to master. Is it possible to seperate OpenAL from linux/osx?

Regards

[opcon]

Yeah I can do that, the actual commits themselves are separate so I can just pick the OpenAL ones.

I'll cherry pick them on to your current OpenAL branch and then submit a new pull request?

[filoe]

That would be great. I would like to seperate OpenAL from linux/osx/... development.
As soon as OpenAL is stable enough, I can merge it back to master.

[filoe]

In order to pass the unit tests and make it possible to play audio on multiple devices at once, I've did some major changes to the AL implementation.
Unfortunately, it seems like the windows OpenAL implementation is really bad. I was not able to use multiple contexts in one appdomain yet. If I switch the context, it stops playing the old context. If did some tests with ubuntu and it seems like it works fine on ubuntu. But I don't really see a huge problem in it. If it works on mac, I'm happy with it. The user would have to use Wasapi on Windows and OpenAL on the other platforms. Hopefully microsoft fixes this ...

[opcon]

Thank you for your effort, that's a massive rewrite!! It looks good!

Yeah, Windows doesn't have the greatest OpenAL support. I agree that the main use for OpenAL is to support Mac and Linux, Windows users have other options.

I found a spelling mistake in ALSoundOut, CheckForDisposed() line 613:

if (_disposed)
    throw new ObjectDisposedException("WasapiOut")

WasapiOut -> ALOut

Additionally it appears that restarting a stopped source does not work unless I call Initialize() again. It seems like it's something to do with the buffer filling, since all I hear is the same few milliseconds of audio repeated over and over.

I couldn't find a fix in the quick time I looked at it, but it's not a drastic bug since calling Initialize() before restarting a stopped source is easy to do.

Thank you again for your rewrite!

[filoe]

Thanks for reporting that bug. Did not recognice that. Gonna have a look at it.

Am 28.07.2016 um 03:08 schrieb Patrick Yates [email protected]:

Thank you for your effort, that's a massive rewrite!! It looks good!

Yeah, Windows doesn't have the greatest OpenAL support. I agree that the main use for OpenAL is to support Mac and Linux, Windows users have other options.

I found a spelling mistake in ALSoundOut, CheckForDisposed() line 613:

if (_disposed)
throw new ObjectDisposedException("WasapiOut")
WasapiOut -> ALOut

Additionally it appears that restarting a stopped source does not work unless I call Initialize() again. It seems like it's something to do with the buffer filling, since all I hear is the same few milliseconds of audio repeated over and over.

I couldn't find a fix in the quick time I looked at it, but it's not a drastic bug since calling Initialize() before restarting a stopped source is easy to do.

Thank you again for your rewrite!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[Mr-Crabman]

Is the linux branch functional?, I'm finding this kinda ambiguous.

[Bond-009]

#115 Support for .Net core?

[filoe]

The linux branch is currently functional but not with all features (see http://fs5.directupload.net/images/160528/9brj96dh.png). Therefor it is currently not released.

[Mr-Crabman]

Thanks for the information, how do I download/use it though?, I'm new to Github.

EDIT: the linux branch I mean, I probably should have specified.

[filoe]

After some time, there is finally some kind of progress:
The usage of ffmpeg shared libraries is ready to use. It was already tested on windows and linux(Ubuntu). Since I don't have any access to OSX systems I can't test it.

[opcon]

Wonderful news! I can test the ffmpeg output on OSX in the next couple of days and let you know how it goes.

Would you be happy to include the OSX output in the same release as the ffmpeg support? If so, I'll create a pull request with my OSX code.

In addition I changed the CodecFactory class to allow adding additional decoders to it at runtime, as well as coding some checks to prevent loading Windows APIs on Linux/OSX. I'll create a separate pull request for these.

[filoe]

Would be great if you could test it on OSX.

[allquixotic]

Hi, I'm on macOS Sierra, and I'm having difficulty using the filoe/cscore git master version of CSCore.Ffmpeg. I have it compiled successfully using Visual Studio for Mac beta, and I'm trying to use Mono 4.8 (part of .NET Core SDK from Microsoft) to run it.

The problem I'm having is related to DLL mapping.

I see the Interop classes try to load the following: avformat-57, avutil-55, avcodec-57, and swresample-2.

So I wrote a test.exe.config as follows:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
        <dllmap dll="avformat-57" target="libavformat.57.dylib"/>
        <dllmap dll="avcodec-57" target="libavcodec.57.dylib"/>
        <dllmap dll="avutil-55" target="libavutil.55.dylib"/>
        <dllmap dll="swresample-2" target="libswresample.2.dylib"/>
</configuration>

Unfortunately, even after setting the env var DYLD_FALLBACK_LIBRARY_PATH (equivalent of Linux LD_LIBRARY_PATH on macOS) to /usr/local/lib:., and having Homebrew FFMpeg libs (which are those same exact versions mentioned above, 57/55/2 accordingly) in /usr/local/lib and copied to the bin/Debug dir with the executable, it still doesn't work.

I get a DllNotFoundException on avformat-57.

Any tips?

EDIT: Right after posting this, I had the genius idea of renaming test.exe.config to CSCore.Ffmpeg.dylib.config. Now I'm getting somewhere:

[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: The type initializer for 'CSCore.Ffmpeg.FfmpegCalls' threw an exception. ---> System.DllNotFoundException: libavformat.57.dylib
  at (wrapper managed-to-native) CSCore.Ffmpeg.Interops.ffmpeg:av_register_all ()
  at CSCore.Ffmpeg.FfmpegCalls..cctor () [0x000a8] in <8897487fb95445f2be1733c8853021bf>:0
   --- End of inner exception stack trace ---

...At least it's actually trying to find the lib I specified in the .config file! Now to hack on it until it works.

[allquixotic]

My previous problem was that the .NET Core SDK ships a 32-bit version of Mono, and homebrew installs a 64-bit version of ffmpeg. Argh! So I used homebrew's 64-bit Mono instead.

Now a trivial 1-line test program hangs indefinitely.

new FfmpegDecoder(new MemoryStream()) hangs (without error, even with all debug/verbose stuff opened up) in the following call stack:

FfmpegDecoder(Stream) constructor ->
new AvFormatContext(FfmpegStream) constructor ->
FfmpegCalls.AvformatOpenInput(pformatContext, stream.AvioContext) ->

this line in FfmpegCalls.cs::AvformatOpenInput() hangs:

int result = ffmpeg.avformat_open_input(formatContext, "DUMMY-FILENAME", null, null);

"DUMMY-FILENAME" doesn't sound good; that looks like the code isn't complete. The FFmpeg libavformat docs don't say anything about "if you pass DUMMY-FILENAME, blah blah will happen".

This is on macOS with 64-bit ffmpeg 3.2.2 and 64-bit mono 4.6.2 and git master CSCore.

[filoe]

The ffmpeg component was never tested on mac. I currently don't even have mac build of ffmpeg.
I'm planing to do that till the next cscore 1.3.
DUMMY-FILENAME is correct because in that cause the avioContext contains the callbacks, setup by the FfmpegStream class. It is definitly finished. The only problem you're currently facing is probably the missing build of ffmpeg for Mac.

[filoe]

"Support for channel-conversion based on a channel matrix" is done.
Is there any progress on the MacOSX implemenation @opcon?

[Mr-Crabman]

How did you get the visualization sample working?, trying to play a wav file just makes the program shut down.

Compiling has no errors of any kind either.

EDIT: The xterm window displays a message when the shutdown happens, but for some reason I can't copy it, it does mention System.Windows.Forms a lot though.

[opcon]

@filoe The MacOS implementation is complete (for decoding audio files at least). I need to go through and clean up/produce an OSX branch suitable for merging, I still haven't had time to do that unfortunately. Hopefully I'll get to it in a week or so.

What's the status on merging your OpenAL branch into master?

Also I have made a few changes to the way CodecFactory works in my branch which makes supported codec detection for the current platform easier, I'll create a separate pull request for those and we can discuss those changes separately.

[Mr-Crabman]

Does anyone know how to get the winformsvisualization sample working on Linux?, I'm getting System.DllNotFoundException: libopenal.so.1, and I'm completely confused, as I'm sure I built it properly.

[filoe]

Does your system has openal installed. Which system are you using?

…

Am 25.01.2017 um 17:03 schrieb VortexGator ***@***.***>: Does anyone know how to get the winformsvisualization sample working on Linux?, I'm getting System.DllNotFoundException: libopenal.so.1, and I'm completely confused. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Mr-Crabman]

Openal doesn't come preinstalled as part of the OS?, well that explains it.

I'm getting a different error now, yay!.

shm_open() failed: No such file or directory
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"
shm_open() failed: No such file or directory
ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Protocol error
CSCore.SoundOut.AL.ALException: Could not open device "ALSA Default".

Not sure what you mean by system?.

[filoe]

I am sorry. I ve meant, which operating system are you using?

…

Am 25.01.2017 um 20:54 schrieb VortexGator ***@***.***>: Openal doesn't come preinstalled as part of the OS?, well that explains it. I'm getting a different error now, yay!. shm_open() failed: No such file or directory AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse" shm_open() failed: No such file or directory ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Protocol error Not sure what you mean by system?. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Mr-Crabman]

Xubuntu.

[Mr-Crabman]

Strange, when I restart the computer, it works temporarily, but later (few minutes) gives the error:

shm_open() failed: No such file or directory
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"
shm_open() failed: No such file or directory
ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Protocol error
CSCore.SoundOut.AL.ALException: Could not open device "ALSA Default".

After some time.

[filoe]

Is there some kind of stacktrace?

…

Am 01.02.2017 um 15:14 schrieb VortexGator ***@***.***>: Strange, when I restart the computer, it works temporarily, but later (few minutes) gives the error: shm_open() failed: No such file or directory AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse" shm_open() failed: No such file or directory ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Protocol error CSCore.SoundOut.AL.ALException: Could not open device "ALSA Default". After some arbitrary amount of time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Mr-Crabman]

@filoe

Is there some kind of stacktrace?

Not sure what you mean?, like this?:

Thread started:  #2
Thread finished:  #2
Thread started:  #3
Thread finished:  #3
shm_open() failed: No such file or directory
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"
shm_open() failed: No such file or directory
ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Protocol error

AL lib: (EE) ALCplaybackAlsa_open: Could not open playback device 'default': Connection refused
CSCore.SoundOut.AL.ALException: Could not open device "ALSA Default".
  at CSCore.SoundOut.AL.ALDevice.get_DeviceHandle () [0x00000] in cscore-linux/CSCore/SoundOut/AL/ALDevice.cs:31 
  at CSCore.SoundOut.AL.ALContext..ctor (CSCore.SoundOut.AL.ALDevice device) [0x00000] in cscore-linux/CSCore/SoundOut/AL/ALContext.cs:69 
  at CSCore.SoundOut.ALSoundOut.Initialize (CSCore.IWaveSource source) [0x00000] in cscore-linux/CSCore/SoundOut/ALSoundOut.cs:327 
  at WinformsVisualization.Form1.openToolStripMenuItem_Click (System.Object sender, System.EventArgs e) [0x00000] in cscore-linux/Samples/WinformsVisualization/Form1.cs:70 

[filoe]

I've created an issue. But it will be quite hard to reproduce and find the error. See #210.

[ststeiger]

Why don't you just directly use the ALSA API (libasound) to play a sound on Linux?
There's a C# wrapper (don't know how complete):
https://github.com/atsushieno/alsa-sharp

Example playback program (in plain C):
https://gist.github.com/ghedo/963382/815c98d1ba0eda1b486eb9d80d9a91a81d995283

You could also use PulseAudio, but PulseAudio is just a sorry-construct ontop of ALSA, and not necessarely present on small distros, while ALSA is in the kernel.

For iOS/Android, take a look at XamarinAudioManager

OSX you seem to already have covered with CoreAudio, more or less.

[allquixotic]

Why don't you just directly use the ALSA API (libasound) to play a sound on Linux?
There's a C# wrapper (don't know how complete):
https://github.com/atsushieno/alsa-sharp

Example playback program (in plain C):
https://gist.github.com/ghedo/963382/815c98d1ba0eda1b486eb9d80d9a91a81d995283

You could also use PulseAudio, but PulseAudio is just a sorry-construct ontop of ALSA, and not necessarely present on small distros, while ALSA is in the kernel.

For iOS/Android, take a look at XamarinAudioManager

OSX you seem to already have covered with CoreAudio, more or less.

The point of an audio API like cscore is to abstract away platform differences, so the developer can release software for multiple platforms with only one audio playback implementation in their app. Furthermore, this can be an issue if, like you said, there’s more than one valid audio API for a given platform and you want to support each. Only supporting ALSA on Linux is a really bad idea because most desktop systems do use PulseAudio, and the shim that lets you play audio back to PulseAudio using the ALSA API is very inefficient, error-prone, and restricts you to a subset of the ALSA API. Accessing PulseAudio directly is so much better.

There’s even more to it: on more than one occasion a platform owner has introduced a new audio API and officially deprecated an old one. If an application directly codes against a platform sound API, well, that’s a rewrite of their sound handling code to support the new API. If they use an abstraction later like cscore, the new API can be added as a backend to cscore and your app can bring in that support with minimal or no new code.

Your comment sounds like an attempt to evade this real problem, which is strange since I would normally assume that someone contributing ideas to a sound abstraction layer would at least advance arguments acknowledging the reason for that abstraction layer to exist. Indeed, if every app developer just coded against ALSA directly, we could just delete this project from GitHub and move on.

[ststeiger]

@allquixotic: Well, I think you misunderstood something there.
I didn't mean he should code using ALSA instead of CSCore.
I meant CSCore should use ALSA as backend instead of OpenAL.
That way, you don't need an additional native library (which may not be present on some systems).
Also, that way you don't have to depend (=wait for) on OpenAL for platform support.

[

[filoe]

There is no problem with supporting ALSA directly as playback interface.
Unfortunatly I currently have very limited resources on this project. I will try to maintain bugs, ... but in order to support such big new features, I would have to rely on the help or on contributions of others.
So if there is anyone willing to contribute or even maintain the library, that would be great in order to bring cscore stable to linux and mac. There is already the netstandard branch which acts as a base for that.