(I thought padding size ranged from 5 frames to ~30 frames (based around
RESAMPLER_ZERO_CROSSINGS, which is 5), but it's actually between 512 and
several thousands (based on RESAMPLER_SAMPLES_PER_ZERO_CROSSING)). It gets
big fast when downsampling.
Previously, the padding was silence, which was a problem when streaming since
you would sample a little bit of this silence between each buffer.
We still need a means to get padding data for the right hand side, but this
patch makes the resampler output more correct.
This time it's using real math from a real whitepaper instead of my previous
amateur, fast-but-low-quality attempt. The new resampler does "bandlimited
interpolation," as described here: https://ccrma.stanford.edu/~jos/resample/
The output appears to sound cleaner, especially at high frequencies, and of
course works with non-power-of-two rate conversions.
There are some obvious optimizations to be done to this still, and there is
other fallout: this doesn't resample a buffer in-place, the 2-channels-Sint16
fast path is gone because this resampler does a _lot_ of floating point math.
There is a nasty hack to make it work with SDL_AudioCVT.
It's possible these issues are solvable, but they aren't solved as of yet.
Still, I hope this effort is slouching in the right direction.
This would cause playback problems in certain situations, such as on the
Raspberry Pi. The device that the wait was added for seems to not benefit from
it in modern times, and standard desktop Linux seems to do the right thing
when a USB device is unplugged now, without this patch.
Fixes Bugzilla #3599.
Simon Hug
This issue actually raises the question if this API change (requirement of initialized audio subsystem) is breaking backwards compatibility. I don't see the documentation saying it is needed in 2.0.5.
"Major changes, roughly in order of appearance:
- Use float math everywhere, instead of promoting to double and casting back
all the time.
- Conserve sound energy when downmixing any channel into two other channels.
- Add a QuadToStereo filter. (The previous technique of reusing StereoToMono
never worked, since it assumed an incorrect channel layout for 4.0.)
- Add a 71to51 filter. This removes just under half of the cases the previous
code would silently break in.
- Add a QuadTo51 filter. More silent breakage fixed.
- Add a 51to71 filter, removing another almost-half of the silently broken
cases.
- Add 8 to the list of values SDL_SupportedChannelCount will accept.
- Change SDL_BuildAudioCVT's channel-related logic to handle every case, and
to actually fail if it fails instead of silently corrupting sound data and/or
crashing down the road."
(Note that SDL doesn't otherwise support 7.1 audio yet, but hopefully it will
soon and the 7.1 converters are an important piece of that. --ryan.)
Fixes Bugzilla #3727.
David Ludwig
I've created a new set of patches. I am happy to create more, if it would help.
One version only copies 'size'.
A second version copies both 'size' and 'silence'. When looking over the documentation for SDL_OpenAudio in SDL_audio.h, it mentioned that both 'size' and 'silence' were things that SDL_OpenAudio would calculate.
Regarding *both* patches, I did notice that SDL 1.2 appears to have always modified desired's size and silence fields. The SDL wiki, at https://wiki.libsdl.org/SDL_OpenAudio#Remarks , does note:
manuel.montezelo
Original bug report (note that it was against 2.0.0, it might have been fixed in between): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733015
--------------------------------------------------------
Package: libsdl2-2.0-0
Version: 2.0.0+dfsg1-3
Severity: normal
Tags: patch
I have occasional crashes here caused by the X11 backend of SDL2. It seems to
be caused by the X11_Pending function trying to add a high number (> 1024)
file descriptor to a fd_set before doing a select on it to avoid busy waiting
on X11 events. This causes a buffer overflow because the file descriptor is
larger (or equal) than the limit FD_SETSIZE.
Attached is a possible workaround patch.
Please also keep in mind that fd_set are also used in following files which
may have similar problems.
src/audio/bsd/SDL_bsdaudio.c
src/audio/paudio/SDL_paudio.c
src/audio/qsa/SDL_qsa_audio.c
src/audio/sun/SDL_sunaudio.c
src/joystick/linux/SDL_sysjoystick.c
--------------------------------------------------------
On Tuesday 24 December 2013 00:43:13 Sven Eckelmann wrote:
> I have occasional crashes here caused by the X11 backend of SDL2. It seems
> to be caused by the X11_Pending function trying to add a high number (>
> 1024) file descriptor to a fd_set before doing a select on it to avoid busy
> waiting on X11 events. This causes a buffer overflow because the file
> descriptor is larger (or equal) than the limit FD_SETSIZE.
I personally experienced this problem while hacking on the python bindings
package for SDL2 [1] (while doing make runtest). But it easier to reproduce in
a smaller, synthetic testcase.
Simon Hug
Some code in SDL loads libraries with SDL_LoadObject to get more information or use newer APIs. SDL_LoadObject may fail, set an error message and SDL will continue with some fallback code. Since SDL will overwrite the error or exit the function with a return value that indicates success, the error form SDL_LoadObject for the optional stuff might as well be cleared right away.
kdrakehp
The attached patch adds capture support to the sndio backend.
The patch also allows the `OpenDevice' function to accept arbitrary device names.
Simon Hug
There's a chance that an audio conversion from many channels to a few can use more than 9 audio filters. SDL_AudioCVT has 10 SDL_AudioFilter pointers of which one has to be the terminating NULL pointer. The SDL code has no checks for this limit. If it overflows there can be stack or heap corruption or a call to 0xa.
Attached patch adds a function that checks for this limit and throws an error if it is reached. Also adds some documentation.
Test parameters that trigger this issue:
AUDIO_U16MSB with 224 channels at 46359 Hz
V
AUDIO_S16MSB with 6 channels at 27463 Hz
The fuzzer program I uploaded in bug 3667 has more of them.
We don't fill buffers just to throw them away during shutdown now, we let the
AudioQueue free its own buffers during disposal (which fixes possible warnings
getting printed to stderr by CoreAudio), and we stop the queue after running
any queued audio during shutdown, which prevents dropping the end of the
audio playback if you opened the device with an enormous sample buffer.
Fixes Bugzilla #3555.
We need more than two buffers to flip between if they are small, or CoreAudio
won't make any sound; apparently it needs X milliseconds of audio queued when
it needs to play more or it drops any queued buffers. We are currently
guessing 50 milliseconds as a minimum, but there's probably a more proper
way to get the minimum time period from the system.
Fixes Bugzilla #3656.
This gracefully recovers when a device format is changed, and will switch
to the new default device if the current one is unplugged, etc.
This does not handle when a new default device is added; it only notices
if the current default goes away. That will be fixed by implementing the
stubbed-out MMNotificationClient_OnDefaultDeviceChanged() function.
* alsa hotplug thread is low priority
* give a chance for other threads to catch up when audio playback is not progressing
* use nonblocking for alsa audio capture
There is a bug with SDL hanging when an audio capture USB device is removed, because poll never returns
We will throw away the data anyhow, but some apps depend on the callback
firing to make progress; testmultiaudio.c, if nothing else, is an example
of this.
Capture also will now fire the callback in these conditions, offering nothing
but silence.
Apps can check SDL_GetAudioDeviceStatus() or listen for the
SDL_AUDIODEVICEREMOVED event if they want to gracefully deal with
an opened audio device that has been unexpectedly lost.
This is just enough to get you through a file that just used the extended
header for float or int data. It doesn't handle all the other things that
you expect from this header, like 24-bit samples inside a 32-bit container
or speaker masks.
This should remain binary compatible with Windows XP, as we dynamically
load anything we need and fall back to DirectSound/WinMM/XAudio2 if not
available.
Walter van Niftrik
We have found that since SDL 2.0.5 the audio callback thread is created with a very small stack size. In our application this is leading to stack overflows.
We believe there is a bug at http://hg.libsdl.org/SDL/file/391fd532f79e/src/audio/SDL_audio.c#l1132, where the is_internal_thread flag appears to be inverted.
This defaults to the internal SDL resampler, since that's the likely default
without a system-wide install of libsamplerate, but those that need more can
tweak this.
This currently favors libsamplerate over the fast path (quality over speed),
but I'm not sure that's the correct approach, as there may be surprising
changes in performance metrics depending on what packages are available on
a user's system. That being said, currently, the only thing with access to
SDL_AudioStream is an SDL audio device's thread, and it might be mostly idle
otherwise, so maybe this is generally good.
Turns out that iterating from 0 to channels-1 was a serious performance hit!
These cases now tend to match or beat the original audio resampler's speed!
This allows us to avoid an extra copy, allocate less memory and reduce cache
pressure. On the downside: we have to do a lot of tapdancing to resample the
buffer in reverse when the output is growing.
It's expensive and (hopefully) unnecessary. If this becomes an overflow
problem, we could multiply both values by 0.5f before adding them, but let's
see if we can get by without the extra multiplication first.
We never seem to overflow the source buffer now; this might have been a
leftover from a bug that was covered by Vitaly's fixes?
Removing this conditional makes the resampler 10-20% faster. Left an
assert in there for debug builds, in case this still happens.
Removed some needless things ("len / sizeof (Uint8)"), and made sure the
int32 -> float code uses doubles to avoid working with large integer values
in a 32-bit float.
Lukasz Biel
Tried to compile SDL2 using newest version of VS.
Got:
SDL_audiocvt.obj : error LNK2019: unresolved external symbol memcpy referenced in function SDL_ResampleCVT
1>E:\Users\dotPo\Lib\SDL\VisualC\x64\Release\SDL2.dll : fatal error LNK1120: 1 unresolved externals
whole compilation process: http://pastebin.com/eWDAvBce
Steps to reproduce:
clone http://hg.libsdl.org/SDL using tortoise hg,
open SDL\VisualC\SDL.sln,
when promted if should retarget solution click ok,
select release x64 build type,
Build/Build Solution
attempt 2, using Visual Studio cmake support:
open folder SDL\
select release x64 build type,
run CMake\Build CMakeLists.txt
build fails
When switched to debug build type, buils succeeds in both cases.
VS 2017 is still beta.
It causes audio pops if you're converting in chunks (and needs to
allocate/initialize/free on each convert). We'll either adjust this interface
when we break ABI for 2.1 to make this usable, or publish the SDL_AudioStream
API for those that want a streaming solution.
In the meantime, the "simple" resampler produces "good enough" audio without
pops and doesn't have to be initialized, so that'll do for now on the
SDL_AudioCVT interface.
There was a draft of this where it did audio conversion into the final buffer,
if there was enough room available past what you asked for, but that interface
got removed, so the parameters didn't make sense (and we were using the
wrong one in any case, too!).
Coriiander
This notice is about a misplaced comment.
Often times when we use an #if #endif sequence, the #endif is followed by a comment to indicate what #if statement it belonged to. The SDL_xaudio2.c file contains a misplaced comment, as follows (I stripped the other comments):
#ifdef __GNUC__
# define SDL_XAUDIO2_HAS_SDK 1
#elif defined(__WINRT__)
# define SDL_XAUDIO2_HAS_SDK
#include "SDL_xaudio2.h"
#else
#if 0
#include <dxsdkver.h>
#if (!defined(_DXSDK_BUILD_MAJOR) || (_DXSDK_BUILD_MAJOR < 1284))
# pragma message("Your DirectX SDK is too old. Disabling XAudio2 support.")
#else
# define SDL_XAUDIO2_HAS_SDK 1
#endif
#endif
#endif /* 0 */
That final /* 0 */ should be moved one line up. Like this (I tabbed it out for you to make it more clear):
This was a leftover of simplifying the resamplers down from autogenerated
code; I forgot to make something that the generator hardcoded into something
variable.
Fixes Bugzilla #3507.
Vitaly Novichkov
Okay, when I researched code and algorithm, I tried to replace condition "while(dst >= target)" with "while(dst > target)" and crashes are gone.
Seems on some moments it tries to write into the place before memory block begin, therefore phantom crashes appearing after some moments.
This no longer uses a script to generate code for every possible type
conversion or resampler. This caused a bloat in binary size and and compile
times. Now we use a handful of more generic functions and assume staying in
the CPU cache is the most important thing anyhow.
This shrinks the size of the final build (in this case: macOS X amd64, -Os to
optimize for size) by 15%. When compiling on a single core, build times drop
by about 15% too (although the previous cost was largely hidden by multicore
builds).
Apparently some systems see "hw:", some see "default:" and some see
"sysdefault:" (and maybe others!). My workstation sees both "hw:" and
"sysdefault:" ...
Try to find a prefix we like and prioritize the prefixes we (think) we want
most. If everything else fails, if there's a "default" (not a prefix) device
name, list that by itself so the user gets _something_ here.
If we can't find a prefix we like _and_ there's no "default" device, report
no hardware found at all.
Simon Sandstr?m
As stated in Summary. The switch statement will execute the default case and set a SDL error message: "SDL_MixAudio(): unknown audio format".
There are atleast two more problems here:
1. SDL_MixAudioFormat does not notify the user that an error has occured and that a SDL error message was set. It took me awhile to understand why I couldn't mix down the volume on my AUDIO_U16LSB formatted audio stream.. until I started digging in the SDL source code.
2. The error message is incorrect, it should read: "SDL_MixAudioFormat(): unknown audio format".
This tends to be a frequent spot where drivers hang, and the waits were
often unreliable in any case.
Instead, our audio thread now alerts the driver that we're done streaming audio
(which currently XAudio2 uses to alert the system not to warn about the
impending underflow) and then SDL_Delay()'s for a duration that's reasonable
to drain the DMA buffers before closing the device.
This tries to make SDL robust against device drivers that have hung up,
apps don't freeze in catastrophic (but not necessarily uncommon) conditions.
Now we detach the audio thread and let it clean up and don't care if it
never actually runs to completion.
James Zipperer
The problem I was seeing was that the the ALSA hotplug thread would call SDL_RemoveAudioDevice, but my application code was not seeing an SDL_AUDIODEVICEREMOVED event to go along with it. To fix it, I added some code into SDL_RemoveAudioDevice to call SDL_OpenedAudioDeviceDisconnected on the corresponding open audio device. There didn't appear to be a way to cross reference the handle that SDL_RemoveAudioDevice gets and the SDL_AudioDevice pointer that SDL_OpenedAudioDeviceDisconnected needs, so I ended up adding a void *handle field to struct SDL_AudioDevice so that I could do the cross reference.
Is there some other way beside adding a void *handle field to the struct to get the proper information for SDL_OpenedAudioDeviceDisconnected?
James Zipperer
Close the audio device before waiting for the audio thread to complete, which fixes a situation where the audio thread never completes
Add an additional check in the audio thread to see if the device is enabled and bail out if the device is no longer enabled
The Apple TV remote is currently exposed as a joystick with its touch surface treated as two axes. Key presses are also generated when its buttons and touch surface are used.
A new hint has been added to help deal with deciding whether to background the app when the remote's menu button is pressed: SDL_HINT_APPLE_TV_CONTROLLER_UI_EVENTS.
AudioQueues are available in Mac OS X 10.5 and later (and iOS 2.0 and later).
Their API is much more clear (and if you don't mind the threading tapdance
to get its own CFRunLoop) much easier to use in general for our purposes.
As an added benefit: they seemlessly deal with format conversion in ways
AudioUnits don't: for example, my MacBook Pro's built-in microphone won't
capture at 8000Hz and the AudioUnit version wouldn't resample to hide this
fact; the AudioQueue version, however, can handle this.
Moved this code from winmm into core so both can use it.
DirectSound (at least on Win10) also returns truncated device names, even
though it's handed in as a string pointer and not a static-sized buffer. :/
Otherwise, if you had a massive, one-time queue buildup, the memory from that
remains allocated until you close the device. Also, if you are just using a
reasonable amount of space, this would previously cause you to reallocate it
over and over instead of keeping a little bit of memory around.
I think this was important for SDL 1.2 because some targets needed
special device memory for DMA buffers or locked memory buffers for use in
hardware interrupts or something, but since it just defines to SDL_malloc
and SDL_free now, I took it out for clarity's sake.
- It's now always called if device->hidden isn't NULL, even if OpenDevice()
failed halfway through. This lets implementation code not have to clean up
itself on every possible failure point; just return an error and SDL will
handle it for you.
- Implementations can assume this->hidden != NULL and not check for it.
- implementations don't have to set this->hidden = NULL when done, because
the caller is always about to free(this).
- Don't reset other fields that are in a block of memory about to be free()'d.
- Implementations all now free things like internal mix buffers last, after
closing devices and such, to guarantee they definitely aren't in use anymore
at the point of deallocation.
Turns out that libartsc isn't thread-safe, so if we run a capture and playback
device at the same time, it often crashes in arts's internal event loop.
We could throw mutexes around the read/write calls, but these are meant to
block, so one device could cause serious latency and stutter in the other.
Since this audio target isn't in high-demand (Ubuntu hasn't offered a libartsc
package for years), I'm just backing out the capture support. If someone needs
it, they can pull it out of the revision history.
(We probably never noticed because this is meant to block until it fully
writes a buffer, and would only trigger an issue if we had a short write
that wasn't otherwise an error condition.)
This allows us to set an explicit stack size (overriding the system default
and the global hint an app might have set), and remove all the macro salsa
for dealing with _beginthreadex and such, as internal threads always set those
to NULL anyhow.
I've taken some guesses on reasonable (and tiny!) stack sizes for our
internal threads, but some of these might turn out to be too small in
practice and need an increase. Most of them are simple functions, though.
This is kind of nasty, because ALSA reports dozens of "devices" that aren't
really things you'd ever want, or things that should be listed this way, but
the default path still works as before, and it at least allows these devices
to be available to apps.
This does not handle hotplugging yet. You get a device list at init time
and that's it.
xaudio2 is not linked against sdl but the sdk already handles dynamically loading (XAudio2Create is inlined and just loads a com object). Updated SDL_xaudio2.c
"UWP" appears to be Microsoft's new name for WinRT/Windows-Store APIs.
This set of changes updates SDL's WinRT backends to support the Win10 flavor
of WinRT. It has been tested on Win10 on a desktop. In theory, it should
also support Win10 on other devices (phone, Xbox One, etc.), however further
patches may be necessary.
This adds:
- a set of MSVC 2015 project files, for use in creating UWP apps
- modifications to various pieces of SDL, in order to compile via MSVC 2015 +
the Win10 API set
- enables SDL_Window resizing and programmatic-fullscreen toggling, when using
the WinRT backend
- WinRT README updates
- disable compiling in XAudio2 support. We both need the DX SDK to make this code plus we need to work out the runtime dependency problem this code bring in on windows (needing the DX runtime installed).
CR: SamL
The internal function SDL_EGL_LoadLibrary() did not delete and remove a mostly
uninitialized data structure if loading the library first failed. A later try to
use EGL then skipped initialization and assumed it was previously successful
because the data structure now already existed. This led to at least one crash
in the internal function SDL_EGL_ChooseConfig() because a NULL pointer was
dereferenced to make a call to eglBindAPI().
Ryan C. Gordon
Sam Lantinga
Philipp Wiesemann
Alex Szpakowski
Juha Kuikka
Alon Zakai
David Ludwig
Alfred Reynolds