We never seem to overflow the source buffer now; this might have been a
leftover from a bug that was covered by Vitaly's fixes?
Removing this conditional makes the resampler 10-20% faster. Left an
assert in there for debug builds, in case this still happens.
83454c821f