Time & Latency questions

[mgstauffer]

Hi,

I'm new to audio programming and PortAudio. I've done realtime
multi-threaded MIDI programming for a while, so am not completely green in
these things, which should help with audio, but I'm still unsure of some
basics. Your insights would be a great help.

I'm concerned about latency issues for a realtime (not TRUE realtime, but as
close to it as possible) app that will support simultaenious MIDI, audio and
ReWire.

While reading and playing with PortAudio (thanks for the helpful patest*
code!), I've got some questions:

PaStreamCallbackTimeInfo::currentTime - Does this represent the time
PortAudio internally got the callback from the driver and is passing it to
my callback? Is this time direct from the driver's reported time? Or is it
modified in anyway by PortAudio? If from the driver, then it's
device/driver-dependent? If I have more than one device open, each
device/driver could return a different time in this member during callbacks
that happened simultaneously? If so, I figure that's why it said somewhere
in the docs that sync between multiple devices/drivers is difficult?

Regarding PaStreamCallbackTimeInfo::outputBufferDacTime - assuming that the
driver is working properly, that the OS lets the driver work optimally, and
that I return from the callback with the output buffer in time, does this
time represent the time (in terms of the driver's audio time) that the first
sample in the buffer will actually play from the DAC? How reliable are these
times? I suppose that's driver/device dependent?

PaStreamCallbackTimeInfo::inputBufferAdcTime: this is the time that the
first sample in the input buffer was received/generated by the driver,
correct? It's not the time the buffer gets to your callback, which wouldn't
be too useful, I suppose? So currentTime - inputBufferAdcTime is a measure
of the driver's acquistion-to-callback input latency?

What's the resolution of the timestamps? I guess that's driver/device
dependent? Would it generally be the sample rate, making sample-accurate
timestamping straightforward? I read that the ASIO driver on Windows uses
the 1msec multimedia timer, so I guess sample-accurate for that would be
impossible?

In patest_latency.c, I was messing with outputParameters.suggestedLatency,
and it's clear to hear how that effects the playback latency when playing
the ASCII keyboard. But I noticed that as I changed the suggestedLatency,
the buffer length (and sample rate) stayed the same, which means the latency
in this case is not buffer-length dependent, and the driver (Windows MME/DX)
is adding an artificial latency because of the suggestedLatency request. Is
that correct? I otherwise I would have figured the output latency was
dependent on the buffer length and number of buffers in use. Does the same
apply for suggestedLatency on input devices?

Generally, I'm thinking ones buffer has to be *minimally* long enough to
cover any latency between the driver's acquistion of data, its delivery to
your callback, and its receipt of the new output buffer and transfer to its
DAC, to ensure that by the time it finishes playing an output buffer, it has
a new one ready to go. Any processing you do in the callback before
returning the output buffer will only add to this minimum. Is that correct?

Thanks!

Cheers,
Michael

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[Ross Bencina-3]

Hi Michael

> PaStreamCallbackTimeInfo::currentTime - Does this represent the time
> PortAudio internally got the callback from the driver and is passing it to
> my callback? Is this time direct from the driver's reported time? Or is it
> modified in anyway by PortAudio? If from the driver, then it's
> device/driver-dependent? If I have more than one device open, each
> device/driver could return a different time in this member during
> callbacks
> that happened simultaneously? If so, I figure that's why it said somewhere
> in the docs that sync between multiple devices/drivers is difficult?

As you have noticed, PaStreamCallbackTimeInfo::currentTime is
underspecified -- and probably inconsistently implemented. However the way I
have implemented it for ASIO, WMME and DS is that it is the time at which
the host API calls into PortAudio (even if PortAudio then calls the user
callback multiple times). For some APIs (ASIO) this time is supplied by the
driver, in other cases PortAudio calls a system timer function to get this
information -- this is also why PA only provides a per-stream time
reference, not a global time reference.


> Regarding PaStreamCallbackTimeInfo::outputBufferDacTime - assuming that
> the
> driver is working properly, that the OS lets the driver work optimally,
> and
> that I return from the callback with the output buffer in time, does this
> time represent the time (in terms of the driver's audio time) that the
> first
> sample in the buffer will actually play from the DAC?

That is the intention yes.

> How reliable are these
> times? I suppose that's driver/device dependent?

Yes it is dependent on a number of factors. For some host APIs we can only
estimate based on the size of the buffers PA uses -- the OS and driver may
introduce additional buffering, and to the best of my knowledge we don't
currently account for that (we could probably get some pretty decent
estimates about kmixer latency on different versions of Windows for
example). For other host APIs, the latency value and timings are provided by
the driver (I think ASIO and CoreAudio do this) so in that case it can
potentially be quite accurate -- although highly driver dependent.


> PaStreamCallbackTimeInfo::inputBufferAdcTime: this is the time that the
> first sample in the input buffer was received/generated by the driver,
> correct? It's not the time the buffer gets to your callback, which
> wouldn't
> be too useful, I suppose?

It is supposed to be the time the first input sample hit the ADC.

> So currentTime - inputBufferAdcTime is a measure
> of the driver's acquistion-to-callback input latency?

Correct. But in practice in many cases inputBufferAdcTime will just be
calculated by subtracting a precomputed constant from currentTime (see
latency values in PaStreamInfo), since few host APIs provide this
information directly (CoreAudio does.. not sure about others).


> What's the resolution of the timestamps? I guess that's driver/device
> dependent? Would it generally be the sample rate, making sample-accurate
> timestamping straightforward? I read that the ASIO driver on Windows uses
> the 1msec multimedia timer, so I guess sample-accurate for that would be
> impossible?

Correct. The resolution is host-api dependent. The ASIO spec uses 64 bit
ints for time and could represent things at a higher resolution, but since
the multimedia timer is the most stable (AFAIK) on Windows, they use that.

> In patest_latency.c, I was messing with outputParameters.suggestedLatency,
> and it's clear to hear how that effects the playback latency when playing
> the ASCII keyboard. But I noticed that as I changed the suggestedLatency,
> the buffer length (and sample rate) stayed the same, which means the
> latency
> in this case is not buffer-length dependent, and the driver (Windows
> MME/DX)
> is adding an artificial latency because of the suggestedLatency request.
> Is
> that correct? I otherwise I would have figured the output latency was
> dependent on the buffer length and number of buffers in use.

I'm not sure what you mean by "buffer length" here. If you mean the
framesPerBuffer parameter -- this is a user defined buffer size which will
only vary if you change the framesPerBuffer value explicitly (or use
paFramesPerBufferUnspecified). PA uses various heuristics to select the best
combination of host buffer sizes and buffer counts depending on the host
API.


> Does the same
> apply for suggestedLatency on input devices?

The behavior would be symmetric yes.


> Generally, I'm thinking ones buffer has to be *minimally* long enough to
> cover any latency between the driver's acquistion of data, its delivery to
> your callback, and its receipt of the new output buffer and transfer to
> its
> DAC, to ensure that by the time it finishes playing an output buffer, it
> has
> a new one ready to go.

Yes, I think that  would be the case in a double-buffered system where there
is no additional buffering inside the driver (or inside PA). If you have
more than two buffers then you can use shorter buffers. Not many driver
models actually work this way as far as i can see (WMME uses multiple buffer
scatter-gather, DirectSound uses a circular buffer).

> Any processing you do in the callback before
> returning the output buffer will only add to this minimum. Is that
> correct?

I'm not sure what you mean by this. The way I think of it is as follows:

In a double buffered system you need to pass the following crude invariant,
which is basically the usual throughput vs. latency trade off:

totalTimeToComputeBuffer < (totalTimeToPlayBuffer - systemOverhead -
communicationCost)

Where systemOverhead is other high-priority stuff the OS has to do, and
communicationCost is the audio driver-specific cost of invoking the callback
and making the buffer available to hardware -- this includes scheduling
latency+jitter and any latency between passing the buffer back to the driver
and when the hardware can play it.

If you use more than two buffers (or more than two callbacks per ringbuffer
period), some of the communication cost is masked since you are no longer
critically dependent on the scheduling latency and other factors in order to
make sure the hardware isn't starved.


If you havn't looked at it already you might get something out of the
following paper:
PortAudio and Media Synchronisation
http://www.portaudio.com/docs/portaudio_sync_acmc2003.pdf

Cheers

Ross.

> Thanks!
>
> Cheers,
> Michael
>
> _______________________________________________
> Portaudio mailing list
> Portaudio@...
> http://techweb.rfa.org/mailman/listinfo/portaudio
>

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[mgstauffer]

Hi Ross,

Thanks very much for your detailed reply. It's all very good to know. The
variability between drivers gives me angst of course, but that seems
unavoidable! A couple further questions below...

<snip>

> > In patest_latency.c, I was messing with
> > outputParameters.suggestedLatency,
> > and it's clear to hear how that effects the playback
> > latency when playing the ASCII keyboard. But I noticed that as I changed
the
> > suggestedLatency, the buffer length (and sample rate) stayed the same,
which
> > means the latency in this case is not buffer-length dependent, and the
driver
> > (Windows MME/DX) is adding an artificial latency because of the
suggestedLatency request.
> > Is that correct? I otherwise I would have figured the output latency was
dependent
> > on the buffer length and number of buffers in use.

>
> I'm not sure what you mean by "buffer length" here. If you mean the
> framesPerBuffer parameter -- this is a user defined buffer
> size which will
> only vary if you change the framesPerBuffer value explicitly (or use
> paFramesPerBufferUnspecified). PA uses various heuristics to
> select the best
> combination of host buffer sizes and buffer counts depending
> on the host
> API.

Yes, by "buffer length" I mean framesPerBuffer, as passed into
PaStreamCallback. From what I've read (including your article on Media
Synchronization, thanks), I thought the output latency in msec would depend
on the number of buffers, the length of the buffers, and sample rate. So if
everything else stays the same and you increase framesPerBuffer, the output
latency would increase. What I noticed is that increasing
outputParameters.suggestedLatency made for a noticable increase in output
latency, but the reported framesPerBuffer and sample rate didn't change.
Does that mean the # of buffers changed, or is the increased latency due to
the driver, or PortAudio, simply holding onto the data for longer before
sending it out?

It says in the docs for Pa_OpenStream I should ideally set framePerBuffer to
paFramesPerBufferUnspecified, and this allows PA to set the optimal
framesPerBuffer for the latency requsted in PaStreamParameters. So
generally, should I just set suggstedLatency to either of the default high
or low latencies for the device, and use paFramesPerBufferUnspecified to get
the optimal buffer length? And if I need to let the user adjust latency,
then let them adjust it in msec, and let the system figure out the best
buffer length by reopening the device with the user's desired latency and
passing paFramesPerBufferUnspecified again. Is that correct?

As for getting the actual reported latency to show to the user or for my own
needs: PaStreamInfo says its latency values can be different than what was
requested in Pa_OpenStream(). Presumably the value is set to what the driver
actual reports (or is deduced by PA) after stream creation. Does this happen
only once after stream creation, or can it change anytime thereafter? If the
latter, then how do I get that information at callback time since
PaStreamInfo isn't passed into the callback, and I'm not supposed to call
any PA calls other than cpuLoad during the callback?

Is PaStreamInfo::outputLatency ==
(PaStreamCallbackTimeInfo::outputBufferDacTime -
PaStreamCallbackTimeInfo::currentTime)? Or at least ideally? I know you said
inputBufferAdcTime is sometimes calculated similarly.

As you may have guessed, I need to track actual latencies as closely as
possible during realtime processing. This must be a common need, for example
when playing a reference audio track and recording another track, the
various latencies must be coordinated.

> If you havn't looked at it already you might get something out of the
> following paper:
> PortAudio and Media Synchronisation
> http://www.portaudio.com/docs/portaudio_sync_acmc2003.pdf
>

Thanks, this is very interesting.

Cheers,
Michael

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[Ross Bencina-3]

Hi Michael

> As you may have guessed, I need to track actual latencies as closely as
> possible during realtime processing. This must be a common need, for
> example
> when playing a reference audio track and recording another track, the
> various latencies must be coordinated.

I think this is a common need, but is it also a common problem area which
not many host APIs do not address well. There are possibly other people here
(from Audacity say) who can comment more on your specific use-case. My own
experience is with synchronising audio to on-screen display, and with
external timecode (both chasing and generating) -- both of which are a
little bit different from synchronising playback and recording.

> Yes, by "buffer length" I mean framesPerBuffer, as passed into
> PaStreamCallback. From what I've read (including your article on Media
> Synchronization, thanks), I thought the output latency in msec would
> depend
> on the number of buffers, the length of the buffers, and sample rate. So
> if
> everything else stays the same and you increase framesPerBuffer, the
> output
> latency would increase.

framesPerBuffer is the size of the buffers passed to the callback. this may
or may not have anything to do with the size of the buffers used to talk to
the host API -- in terms of the semantics of the API, framesPerBuffer and
the actual latency are completely decoupled -- in practice there is some
relationship, but as you have observed it is not predictable. The general
idea is that framesPerBuffer is mainly relevant to the algorithms you're
implementing, and the latency is something which needs to be tweaked by the
user (ie depenent on many external factors).

> Does that mean the # of buffers changed, or is the increased latency due
> to
> the driver, or PortAudio, simply holding onto the data for longer before
> sending it out?

It could be any of the above, but in general PA will send audio out as soon
as it can.

> It says in the docs for Pa_OpenStream I should ideally set framePerBuffer
> to
> paFramesPerBufferUnspecified, and this allows PA to set the optimal
> framesPerBuffer for the latency requsted in PaStreamParameters. So
> generally, should I just set suggstedLatency to either of the default high
> or low latencies for the device, and use paFramesPerBufferUnspecified to
> get
> the optimal buffer length? And if I need to let the user adjust latency,
> then let them adjust it in msec, and let the system figure out the best
> buffer length by reopening the device with the user's desired latency and
> passing paFramesPerBufferUnspecified again. Is that correct?

I think that is the best way to do it portably yes. There are host-api
specific settings which give finer grained controll (see the host-api
specific headers).

> As for getting the actual reported latency to show to the user or for my
> own
> needs: PaStreamInfo says its latency values can be different than what was
> requested in Pa_OpenStream(). Presumably the value is set to what the
> driver
> actual reports (or is deduced by PA) after stream creation. Does this
> happen
> only once after stream creation, or can it change anytime thereafter?

Yes the actual latency is based on the driver and on the buffer sizes
PortAudio chooses. It should not change after the stream is created.

> Is PaStreamInfo::outputLatency ==
> (PaStreamCallbackTimeInfo::outputBufferDacTime -
> PaStreamCallbackTimeInfo::currentTime)? Or at least ideally? I know you
> said
> inputBufferAdcTime is sometimes calculated similarly.

This is poorly documented isn't it.

There's some info here:
http://www.portaudio.com/docs/proposals/015-ImproveCallbackTimestampInfo.html
http://www.portaudio.com/docs/proposals/003-ImproveLatencySpecificationInterface.html

The second link says latency can be calculated as (oneBufferDuration *
(numberOfBuffers-1)). This implies that the interpretation of  latency is
the difference in time from when the buffer is returned to the system and
the time at which it plays out the DAC eg:

PaStreamInfo::outputLatency ==
(PaStreamCallbackTimeInfo::outputBufferDacTime -
(PaStreamCallbackTimeInfo::currentTime + bufferDuration))

where PaStreamCallbackTimeInfo::currentTime + bufferDuration is basically
the deadline for when the system needs the buffer to avoid underruns.

I'm not sure this is the best definition of latency but it is as solid as
the spec is right now. Can anyone else offer an opinion?

Cheers

Ross.


----- Original Message -----
From: "Michael Stauffer" <mstauff@...>
To: "'Ross Bencina'" <rossb-lists@...>
Cc: "PortAudio_List_mstauff@ver" <portaudio@...>
Sent: Friday, June 27, 2008 2:56 PM
Subject: Re: [Portaudio] Time & Latency questions

> Hi Ross,
>
> Thanks very much for your detailed reply. It's all very good to know. The
> variability between drivers gives me angst of course, but that seems
> unavoidable! A couple further questions below...
>
> <snip>
>
>> > In patest_latency.c, I was messing with
>> > outputParameters.suggestedLatency,
>> > and it's clear to hear how that effects the playback
>> > latency when playing the ASCII keyboard. But I noticed that as I
>> > changed
> the
>> > suggestedLatency, the buffer length (and sample rate) stayed the same,
> which
>> > means the latency in this case is not buffer-length dependent, and the
> driver
>> > (Windows MME/DX) is adding an artificial latency because of the
> suggestedLatency request.
>> > Is that correct? I otherwise I would have figured the output latency
>> > was
> dependent
>> > on the buffer length and number of buffers in use.
>
>>
>> I'm not sure what you mean by "buffer length" here. If you mean the
>> framesPerBuffer parameter -- this is a user defined buffer
>> size which will
>> only vary if you change the framesPerBuffer value explicitly (or use
>> paFramesPerBufferUnspecified). PA uses various heuristics to
>> select the best
>> combination of host buffer sizes and buffer counts depending
>> on the host
>> API.
>
> Yes, by "buffer length" I mean framesPerBuffer, as passed into
> PaStreamCallback. From what I've read (including your article on Media
> Synchronization, thanks), I thought the output latency in msec would
> depend
> on the number of buffers, the length of the buffers, and sample rate. So
> if
> everything else stays the same and you increase framesPerBuffer, the
> output
> latency would increase. What I noticed is that increasing
> outputParameters.suggestedLatency made for a noticable increase in output
> latency, but the reported framesPerBuffer and sample rate didn't change.
> Does that mean the # of buffers changed, or is the increased latency due
> to
> the driver, or PortAudio, simply holding onto the data for longer before
> sending it out?
>
> It says in the docs for Pa_OpenStream I should ideally set framePerBuffer
> to
> paFramesPerBufferUnspecified, and this allows PA to set the optimal
> framesPerBuffer for the latency requsted in PaStreamParameters. So
> generally, should I just set suggstedLatency to either of the default high
> or low latencies for the device, and use paFramesPerBufferUnspecified to
> get
> the optimal buffer length? And if I need to let the user adjust latency,
> then let them adjust it in msec, and let the system figure out the best
> buffer length by reopening the device with the user's desired latency and
> passing paFramesPerBufferUnspecified again. Is that correct?
>
> As for getting the actual reported latency to show to the user or for my
> own
> needs: PaStreamInfo says its latency values can be different than what was
> requested in Pa_OpenStream(). Presumably the value is set to what the
> driver
> actual reports (or is deduced by PA) after stream creation. Does this
> happen
> only once after stream creation, or can it change anytime thereafter? If
> the
> latter, then how do I get that information at callback time since
> PaStreamInfo isn't passed into the callback, and I'm not supposed to call
> any PA calls other than cpuLoad during the callback?
>
> Is PaStreamInfo::outputLatency ==
> (PaStreamCallbackTimeInfo::outputBufferDacTime -
> PaStreamCallbackTimeInfo::currentTime)? Or at least ideally? I know you
> said
> inputBufferAdcTime is sometimes calculated similarly.
>
> As you may have guessed, I need to track actual latencies as closely as
> possible during realtime processing. This must be a common need, for
> example
> when playing a reference audio track and recording another track, the
> various latencies must be coordinated.
>
>> If you havn't looked at it already you might get something out of the
>> following paper:
>> PortAudio and Media Synchronisation
>> http://www.portaudio.com/docs/portaudio_sync_acmc2003.pdf
>>
>
> Thanks, this is very interesting.
>
> Cheers,
> Michael
>
> _______________________________________________
> Portaudio mailing list
> Portaudio@...
> http://techweb.rfa.org/mailman/listinfo/portaudio
>

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[mgstauffer]

Hi Ross,

Thanks again for such a detailed response, it's very helpful.

So I figure I'll start with setting up devices with their default low
latencies and pass paFramesPerBufferUnspecified to let it set its buffer
size. I'll see how well it works across different devices and see if I need
to dive into driver-specific calls.

Generally speaking, would you think this "defaults" approach should yield
accurate-enough timing to sync with MIDI I/O that's run from a separate
hi-res timer, or will it just be driver-specific again in the end? I'll time
the MIDI I/O relative to the reported audio latencies, and I can watch the
two different clocks to account for drift.

Cheers,
Michael

> -----Original Message-----
> From: Ross Bencina [mailto:rossb-lists@...]
> Sent: Friday, June 27, 2008 2:56 AM
> To: Michael Stauffer
> Cc: PortAudio_List_mstauff@ver
> Subject: Re: [Portaudio] Time & Latency questions
>
>
> Hi Michael
>
> > As you may have guessed, I need to track actual latencies
> as closely
> > as possible during realtime processing. This must be a common need,
> > for example when playing a reference audio track and
> recording another
> > track, the various latencies must be coordinated.
>
> I think this is a common need, but is it also a common
> problem area which
> not many host APIs do not address well. There are possibly
> other people here
> (from Audacity say) who can comment more on your specific
> use-case. My own
> experience is with synchronising audio to on-screen display, and with
> external timecode (both chasing and generating) -- both of
> which are a
> little bit different from synchronising playback and recording.
>
> > Yes, by "buffer length" I mean framesPerBuffer, as passed into
> > PaStreamCallback. From what I've read (including your
> article on Media
> > Synchronization, thanks), I thought the output latency in
> msec would
> > depend on the number of buffers, the length of the buffers,
> and sample
> > rate. So if
> > everything else stays the same and you increase
> framesPerBuffer, the
> > output
> > latency would increase.
>
> framesPerBuffer is the size of the buffers passed to the
> callback. this may
> or may not have anything to do with the size of the buffers
> used to talk to
> the host API -- in terms of the semantics of the API,
> framesPerBuffer and
> the actual latency are completely decoupled -- in practice
> there is some
> relationship, but as you have observed it is not predictable.
> The general
> idea is that framesPerBuffer is mainly relevant to the
> algorithms you're
> implementing, and the latency is something which needs to be
> tweaked by the
> user (ie depenent on many external factors).
>
> > Does that mean the # of buffers changed, or is the
> increased latency
> > due
> > to
> > the driver, or PortAudio, simply holding onto the data for
> longer before
> > sending it out?
>
> It could be any of the above, but in general PA will send
> audio out as soon
> as it can.
>
> > It says in the docs for Pa_OpenStream I should ideally set
> > framePerBuffer
> > to
> > paFramesPerBufferUnspecified, and this allows PA to set the optimal
> > framesPerBuffer for the latency requsted in PaStreamParameters. So
> > generally, should I just set suggstedLatency to either of
> the default high
> > or low latencies for the device, and use
> paFramesPerBufferUnspecified to
> > get
> > the optimal buffer length? And if I need to let the user
> adjust latency,
> > then let them adjust it in msec, and let the system figure
> out the best
> > buffer length by reopening the device with the user's
> desired latency and
> > passing paFramesPerBufferUnspecified again. Is that correct?
>
> I think that is the best way to do it portably yes. There are
> host-api
> specific settings which give finer grained controll (see the host-api
> specific headers).
>
> > As for getting the actual reported latency to show to the
> user or for
> > my
> > own
> > needs: PaStreamInfo says its latency values can be
> different than what was
> > requested in Pa_OpenStream(). Presumably the value is set
> to what the
> > driver
> > actual reports (or is deduced by PA) after stream creation.
> Does this
> > happen
> > only once after stream creation, or can it change anytime
> thereafter?
>
> Yes the actual latency is based on the driver and on the buffer sizes
> PortAudio chooses. It should not change after the stream is created.
>
> > Is PaStreamInfo::outputLatency ==
> > (PaStreamCallbackTimeInfo::outputBufferDacTime -
> > PaStreamCallbackTimeInfo::currentTime)? Or at least ideally? I know
> > you said inputBufferAdcTime is sometimes calculated similarly.
>
> This is poorly documented isn't it.
>
> There's some info here:
> http://www.portaudio.com/docs/proposals/015-ImproveCallbackTim
> estampInfo.html
> http://www.portaudio.com/docs/proposals/003-ImproveLatencySpec
> ificationInterface.html
>
> The second link says latency can be calculated as
> (oneBufferDuration *
> (numberOfBuffers-1)). This implies that the interpretation of
>  latency is
> the difference in time from when the buffer is returned to
> the system and
> the time at which it plays out the DAC eg:
>
> PaStreamInfo::outputLatency ==
> (PaStreamCallbackTimeInfo::outputBufferDacTime -
> (PaStreamCallbackTimeInfo::currentTime + bufferDuration))
>
> where PaStreamCallbackTimeInfo::currentTime + bufferDuration
> is basically
> the deadline for when the system needs the buffer to avoid underruns.
>
> I'm not sure this is the best definition of latency but it is
> as solid as
> the spec is right now. Can anyone else offer an opinion?
>
> Cheers
>
> Ross.
>

<snip>

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[Ross Bencina-3]

Hi Michael

>>>
Generally speaking, would you think this "defaults" approach should yield
accurate-enough timing to sync with MIDI I/O that's run from a separate
hi-res timer, or will it just be driver-specific again in the end? I'll time
the MIDI I/O relative to the reported audio latencies, and I can watch the
two different clocks to account for drift.
<<<

It will probably be driver specific, but it kind of depends what you're
doing with your MIDI -- MIDI timing accuracy/jitter is ~1ms anyway which is
relatively large. If you can code something to deal with the clock drift you
should be fine.

Best wishes

Ross.


----- Original Message -----
From: "Michael Stauffer" <mstauff@...>
To: "'Ross Bencina'" <rossb-lists@...>
Cc: "'PortAudio_List_mstauff@ver'" <portaudio@...>
Sent: Friday, July 04, 2008 10:13 AM
Subject: RE: [Portaudio] Time & Latency questions


Hi Ross,

Thanks again for such a detailed response, it's very helpful.

So I figure I'll start with setting up devices with their default low
latencies and pass paFramesPerBufferUnspecified to let it set its buffer
size. I'll see how well it works across different devices and see if I need
to dive into driver-specific calls.

Generally speaking, would you think this "defaults" approach should yield
accurate-enough timing to sync with MIDI I/O that's run from a separate
hi-res timer, or will it just be driver-specific again in the end? I'll time
the MIDI I/O relative to the reported audio latencies, and I can watch the
two different clocks to account for drift.

Cheers,
Michael

> -----Original Message-----
> From: Ross Bencina [mailto:rossb-lists@...]
> Sent: Friday, June 27, 2008 2:56 AM
> To: Michael Stauffer
> Cc: PortAudio_List_mstauff@ver
> Subject: Re: [Portaudio] Time & Latency questions
>
>
> Hi Michael
>
> > As you may have guessed, I need to track actual latencies
> as closely
> > as possible during realtime processing. This must be a common need,
> > for example when playing a reference audio track and
> recording another
> > track, the various latencies must be coordinated.
>
> I think this is a common need, but is it also a common
> problem area which
> not many host APIs do not address well. There are possibly
> other people here
> (from Audacity say) who can comment more on your specific
> use-case. My own
> experience is with synchronising audio to on-screen display, and with
> external timecode (both chasing and generating) -- both of
> which are a
> little bit different from synchronising playback and recording.
>
> > Yes, by "buffer length" I mean framesPerBuffer, as passed into
> > PaStreamCallback. From what I've read (including your
> article on Media
> > Synchronization, thanks), I thought the output latency in
> msec would
> > depend on the number of buffers, the length of the buffers,
> and sample
> > rate. So if
> > everything else stays the same and you increase
> framesPerBuffer, the
> > output
> > latency would increase.
>
> framesPerBuffer is the size of the buffers passed to the
> callback. this may
> or may not have anything to do with the size of the buffers
> used to talk to
> the host API -- in terms of the semantics of the API,
> framesPerBuffer and
> the actual latency are completely decoupled -- in practice
> there is some
> relationship, but as you have observed it is not predictable.
> The general
> idea is that framesPerBuffer is mainly relevant to the
> algorithms you're
> implementing, and the latency is something which needs to be
> tweaked by the
> user (ie depenent on many external factors).
>
> > Does that mean the # of buffers changed, or is the
> increased latency
> > due
> > to
> > the driver, or PortAudio, simply holding onto the data for
> longer before
> > sending it out?
>
> It could be any of the above, but in general PA will send
> audio out as soon
> as it can.
>
> > It says in the docs for Pa_OpenStream I should ideally set
> > framePerBuffer
> > to
> > paFramesPerBufferUnspecified, and this allows PA to set the optimal
> > framesPerBuffer for the latency requsted in PaStreamParameters. So
> > generally, should I just set suggstedLatency to either of
> the default high
> > or low latencies for the device, and use
> paFramesPerBufferUnspecified to
> > get
> > the optimal buffer length? And if I need to let the user
> adjust latency,
> > then let them adjust it in msec, and let the system figure
> out the best
> > buffer length by reopening the device with the user's
> desired latency and
> > passing paFramesPerBufferUnspecified again. Is that correct?
>
> I think that is the best way to do it portably yes. There are
> host-api
> specific settings which give finer grained controll (see the host-api
> specific headers).
>
> > As for getting the actual reported latency to show to the
> user or for
> > my
> > own
> > needs: PaStreamInfo says its latency values can be
> different than what was
> > requested in Pa_OpenStream(). Presumably the value is set
> to what the
> > driver
> > actual reports (or is deduced by PA) after stream creation.
> Does this
> > happen
> > only once after stream creation, or can it change anytime
> thereafter?
>
> Yes the actual latency is based on the driver and on the buffer sizes
> PortAudio chooses. It should not change after the stream is created.
>
> > Is PaStreamInfo::outputLatency ==
> > (PaStreamCallbackTimeInfo::outputBufferDacTime -
> > PaStreamCallbackTimeInfo::currentTime)? Or at least ideally? I know
> > you said inputBufferAdcTime is sometimes calculated similarly.
>
> This is poorly documented isn't it.
>
> There's some info here:
> http://www.portaudio.com/docs/proposals/015-ImproveCallbackTim
> estampInfo.html
> http://www.portaudio.com/docs/proposals/003-ImproveLatencySpec
> ificationInterface.html
>
> The second link says latency can be calculated as
> (oneBufferDuration *
> (numberOfBuffers-1)). This implies that the interpretation of
>  latency is
> the difference in time from when the buffer is returned to
> the system and
> the time at which it plays out the DAC eg:
>
> PaStreamInfo::outputLatency ==
> (PaStreamCallbackTimeInfo::outputBufferDacTime -
> (PaStreamCallbackTimeInfo::currentTime + bufferDuration))
>
> where PaStreamCallbackTimeInfo::currentTime + bufferDuration
> is basically
> the deadline for when the system needs the buffer to avoid underruns.
>
> I'm not sure this is the best definition of latency but it is
> as solid as
> the spec is right now. Can anyone else offer an opinion?
>
> Cheers
>
> Ross.
>

<snip>


_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio

[mgstauffer]

> >>>
> Generally speaking, would you think this "defaults" approach
> should yield accurate-enough timing to sync with MIDI I/O
> that's run from a separate hi-res timer, or will it just be
> driver-specific again in the end? I'll time the MIDI I/O
> relative to the reported audio latencies, and I can watch the
> two different clocks to account for drift. <<<
>
> It will probably be driver specific, but it kind of depends
> what you're
> doing with your MIDI -- MIDI timing accuracy/jitter is ~1ms
> anyway which is
> relatively large. If you can code something to deal with the
> clock drift you
> should be fine.
>
> Best wishes
>
> Ross.

I'm comfortable with the ~1ms accuracy of MIDI for my project. I'll forge
ahead and test things along the way. Thanks again, and wish me luck! :)

Cheers,
Michael

_______________________________________________
Portaudio mailing list
Portaudio@...
http://techweb.rfa.org/mailman/listinfo/portaudio