From: philburk Updated: January 7, 2002 Updated: January 11, 2002 This document describes modifications to the PortAudio API currently being considered by the developer community. It is intended to capture the current state of discussions. The authors are the various members of the PortAudio community and are too numerous to mention. Please refer to the mailing list archives to see who said what. We are still at the design stage with all of these proposals - if you think something is missing, could be improved, or would just like to comment please do so on the PortAudio mailing list. http://techweb.rfa.org/mailman/listinfo/portaudio The proposals above which are marked with a * are under construction, those marked with |C| are essentially complete but require approval or comment before they are considered complete. The remaining proposals are essentially complete and will be implemented as-is unless significant objections are raised. ____________ The proposals above which are marked with a * are under construction, those marked with |C| are essentially complete but require community approval or comment before they are considered complete. The remaining proposals are essentially complete and will be implemented as-is unless significant objections are raised. This document describes proposals that range in complexity from clarifications of existing API functionality, through to renaming API functions for consistency, through to significant feature additions. Due to the interdependency of many of the proposed changes, we plan to introduce all API changes simultaneously as part of release 19. The developer community is in the process of reviewing all proposals. Most proposals are defined in sufficient detail to implement, however some completed proposals are dependent on proposals that are still under construction. Of the remaining unfinished proposals, only Improve Device Format Query System, Improve Latency Mechanisms, Allow Callbacks to Accept Variable framesPerBuffer, and Handling of Host-Specific Error Codes require significant attention. Both the Coding Style Guidelines and Revise Internal Host API proposals only effect implementors and do not effect the API definition itself. Refinement of these proposals is desirable but not critical to the development of release 19. Interested readers are advised to consult the Dependencies section of each proposal for more detailed information. Some of the proposed changes will break existing client code. Usually these breakages involve renaming of constant identifiers or function calls. In a small number of cases additional parameters will be added to existing API functions. Concerned readers are advised to consult the Impact Analysis section of each proposal for further details. ____________ This proposal is sufficiently well defined to be implemented It has been noted that the current method (Pa_GetDeviceInfo()) of querying devices for supported sample formats, channels and sample rates is weak. It does not cleanly differentiate between 'PortAudio supported' formats and 'native' formats, and it is incapable of representing formats where the parameters are interdependent (eg where full duplex is only supported for certain sample rates.) We have also found that a static structure is not a good match for many driver models where format discovery is performed by polling the driver. Even if a sound card supports arbitrary sample rates, the driver model may only allow a client to poll to see whether a rate is available rather than providing the available rate ranges. It has been noted that most (platform specific) audio APIs do a pretty bad job of allowing for device capability querying. Even the better APIs (ALSA is perhaps one) don't necessarily provide accurate information. This proposal should seek to maximise the amount of information that can be extracted from existing APIs while remaining expressive enough to take full advantage of APIs with more advanced capability querying systems should they become available in the future. Use PaDeviceInfo only for representing information that can be expressed without polling the driver multiple times, or only check for "standard" formats, or leave it unchanged, or do away with it completely. And/Or We could do things the way MME does: if Pa_OpenStream() is called with a NULL stream parameter then the stream isn't opened, but it is checked to see if the device supports the specified format - if the format is supported then paNoError would be returned, otherwise an error would be returned. A number of options are being considered with regard to supplying clients with format information: Use PaDeviceInfo only for representing information that can be expressed without polling the driver multiple times, or only check for "standard" formats, or leave it unchanged. And/Or Add an IsFormatSupported() function: Add a Pa_IsFormatSupported() function: At present it seems desirable to retain Pa_GetDeviceInfo() and the PaDeviceInfo structure. At a minimum PaDeviceInfo needs to contain name and driver model type code fields: It was suggested that we could do things the way MME does: if Pa_OpenStream() is called with a NULL stream parameter then the stream isn't opened, but it is checked to see if the device supports the specified format - if the format is supported then paNoError would be returned, otherwise an error code would be returned. However, this has been ruled to be unsatisfactory, since querying for supported formats is really a different function from opening a stream. This proposal will provide clients with more expressive methods for querying device capabilities, which should improve the utility of PortAudio. It is not yet clear what the full impact of this proposal will be. ____________ See this thread: http://techweb.rfa.org/pipermail/portaudio/2001-October/000196.html The numBuffers parameter for Pa_OpenStream() could accept the following values in addition to the normal N>0 values. Use PA_LATENCY_LOW for interactive performance, when response time is critical. Use PA_LATENCY_HIGH when playing sound files or other non-interactive uses, when glitch free operation is more critical. #define PA_LATENCY_WHATEVER (0) /* or _NO_OPINION or? */ Currently, when numBuffers>0, Pa_OpenStream will constrain the actual numBuffers so that the latency is within a valid range determined by the driver, or an environment variables such as PA_MIN_LATENCY_MSEC. Propose changing the behaviour so that the requested value is honoured as much as possible. This will allow the user to override the minimum if they know their system can handle it. This might be used, for example on patched Linux kernels. By doing this we simplify the implementations and eliminate the need for adding a Pa_SetMinimumLatency() function. /* Negative return values are (double) paError */ /* This would only be valid after Pa_OpenStream() */ This proposal does not yet address the question of whether there needs to be separate numInputBuffers and numOutputBuffers parameters for optimal latency specification. ____________ This proposal is waiting to be written up (assigned to Roger Dannenberg.) This proposal is under construction. Background ____________ This proposal is open for comment. Many PortAudio users have requested a blocking read/write API which will be supported in addition to the current callback based API. A blocking Read/Write API would allow a more natural style of multi-threaded programming, and facilitate single-threaded reactive applications, while insulating clients from platform-specific thread synchronisation facilities. A blocking read/write API would also be useful when binding PortAudio to languages that don't easily support callbacks such as Python, Java, Lisp and Smalltalk. However in this case it has been noted that a blocking API is not sufficient - the host language also needs to support native threads to interact efficiently with blocking. Many PortAudio users have requested a blocking read()/write() API that will be supported in addition to the current callback based API. A blocking Read/Write API would allow a more natural style of multi-threaded programming, and facilitate single-threaded reactive applications, while insulating clients from platform-specific thread synchronisation facilities. A blocking read/write API would also be useful when binding PortAudio to languages that don't easily support callbacks such as Python, Java, Lisp and Smalltalk. However in this case it has been noted that a blocking API is not sufficient - the host language also needs to support native threads to interact efficiently with blocking. Dannenberg observes that native thread support cannot be added without major redesign (based on a study of Python and Squeak), but given blocking calls, there are several ways to structure programs using non-native threads. Adding a blocking Read/Write interface to PortAudio has been discussed on a number of occasions, including the following threads: http://techweb.rfa.org/pipermail/portaudio/2001-August/000063.html a long thread about blocking calls. http://techweb.rfa.org/pipermail/portaudio/2001-August/000137.html this is Roger Dannenberg's proposal and a subsequent discussion http://techweb.rfa.org/pipermail/portaudio/2001-August/000144.html is a thread discussing using blocking APIs with other languages This proposal needs further documentation, but some details are provided below. If a NULL callback parameter is passed to Pa_OpenStream() then the stream is opened in blocking mode. This enables users to call Pa_WriteStream() and Pa_ReadStream() to read and write sample data. For a while there was a talk about only requiring implementations to implement blocking calls and that callbacks would always be implemented on top of blocking. The current direction is that this decision will be made independently for each driver-model. There needs to be a way to call the read and write functions without them blocking or to determine in advance how many samples can be read or written without blocking. One proposal to avoid blocking in environments without native threads is to ask PortAudio how many frames are available, and then to sleep() (using non-native thread management) until the next buffer boundary. Roger Dannenberg wrote the following email that is included here because it didn't seem to make it into the mailing list archive: If a NULL callback parameter is passed to Pa_OpenStream() then the stream will be opened in blocking mode. This enables users to call Pa_WriteStream() and Pa_ReadStream() to read and write sample data. (The PaErrorNum item "paNullCallback" becomes obsolete.) Pa_WriteStream() writes a buffer of frames to a stream. The length of the buffer is arbitrary and specified by the frames parameter. Pa_WriteStream() returns when all samples have been copied from buffer. If necessary, Pa_WriteStream() will wait until buffer space becomes available. (Waiting on Unix will be the by-product of an I/O system call, waiting in Win32 will be implemented by waiting on an Event object, and waiting on MacOS 9 will probably require a busy wait.) High performance applications will want to match the length of the buffer to framesPerBuffer, but this is not a requirement. The buffer parameter has the same semantics and format as the inputbuffer and outputbuffer parameters of a PortAudioCallback function. In particular, non-interleaved data is handled in the same way. Pa_ReadStream() is similar, but it reads rather than writes. Pa_ReadStream() returns paInputUnderflow if input data was discarded by PortAudio after the previous call and before this call. Pa_WriteStream() returns paOutputUnderflow if output data was inserted after the previous call and before this call. The mode flag paNeverDropInput is ignored because Pa_ReadStream() and Pa_WriteStream() are not synchronized. There are two functions to determine the number of frames available for writing and reading. These functions may be called to determine whether calls to Pa_WriteStream() or Pa_ReadStream() will return immediately or will wait. The return value, if non-negative, is the maximum number of frames that can be written or read without blocking or busy waiting. A negative value is a PaErrorNum. The stream functions Pa_CloseStream(), Pa_StartStream(), Pa_StopStream(), Pa_AbortStream(), Pa_StreamActive(), and Pa_StreamTime() work with the blocking API as well as with callbacks. Pa_StreamCPULoad() does not work with the blocking API. PortAudio might be extended to give applications access to the internal routines that compute Pa_StreamCPULoad(). Applications using blocking calls could then bracket audio computation with these calls to determine the CPU load. (This additional functionality is not being proposed here.) A rejected alternative is to allow Pa_WriteStream() and Pa_ReadStream() to return the number of frames actually written or read so that a Mac implementation could return immediately and avoid blocking. This would require applications to be prepared to handle partial read/writes. It seems simpler and more consistent to use "Available" to determine in advance whether blocking or busy waiting will occur if that is a concern. Also, note that data is almost certainly copied; however, it seems likely that the copy will be folded into any format conversion. Implementations may want to provide a way for applications to be notified when data can be written or read. For example, one might want to know the file ID of an ALSA or OSS stream for use in a select() system call. Since this sort of information will be platform-specific and non-portable, no interface is defined here, but implementations can include a device-model-specific access function. If applications commonly need this information, we can think about how to make this more standardized. Implementing blocking i/o will be quite simple for driver models where the underlying API is blocking-based, and quite difficult to do well under Windows where much fun will be had with Win32 synchronisation primitives. Implementing blocking i/o will be quite simple for driver models where the underlying API is blocking-based. Under Windows (MME), the arrival of a buffer will signal an Event passed to waveOutOpen(). Pa_WriteStream() and Pa_ReadStream() will do all the work (no server threads necessary). Writes will make waveOutWrite calls. When no buffer is available, the writer will wait on the event and try again. Reading is similar. On the Mac, a double-buffer scheme can be set up where the Mac callbacks pick up data placed in buffers by Pa_WriteStream(). The double-buffer adds to the latency. Alternatively (and preferably) callbacks can be used only for notification, and Pa_WriteStream() can issue all the calls to write samples. PABLIO currently contains a busy-wait ring buffer in "ringbuffer.c" which is generic, used in many projects and is pretty solid. This code could be a useful starting point for implementing the new blocking API on some platforms. Pa_WriteStream() and Pa_ReadStream() are not thread safe. Applications wanting to call these from multiple threads should manage their own mutual exclusion. [Roger: Is any of PortAudio thread safe? I don't think so. This is good because it avoids many system calls for mutual exclusion.] At one time it was suggested that implementations only implement blocking calls and that callbacks would be required to implement callbacks in terms of the blocking API. The current direction is that this decision should be made independently for each driver-model. This proposal would extend the functionality of PortAudio without requiring any changes to client code. This proposal would extend the functionality of PortAudio without requiring any changes to client code with the exception that the PaErrorNum paNullCallback will no longer be defined. As noted above, implementation complexity is dependent on the target platform. ____________ The current PortAudio/ASIO implementation works as follows : ASIO native buffers are non-interleaved and the de-interleaving, format conversion and copying the data into PortAudio interleaved buffers is done in one loop. But if PortAudio supported non-interleaved buffers then we could use efficient vector operations even for native buffer <==> port audio buffers transfers. A new sample format could be defined: This could be used as a modifier flag to the buffer format fields of Pa_OpenStream(). When present, this flag would indicate that non-interleaved buffers would be passed to the callback. When not present, interleaved buffers would be used as is currently always the case. For example, the following code would open an interleaved stream: The implementation will follow the methodology currently employed in PortMIDI described here: http://techweb.rfa.org/pipermail/portaudio/2001-December/000295.html An implementation of the driver model neutral "Façade" of this proposal exists here: http://www.portaudio.com/docs/pa_drivermodel.c.txt http://www.portaudio.com/docs/pa_drivermodel.h.txt This proposal will involve the changes described below. Note that the string <DM> will be replaced with a driver model tag for each implementation. Each driver model will have it's own Initialize function which PortAudio will call in response to client calls to Pa_Initialize and Pa_Terminate respectively. This will be the only identifier each driver model implementation will be required to expose.Introduction
-Status
+Client Impact
+PortAudioCallback Underflow/Overflow Handling
Status
Proposal
-Pa_IsFormatSupported( deviceId, inputDevice, numInputChannels, inputSampleFormat, inputDriverInfo, outputDevice, numOutputChannels, outputSampleFormat, outputDriverInfo, sampleRate );
+Pa_IsFormatSupported( PaDeviceID inputDevice,
+ int numInputChannels,
+ PaSampleFormat inputSampleFormat,
+ void *inputDriverInfo,
+ PaDeviceID outputDevice,
+ int numOutputChannels,
+ PaSampleFormat outputSampleFormat,
+ void *outputDriverInfo,
+ double sampleRate );
+Discussion
+typedef struct{
+ int structVersion;
+ const char *name;
+ PaDriverModelTypeCode driverModel;
+} PaDeviceInfo;
+Impact Analysis
Proposal
-#define PA_LATENCY_LOW (-1) /* For interactive performance. */
-#define PA_LATENCY_HIGH (-2) /* For playing sound files. */
+#define PA_LATENCY_WHATEVER (0) /* or _NO_OPINION or? */
+#define PA_LATENCY_LOW (-1) /* For interactive performance. */
+#define PA_LATENCY_HIGH (-2) /* For playing sound files. */
-double Pa_GetMinimumLatency( int deviceId, double frameRate );
-double Pa_GetPreferredLatency( int deviceId, double frameRate );
-double Pa_GetMaximumLatency( int deviceId, double frameRate );
-double Pa_GetActualLatency( int deviceId, double frameRate );
+/* Negative return values are (double) paError */
+double Pa_GetMinimumLatency( int deviceId, double frameRate );
+double Pa_GetPreferredLatency( int deviceId, double frameRate );
+double Pa_GetMaximumLatency( int deviceId, double frameRate );
+/* the following would operate directly on streams */
+double Pa_StreamInputLatency( PortAudioStream *stream );
+double Pa_StreamOutputLatency(PortAudioStream *stream );
Discussion
Blocking Read/Write API
+Allow Callbacks to Accept Variable framesPerBuffer
Status
-
+
+Some devices prefer, or even require, a particular value for framesPerBuffer. Some applications also prefer a particular buffer size because they may, for example, be doing FFTs. If possible, an integral number of application buffers can be fit into a device buffer. If an integral number of application buffers cannot be fit into an application buffer then some system for adapting between them is required. Stephane Letz implemented a block adapter for the ASIO driver that could perhaps be made part of a utility library for PortAudio.
+
+Adapting between different block size, unfortunately, involves extra memory movement and should be avoided. One way to avoid this is to allow an application to say "I can handle any value for framesPerBuffer". An example would be a simple algorithm like the one in "pa_tests/pa_fuzz.c". Its callback function could easily work with any value for FramesPerBuffer.
+
+Proposal
+
+Pa_OpenStream() should be able to accept a zero value for framesPerBuffer. When framesPerBuffer is zero, the implementation can choose any value that it thinks will work best given the other constraints. That value will be passed to the user callback.
+
+To make the code more obvious, this constant could be used:
+
+#define paFramesPerBufferUnspecified (0)
+
+In order to find out what value the implementation chose, and also to help the application determine the actual latency, the following calls could be added:
+
+int Pa_GetFramesPerBuffer( PortAudioStream* stream );
+int Pa_GetNumOutputBuffers( PortAudioStream* stream );
+int Pa_GetNumInputBuffers( PortAudioStream* stream );
+Blocking Read/Write API
+Status
+Background
-Proposal
-> It would be nice if the buffer could be returned to PortAudio as soon
+
-> as the client had finished with it. For example, in the MME
-> implementation this would allow the buffer to be requeued with
-> the driver sooner.
-
-My model is that, in general, you should target the worst case where audio processing takes most of the CPU time. In that case, the app finishes reading and writing buffers just before it's time to start the next buffer. In that case, there's little to gain by trying to return buffers early. Alternatively, if CPU time is negligible, wouldn't returning the buffer immediately (to MME) effectively increase the buffer count by one? In other words, you could just allocate one extra buffer and not return read buffers until later.
-
-I agree that ideally you should return the buffer when you can, but this adds complexity to the application and is not typical of a blocking read interface. If the call to release the buffer was *optional*, I'd have no objection, but that puts some additional bookkeeping back on the library side.
-
-> Is GetAvailable() the best way to implement non-blocking i/o?
-It works in practice. Some systems including Windows do not implement this function very well, i.e. the reported available space comes in fits and starts. But at least you can find out if an IO call would block or fail. I'd like to avoid the extra system call when possible though.PaError Pa_WriteStream( PortAudioStream *stream,
+ void *buffer,
+ unsigned long frames );
+
+PaError Pa_ReadStream( PortAudioStream *stream,
+ void *buffer,
+ unsigned long frames );
+long Pa_StreamWriteAvailable( PortAudioStream *stream );
+
+long Pa_StreamReadAvailable( PortAudioStream *stream );
+Discussion
+Implementation Notes
-Impact Analysis
-Non-Interleaved Buffers
Status
@@ -158,7 +216,7 @@ It works in practice. Some systems including Windows do not implement this funct
Proposal
#define paNonInterleaved ((PaSampleFormat) (1<<32))
+#define paNonInterleaved ((PaSampleFormat) (1<<31))
@@ -337,8 +395,8 @@ int Pa_GetMinNumBuffers( PaDeviceID id, int framesPerBuffer, double sampleRate )
Implementation Notes
PaError Pa<DM>_MultiDriverInitialize( PaDriverModelImplementation **impl );
@@ -363,6 +421,10 @@ int Pa_GetMinNumBuffers( PaDeviceID id, int framesPerBuffer, double sampleRate )
fptr start;
fptr stop;
fptr abort;
+ fptr read;
+ fptr write;
+ fptr readAvailable;
+ fptr writeAvailable;
fptr active;
fptr time;
fptr cpuLoad;
@@ -431,15 +493,17 @@ WAVERR_BADFORMAT to paSampleFormatNotSupported (already defined)
This proposal improves the quality of PortAudio diagnostics. Client code that depends on paHostError code to flag certain conditions may be effected.
____________
-Proposal is sufficiently well defined to be implemented. No objections have been raised.
PortAudio functions that return global information typically have names of the form Pa_Get*() (eg. Pa_GetDeviceInfo). Almost all functions operating on PortAudio streams conform to the following implicit naming convention: Pa_*Stream( stream ) performs an action on a stream (eg Pa_StartStream() ) and Pa_Stream*( stream ) returns a stream attribute (eg. Pa_StreamTime() ). The only exception to this convention is the confusingly named Pa_GetCPULoad() which actually returns the CPU consumption of a particular stream's callback.
+PortAudio functions and parameter names that operate on integer identifiers use the string ID (all uppercase.) The paInvalidDeviceId error code is an exception to this convention.
Rename PaGetCPULoad() to Pa_StreamCPULoad() in order to conform to established naming conventions.
+Rename PaGetCPULoad() to Pa_StreamCPULoad() in order to conform to the established naming conventions.
+Rename the paInvalidDeviceId error code to paInvalidDeviceID to conform to the established naming conventions.
This proposal improves the consistency of the naming scheme making the API easier to learn and remember. All clients which call Pa_GetCPULoad() will need to alter their code by renaming the function call.
+This proposal improves the consistency of the naming scheme making the API easier to learn and remember. All clients which call Pa_GetCPULoad() will need to alter their code by renaming the function call. Clients who explicitly check for the paInvalidDeviceId error code will have to edit its capitalisation.
____________
It has been suggested that we make an effort to minimise the use of global and static data in PortAudio implementations. Another related goal is to reduce name pollution in the global scope. Some possible guidelines in this regard are:
+It has been suggested that we make an effort to minimise the use of global and static data in PortAudio implementations. Another related goal is to reduce name pollution in the global scope. Some possible guidelines in this regard are:
The conversion functions could look something like:
void ConversionFunction_DestType_DestInterleave_SrcType_SrcInterleave_ ( void *dest, int destChannels, void *src, int srcChannels, int frames );
The dest and src parameters have the same format as those supplied to the PortAudio client callback.
-Rather than have each implementation call these conversion functions directly, a 'factory function' could be implemented that returnes a pointer to a conversion function based on parameters specifying the format of the source and destination buffers. This factory function could be called as needed when a stream is opened. The conversion functions could then be made static and hidden from the rest of PortAudio. The 'factory function' could have the following form:
+Rather than have each implementation call these conversion functions directly, a 'factory function' could be implemented that returns a pointer to a conversion function based on parameters specifying the format of the source and destination buffers. This factory function could be called as needed when a stream is opened. The conversion functions could then be made static and hidden from the rest of PortAudio. The 'factory function' could have the following form:
enum PaEndiannes { paBigEndian, paSmallEndian, paHostEndian };
PaBufferConversionFunction* Pa_GetBufferConversionFunction(
@@ -536,7 +600,7 @@ PaBufferConversionFunction* Pa_GetBufferConversionFunction(
Note that the interleave/deinterleave status is encoded in the destFormat and srcFormat parameters. paHostEndian is used to represent the endianness of the current platform since some driver models (eg ASIO) allow the driver to use samples in a different endianness from the host endianness. Another alternative is to encode sample endianness in PaSampleFormat - this would allow clients to write sample data of either endianness to PortAudio (e.g. soundfile playback direct from file) and benefit from PortAudio's byte swapping code.
The redundant use of channel parameters in both the conversion functions and the factory function is intentional and would allow channel-optimised conversion functions to be supplied for common cases such as 16-bit stereo.
Discussion
-This proposal currently only addresses buffer conversion functions, but we would like to identify other common code fragments which could be placed in the shared PortAudio library.
+This proposal currently only addresses buffer conversion functions, however it is important to identify other common code fragments that could be placed in the shared PortAudio library.
This proposal has not yet addressed the fact that the conversion functions also need to handle clipping and dithering.
It is not clear whether additional conversion functions will be needed to accommodate the MME interleaved stereo pairs for multichannel devices proposal.
It hasn't been established whether PortAudio will be extended to support all PaSampleFormats on all devices.