home changes contents options help - subscribe edit

last edited 1 year ago by gif
about
piksel08
piksel07
piksel06
piksel05
piksel04
piksel03
piksel code
piksel wiki
pikselplenum
mailinglists
home
Creative Commons License

Interoperability

Discussion of various strategies for software interoperability.

Subtopics:

---

the brainstorming is going on between niels, tom, jaromil, artem, kentaro and salsaman,

so far we tried to find and agree on a protocol to describe functionalities and capabilities of our different applications, so that they can talk and operate between them.

we are talking about EffecTV, PD/PDP, FreeJ, LiVES and VeeJay

we currently feel very good about OSC : http://www.cnmat.berkeley.edu/OpenSoundControl/

now we get on the task of defining our controls and fields in OSC

some notes about default calls:

---

effect description

== formats ==

Here is a list of formats that our common plugin host will support.

The host will also be able to do automatic conversion between plugins handling different formats; when this will happen, it will notice the program (and possibly the user) of the slowdown due to the current plugin chain.

|| YUV422 - YUY2 || Data is found in memory as an array of unsigned characters in which the first byte contains the first sample of Y, the second byte contains the first sample of Cb, the third byte contains the second sample of Y, the fourth byte contains the first sample of Cr; and so on. If data is addressed as an array of two little-endian WORD type variables, the first WORD contains Y0 in the least significant bits and Cb in the most significant bits, and the second WORD contains Y1 in the least significant bits and Cr in the most significant bits. || http://www.fourcc.org/yuv.php#YUY2 ||

|| YUV420P - YV12 & I420 || 8 bit Y plane followed by 8 bit 2x2 subsampled V and U planes (YV12) or in a swapped version U and V planes (I420) || http://www.fourcc.org/yuv.php#YV12 ||

|| RGBA 32 || For example, a 32bpp RGBA image would likely use the top 8 bits of each u_int32 to store the alpha component (the unused byte in normal 32bpp RGB). In this case, the masks reported would be: Red=0x00FF0000, Green=0x0000FF00, Blue=0x000000FF, Alpha=0xFF000000, giving 256 levels of blending per pixel (8 bits of alpha data). || http://www.fourcc.org/rgb.php#RGBA ||


comments:

Commands -- Wed, 05 Nov 2003 21:34:26 +0100 reply
I think we need a /commands/ message to get a list of commands supported by the app., e.g. sending /commands to LiVES would return a list like: play,stop,play_selection,record,set_fps,goto_frame,switch_to_clip,load_asset,stop_load,cancel_load,grab_external,set_pingpong,mute on/off.

I also think it is a good idea for each message to return a status, either OK or an error message. For example, LiVES cannot play whilst loading a clip, unless you switch to another clip first. Or you might try to switch to a clip which has been closed by the user. So if you tell LiVES to play, you should check whether you get an OK back, if not you can either wait or take appropriate action. Another thing - one app might be sending frames too fast to another application, so by waiting for an OK you could drop frames and stop the buffer from overflowing (I am thinking more of sending pixel data down a pipe now).

Also, the effects in LiVES are a little different from the effects in other apps. What you do is switch to a clip, then apply an effect to a selection. LiVES will apply the effect to all those frames in non_real time mode. Maybe that is not very useful, so LiVES shouldn't return any effects at all.

Also, there is some more information you can get about clips: the current image size, bpp, frames, real fps, playback fps, whether the clip has audio or not, and if it does have audio you can get the real rate, playback rate, channels, bps, signedness and endianness.

... -- Thu, 06 Nov 2003 11:03:50 +0100 reply
Older versions of veejay had such an approach (send command & wait for answer) for sending/receiving messages but in practice it is usually not a good idea to wait for a response. It degrades the applications responsiveness.

The current implementation of gveejay processes status information to detect changes and/or errors. When playing a sample, veejay sends a status message at each frame that looks like this

playmode|current sample|start frame|end frame|looptype | ...

When the user presses for example the pingpong button , the GUI will only be updated if the looptype in the status information is changed to from 1 (normal looping) to 2 (pingpong).

Also, by looking at the current playmode ,actions related to Tag or Plain video cannot be applied while playing a sample. A message will pop up in the status text box telling the user there was an error (for example: You are not playing a Tag).

As for piping, you can use a FIFO (first in first out) , the writer will sent the next frame when the reader is done reading the current frame. So a buffer overflow will not occur in this situation unless the buffer at the receiving side is too small.

Some remarcs on OpenSoundControl

status messages -- Fri, 07 Nov 2003 20:23:31 +0100 reply
OK, I have discussed this with Neils, we decided that his suggestion was a good idea. There will be no status message returned after a command, instead an app can query another app and find out its status at any time. During playback, an app should send a sync status message when it is ready to receive another frame.

One suggestion I made which Neils liked was to include a message number with each command (except for status requests), and then the other app should return the number of the last message received as part of its status reply. That way each app can keep track of whether a message has been recieved or not.

--salsaman

libOMC -- Sat, 08 Nov 2003 14:36:20 +0100 reply
Compilable OSC-kit to play with. Also see OpenSoundControl. http://www.piksel.no/pwiki/libOMC.tar.gz

... -- Wed, 12 Nov 2003 08:24:47 +0100 reply

Friday: ogg effort -- Wed, 12 Nov 2003 08:27:12 +0100 reply
Today & tomorrow my time is fully booked. But friday I plan to explore ogg, understand its use and write down some travel tips. Stay tuned!

-- Carlo --

Note on libOMC -- Wed, 12 Nov 2003 17:05:40 +0100 reply
The cvs of pd has a more extensive implementation of OSC , it includes a server where libOMC does dot. http://cvs.sourceforge.net/viewcvs.py/pure-data/externals/OSCx/OSC/ - Niels

... -- Wed, 12 Nov 2003 17:42:40 +0100 reply
Actually libOMC provides an OSC server.




subject:
  ( 28 subscribers )

new subtopic: