3. Development

You may keep the osc4py3-bigpicture graphic on hand. It shows the general organization of the packages, classes, functions, how they interact, how data are transmitted between the different layers.

_images/osc4py3-bigpicture-0.2.svg

The osc4py3 package is cut among:

  • two OSC core modules for packets manipulation and methods routing: oscbuildparse and oscmethod ;

  • four modules to run processing: oscchannel, oscscheduling, oscdistributing and oscdispatching ;

    • Module oscchannel manage transport (emission, reception) of raw OSC packets. It uses monitoring tools from oscscheduling to get information of incoming data or connexion availability for outgoing data.
    • Module oscdistributing transform OSC message and bundles into/from OSC raw packets and provide them to ad-hoc objects/functions for processing.
    • Module oscdispaching identify methods to call from messages address pattern matching and control delayed processing of bundle with future time tag.

  • helped by specialized tool modules, by transport protocol specific modules and by a set of user helper modules providing different scheduling policies.

3.1. Implementation Modules

The two modules described here may be used without core modules if you require a simpler implementation of communications and distribution of OSC packets.

3.1.1. oscbuildparse

This module has an extensive documentation string you are invited to read.

3.1.1.1. Out-Of-Band options

These options are transmitted among building and parsing functions to activate / deactivate some processing alternatives. Options (keys, type, usage) are listed in the module documentation.

That way, you can control encoding, trace data, restrict supported type tags, activate address pattern compression, enable checksum or authentication signature or encryption, etc (warning : some code is untested).

3.1.2. oscmethod

This module has an extensive documentation string you are invited to read.

Pattern matching can use two syntax, OSC defined syntax or Python regular expression syntax (former is rewritten as later).

You basically create a MethodFilter object with at least an address pattern (a string) and a callable object. By default address pattern use OSC messages patterns syntax (parameter patternkind)

3.2. Core Modules

3.2.1. oscchannel

Organize communication channels to send and receive OSC packets from different transport protocols. TransportChannel provides an abstract class with ad-hoc interfaces. It gives some common services to subclasses and allow organization of sockets management to avoid multiplying threads (use of select or ad-hoc platform specific socket monitoring service upon a scheduling scheme). Specific handlers can be installed at transport channel object level, to have special operations occurring at identified processing time of in/out data (see actions_handlers).

3.2.2. oscscheduling

This module does the real job of monitoring the transport channels and transmitting data

3.2.3. oscdistributing

3.2.4. oscdispatching

3.3. Specialized Tools

3.3.1. oscpacketoptions

3.3.2. osctoolspools

3.3.3. oscnettools

3.4. Transport Protocol

3.4.1. Base transport class

Each transport protocol has its own module defining one or more TransportChannel subclasses. All these subclasses use the same construction scheme:

channel = ChannelClass("channelname", mode, { 'option_key': optionvalue })

The mode is a combination of 'r' and 'w' for read and write channels (ie. OSC servers to receive/read packets and OSC clients to send/write packets), and 'e' for stream based channels waiting for connection event.

3.4.1.1. Common channel options

Most parameters of channels are set via the third parameter in the options dictionary. Here is a description of possible keys and values (with default value) :

  • auto_start (True) ‒ bool flag to immediately activate the channel and start monitoring its activity once is has been initialized (else you must call yourself activate() and begin_scheduling() methods)
  • logger (None) ‒ Python logging.Logger to trace activity. If left to None, there is almost no overhead… but you silently pass all errors. You may better setup one logger with an logging.ERROR filtering level.
  • actions_handlers ({}) ‒ map { str : callable / str } of action verb and code or message to call, see “Action Handlers“ below. For advanced usage with personal hacks at some processing times.
  • monitor (Monitoring) ‒ monitor used for this channel - the channel register itself among this monitor when becoming scheduled. This is the tool used to detect state modification on the channel and activate reading or writing of data.
  • monitor_terminate (False) ‒ bool flag to indicate that our monitor must be terminated with the channel deletion. Default upon automatic monitor, or use monitor_terminate key in options.
  • read_forceident (None) ‒ map { source : identification } informations to use for peer identification on read data (don’t use other identification ways).
  • read_dnsident (True) ‒ bool flag to use address to DNS mapping automatically built from oscnettools module.
  • read_datagram (False) ‒ bool flag to consider received data as entire datagrams (no data remain in the buffer, its all processed when received).
  • read_maxsources (500) ‒ int count of maximum different sources allowed to send data to this reader simultaneously, used to limit an eventuel DOS on the system by sending incomplete packets. Set it to 0 to disable the limit. Default to MAX_SOURCES_ALLOWED (500).
  • read_withslip (False) ‒ bool flag to enable SLIP protocol on received data.
  • read_withheader (False) ‒ bool flag to enable detection of packet size in the beginning of data, and use read_headerunpack.
  • read_headerunpack (None) ‒ (str,int,int) for automatic use of struct.unpack() or fct(data) → packsize,headsize to call a function. For struct.unpack(), data is a tuple with: the format to decode header, the fixed header size, and the index of packet length value within the header tuple returned by unpack(). For function, it will receive the currently accumulated data and must return a tuple with: the packet size extracted from the header, and the total header size. If there is no enough data in header to extract packet size, the function must return (0, 0). Default to ("!I", 4, 0) to detect a packet length encoded in 4 bytes unsigned int with network bytes order. If the function need to pass some data in the current buffer (ex. remaining of an old failed communication), it can return an header size corresponding to the bytes count to ignore, and a packet size of 0 ; this will consume data with an -ignored- empty packet.
  • read_headermaxdatasize (1 MiB) ‒ int maximum count of bytes allowed in a packet size field when using headers. Set it to 0 to disable the limit. Default to MAX_PACKET_SIZE_WITH_HEADER (1 MiB).
  • write_workqueue (None) ‒ WorkQueue queue of write jobs to execute. This allow to manage initial writing to peers in their own threads (nice for blocking write() calls) or in an event loop if working without thread. The queue will be filled when we detect that a write can occur (ie same channel will have maximum one write operation in the workqueue, even if there are multiple pending operations in the write_pending queue).
  • write_wqterminate (False) ‒ bool flag to indicate to call work queue termination when terminating the channel.
  • write_withslip (False) ‒ bool flag to enable SLIP protocol on sent data.
  • write_slip_flagatstart (True) ‒ bool flag to insert the SLIP END code (192) at beginning of sent data (when using SLIP).

3.4.1.2. Network address options

For channels using network address and port, the oscnettools module provide a common way to retrieve these informations. This is done via a options with variable prefix :

  • <prefix>_host (no default) ‒ str representing an host, as a host name resolved via DNS, or as an IP address in IPV4 format or IPV6 format. Can use "*" string to specify wildcard address (ex. use with TCP to have server socket on all networks).
  • <prefix>_port (no default) ‒ int or str representing a network port number or service name. Can use 0 integer or "None" string to specify random port.
  • <prefix>_forceipv4 (False) ‒ bool flag to require use of IPV4 address in case of multiple address family resolution by a DNS.
  • <prefix>_forceipv6 (False) ‒ bool flag to require use of IPV6 address in case of multiple address family resolution by a DNS.

3.4.2. Datagram transport class

Module oscudpmc manage transport for datagram protocols over IP : UDP, multicast, broadcast, etc. All these protocols share the same UdpMcChannel transport class, multicast and broadcast simply being enabled via options. An UdpMcChannel object can only act as a server or as a client, not both.

3.4.2.1. Datagram channel options

As such transport channels can be used either as a server (reader, receiving OSC packets) or as a client (writer, sending OSC packets), some options have read or write prefix and are only considered when using channel accordingly.

3.4.2.2. Read options

  • udpread_host ‒ see “Network address options“, above. This is the address where the socket is bound for reading.
  • udpread_port ‒ see “Network address options“, above. This is the port where the socket is bound for reading.
  • udpread_forceipv4 ‒ see “Network address options“, above.
  • udpread_forceipv6 ‒ see “Network address options“, above.
  • udpread_dontcache (False) ‒ bool flag to not cache data in case of DNS resolution. By default resolved DNS addresses are cached in the application.
  • udpread_reuseaddr (True) ‒ bool flag to enable ioctl settings for reuse of socket address.
  • udpread_nonblocking (True) ‒ bool flag to enable non-blocking on the socket.
  • udpread_identusedns (False) ‒ bool flag to translate address to DNS name using oscnettools DNS addresses cache.
  • udpread_identfields (2) ‒ int count of fields of remote address identification to use for source identification. Use 0 for all fields. Default to 2 for (hostname, port) even with IPV6.
  • udpread_asstream (False) ‒ bool flag to process UDP packets with stream-based methods, to manage rebuild of OSC packets from multiple UDP reads. Bad idea - but if you need it, don’t miss to set up options like read_withslip, read_withheader…

3.4.2.3. Write options

  • udpwrite_host ‒ see “Network address options“, above. This is the address where the socket will send written packets.
  • udpwrite_port ‒ see “Network address options“, above. This is the port where the socket will send written packets.
  • udpwrite_outport (0) ‒ int number of port to bind the socket locally. Default to 0 for auto-select.
  • udpwrite_forceipv4 ‒ see “Network address options“, above.
  • udpwrite_forceipv6 ‒ see “Network address options“, above.
  • udpwrite_dontcache (False) ‒ bool flag to not cache data in case of DNS resolution. By default resolved DNS addresses are cached in the application.
  • udpread_reuseaddr (True) ‒ bool flag to enable ioctl settings for reuse of socket address.
  • udpwrite_ttl (None) - int time to leave counter for packets, also used for multicast hops. Default leave OS default settings.
  • udpwrite_nonblocking (True) ‒ bool flag to enable non-blocking on the socket.

3.4.2.4. Multicast & Broadcast options

  • mcast_enabled (False) ‒ bool flag to enable multicast. If True, the udpwrite_host must be a multicast group, and its a good idea to set udpwrite_ttl to 1 (or more if need to reach furthest networks).
  • mcast_loop (None) ‒ bool flag to enable/disable looped back multicast packets to host. Normally enabled by default at the OS level. Default to None (don’t modify OS settings).
  • bcast_enabled (False) ‒ bool flag to enable broadcast. If True, the udpwrite_host must be a network broadcast address, and its a good idea to set udpwrite_ttl to 1 (or more if need to reach furthest networks).

3.4.3. Stream transport class

Network stream based transport using TCP is defined in module osctcp. It uses the class TcpChannel to manage connection and to transmissions.

3.4.3.1. TCP

  • tcp_consocket (None) ‒ socket specified when creating a TcpChannel just after a connection, to manage communications with peer.
  • tcp_consocketspec (None) ‒ tuple specifying socket specs.
  • tcp_host ‒ see “Network address options“, above. For a TCP server, you may generally use "*" here to require a server listening on all networks. For a TCP client, just specify the server host.
  • udpread_port ‒ see “Network address options“, above.
  • udpread_forceipv4 ‒ see “Network address options“, above.
  • udpread_forceipv6 ‒ see “Network address options“, above.
  • tcp_reuseaddr (True) ‒ bool flag to enable ioctl settings for reuse of socket address.

3.5. User Helpers

These modules are described in the user documentation - we will not describe their interface.

3.5.1. as_eventloop

3.5.2. as_allthreads

3.5.3. as_comthreads

3.6. Action Handlers

These are action verbs which can be associated, at the transport channel level, to locally dispatched OSC messages or to direct callback functions to have special processing in some conditions.

3.6.1. Generic

These action handlers apply to all channel kind.

  • activating — The channel is being activated (ie. system access via open() or like). No action parameter.
  • activated — The channel has been activated. No action parameter.
  • deactivating — The channel is being deactivated (ie. stop system access via close() or like). No action parameter.
  • deactivated — The channel has been deactivated.
  • scheduling — The channel is being scheduled (ie. begin monitoring of I/O on the underlying layers). No action parameter.
  • scheduled — The channel has been scheduled. No action parameter.
  • unscheduling — The channel is being unscheduled. No action parameter.
  • unscheduled — The channel has been unscheduled. No action parameter.

TODO: Add handlers to get packetoption structures from the channel.

  • encodepacket — A packet must be encoded to send via the channel.

    Two action parameters, the OSC source packet to encode and the packet options for processing. The handler must be a direct callback (as the second parameter is not valid for sending via OSC messages).

    If the handler return None, the processing of the packet continue (standard encoding, then sending).

    If the handler return (None, None), the processing of the packet stop - we consider that the handler manage itself the packet transmission to the channel.

    If the handler return (rawoscdata, packet option), they are used to transmit the raw packet via the channel, and standard encoding is not used.

  • decodepacket — A packet coming from a transport channel must be decoded.

    Two action parameters, the raw OSC packet data to decode and the packet options for processing. The handler must be a direct callback (as the second parameter is not valid for sending via OSC messages).

    If the handler return None, the processing of the packet continue (standard decoding, then queue for dispatching).

    If the handler return (None, None), the processing of the packet stop - we consider that the handler manage itself the packet transmission to the dispatcher.

    If the handler return (packet, packet option), they are used to queue the packet for dispatching, and standard decoding is not used.

3.6.2. UDP

  • bound — The UDP socket has been bound to a port, waiting for writing or reading. Action parameter is the port number.

3.6.3. TCP

  • conreq — A connexion request has been received and a transport channel will be created to manage communications on the corresponding socket. A callback trigged on this handler can raise an exception to cancel the establishment of TCP communications. If a callback method

    Two parameters: (address, sockfileno). The remote network address as a tuple (maybe more than two items with IPV6) and the socket file number as an integer.

  • connected — A connexion has been established with a remote pair.