[LAD] MIDI-2-TCP, TCP-2-MIDI

[Jonathan E. Brickman]

I need lossless JACK MIDI networking outside of JACK's built-in
networking, and not multicast unless someone can tell me
straightforwardly how to get multicast (qmidinet) to run within
localhost as well as outside it. Thus I am thinking of trying my hand
at using the Mido library to bridge JACK MIDI and TCP. I have never
done this sort of coding before, programmatorially I am mostly a deep
scripting guy, Python-heavy with a bunch of Bash on Linux, Powershell-
heavy on Windows of late, with a pile of history on back in Perl on
both and VBA on Windows. Anyone have
hints...suggestions...alternatives...a best or better starting
place? Right now I don't want the applets to do GUI at all, I just
want them to sit quietly in xterms, on JACK servers, keeping
connection, and passing MIDI data to and fro, as other processes and
devices bring it.

[Len Ovens]

I need lossless JACK MIDI networking outside of JACK's built-in networking, and
not multicast unless someone can tell me straightforwardly how to get multicast
(qmidinet) to run within localhost as well as outside it. Thus I am thinking of
trying my hand at using the Mido library to bridge JACK MIDI and TCP. I have
never done this sort of coding before, programmatorially I am mostly a deep
scripting guy, Python-heavy with a bunch of Bash on Linux, Powershell-heavy on
Windows of late, with a pile of history on back in Perl on both and VBA on
Windows. Anyone have hints...suggestions...alternatives...a best or better
starting place? Right now I don't want the applets to do GUI at all, I just want
them to sit quietly in xterms, on JACK servers, keeping connection, and passing
MIDI data to and fro, as other processes and devices bring it.

While I have not had any issues with qmidinet, it is not immune to packet
loss. If you want a place to start I would suggest rtpMIDI would do what
you want and be a great service to the linux community. While there have
been in the past rtpmidi implementations in Linux, they seem to have
suffered bitrot and in fact I don't even know if the source is still
available.

https://en.wikipedia.org/wiki/RTP-MIDI#Linux

They mention Scenic, but anything I tried with that (like building from
source) did not work. (it has been 1 or 2 years since I tried) The full
implementation at least guarantees all note off events make it through.
There was a google repo called MIDIKIT, but google has shut all that stuff
down. I don't know if https://github.com/jpommerening/midikit is the same
code or not as they have no readme and the last commit is 2015.

I don't know as I like to use node, but:
https://github.com/jdachtera/node-rtpmidi
is a bit newer.

rtpmidi that shows up in alsa or jack with zeroconf support would be a
nice addition to Linux audio. (as would a whole pile of other things :)


--
Len Ovens
www.ovenwerks.net

[c***@web.de]

Hi !
I would always prefer a UDP based solutions,  because TCP can really mess up the timing. UDP packetloss usually is below 1%. The bigger problem in this case are WIFI connections, scrambled packet orders and jitter.
Are there any objections to using Open Sound Control based solutions?
To me it makes more sence, because it is an IP-based protocol (32 bit) in contrast to MIDI, which is designed for 8 bit serial interfaces.
A basic MIDI OSC gateway is available here:
https://launchpad.net/oscmidi
Further more, you could use it to control OSC enabled applications like Ardour and provide some customized control mappings.
I wanted to start some extension development on this myself for the public internet (unicast).
Perhaps we can align some effort on this?
I already did a specialized Sysex gateway with OSC which is absolutly relyable. (Remote control of two Yamaha mixing desks).
BR,

I need lossless JACK MIDI networking outside of JACK's built-in networking, and
not multicast unless someone can tell me straightforwardly how to get multicast
(qmidinet) to run within localhost as well as outside it. Thus I am thinking of
trying my hand at using the Mido library to bridge JACK MIDI and TCP. I have
never done this sort of coding before, programmatorially I am mostly a deep
scripting guy, Python-heavy with a bunch of Bash on Linux, Powershell-heavy on
Windows of late, with a pile of history on back in Perl on both and VBA on
Windows. Anyone have hints...suggestions...alternatives...a best or better
starting place? Right now I don't want the applets to do GUI at all, I just want
them to sit quietly in xterms, on JACK servers, keeping connection, and passing
MIDI data to and fro, as other processes and devices bring it.
While I have not had any issues with qmidinet, it is not immune to packet
loss. If you want a place to start I would suggest rtpMIDI would do what
you want and be a great service to the linux community. While there have
been in the past rtpmidi implementations in Linux, they seem to have
suffered bitrot and in fact I don't even know if the source is still
available.
https://en.wikipedia.org/wiki/RTP-MIDI#Linux
They mention Scenic, but anything I tried with that (like building from
source) did not work. (it has been 1 or 2 years since I tried) The full
implementation at least guarantees all note off events make it through.
There was a google repo called MIDIKIT, but google has shut all that stuff
down. I don't know if https://github.com/jpommerening/midikit is the same
code or not as they have no readme and the last commit is 2015.
https://github.com/jdachtera/node-rtpmidi
is a bit newer.
rtpmidi that shows up in alsa or jack with zeroconf support would be a
nice addition to Linux audio. (as would a whole pile of other things :)
--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Len Ovens]

Post by c***@web.de
I would always prefer a UDP based solutions,  because TCP can really mess up the
timing. UDP packetloss usually is below 1%. The bigger problem in this case are
WIFI connections, scrambled packet orders and jitter.
Are there any objections to using Open Sound Control based solutions?
To me it makes more sence, because it is an IP-based protocol (32 bit) in
contrast to MIDI, which is designed for 8 bit serial interfaces.

OSC being lossless has not been my experience. The problem I have had is
the OSC messages are generally one message per packet which means that a
large group of messages can overwhelm udp quite easily. OSC does allow for
using bundles of messages to be performed at the same time, however MIDI
to OSC cannot really determine a group of events that happen at the same
time because of it's (slow) serial nature.

Do note that the osc message "stormes" I have had trouble with are bigger
than what MIDI was designed to handle in realtime (10 events from 10
fingers). I am talking about refreshing a control surface with at least 8
strips with each strip having 20 or so events. So well over 100 events.
When I tried to use bundles, I found that no control surfaces created or
understood bundled messages. I ended up adding a small delay in the sends
to fix this... not very "real time" :) Not noticable while moving one
control like a fader but noticable if performing music.


--
Len Ovens
www.ovenwerks.net

[c***@web.de]

Hey Len,
thanks for the insight.
I never used OSC this way so far.
I also did not know that there are existing RFCs for MIDI over RTP, which is very nice!
So, yeah, lets do that.
I will take a closer look at the code repos you posted.  I definitly want to give this a try!
But I am rather busy at the moment, so don't expect too fast progress in this matter ;-)
BR,

Post by c***@web.de
I would always prefer a UDP based solutions,  because TCP can really mess up the
timing. UDP packetloss usually is below 1%. The bigger problem in this case are
WIFI connections, scrambled packet orders and jitter.
Are there any objections to using Open Sound Control based solutions?
To me it makes more sence, because it is an IP-based protocol (32 bit) in
contrast to MIDI, which is designed for 8 bit serial interfaces.
OSC being lossless has not been my experience. The problem I have had is
the OSC messages are generally one message per packet which means that a
large group of messages can overwhelm udp quite easily. OSC does allow for
using bundles of messages to be performed at the same time, however MIDI
to OSC cannot really determine a group of events that happen at the same
time because of it's (slow) serial nature.
Do note that the osc message "stormes" I have had trouble with are bigger
than what MIDI was designed to handle in realtime (10 events from 10
fingers). I am talking about refreshing a control surface with at least 8
strips with each strip having 20 or so events. So well over 100 events.
When I tried to use bundles, I found that no control surfaces created or
understood bundled messages. I ended up adding a small delay in the sends
to fix this... not very "real time" :) Not noticable while moving one
control like a fader but noticable if performing music.
--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Jonathan E. Brickman]

In general I too am attracted to UDP -- but for MIDI performance
transmission, 0.001% loss is still far too much, because that means one
note in 1,000 might be held and never released, causing massive
encruditation to the moment :-) This is because every time I press a
key there's a MIDI signal for the press, and a separate one for the
release, and if the release is lost, we can have massive
unpleasantry. And a song can easily have thousands of notes. Some of
my tests over the years actually included this behavior!
So it's either TCP only, or it's UDP with complete error
correction. UDP with complete error correction is how NFS over UDP has
been working for ages, so that is clearly an option, but it is also not
exactly trivial programmatically :-)
I have read a lot about OSC. It has seemed to me that it would have to
be an option, given that it seems to have been designed from the
beginning to run over IP, and otherwise to sidestep all of the well-
known MIDI limitations. But whenever I have dug into it in the past, I
have found myself quite profoundly confused by the massive flexibility.
Recently I ran into OSC2MIDI, and if my understanding of what OSC is
is correct, OSC2MIDI should theoretically be able to do the job if it
is on both ends of the stream, correct? I'll do a bit of testing of
this, see if I can figure out a bit of toolchain design, but input of
experienced persons is much desired.
I will also look at the repos for MIDI over RTP. Sounds like it's
being used in production now for loss-tolerant control surfaces though,
and not performance transmission, correct?
I had not realized that TCP could produce timing errors, I do
understand that now, I remember that being a challenge in the early
development of streaming audio. I wonder if OSC2MIDI can use OSC time
tag data to handle:http://opensoundcontrol.org/spec-1_0
I am working on compiling OSC2MIDI right now, does not appear trivial,
we'll see :-)
J.E.B.

Post by c***@web.de
Hey Len,
thanks for the insight.
I never used OSC this way so far.
I also did not know that there are existing RFCs for MIDI over RTP, which is very nice!
So, yeah, lets do that.
I will take a closer look at the code repos you posted. I definitly
want to give this a try!
But I am rather busy at the moment, so don't expect too fast progress in this matter ;-)
BR,
Ck
Mittwoch, 29 August 2018, 09:00nachm. +01:00 von Len Ovens

Post by c***@web.de

I would always prefer a UDP based solutions, because TCP can

really mess up the

timing. UDP packetloss usually is below 1%. The bigger problem in

this case are

WIFI connections, scrambled packet orders and jitter.
Are there any objections to using Open Sound Control based

solutions?

To me it makes more sence, because it is an IP-based protocol (32

bit) in

contrast to MIDI, which is designed for 8 bit serial interfaces.

OSC being lossless has not been my experience. The problem I have had is
the OSC messages are generally one message per packet which means that a
large group of messages can overwhelm udp quite easily. OSC does allow for
using bundles of messages to be performed at the same time, however MIDI
to OSC cannot really determine a group of events that happen at the same
time because of it's (slow) serial nature.
Do note that the osc message "stormes" I have had trouble with are bigger
than what MIDI was designed to handle in realtime (10 events from 10
fingers). I am talking about refreshing a control surface with at least 8
strips with each strip having 20 or so events. So well over 100 events.
When I tried to use bundles, I found that no control surfaces created or
understood bundled messages. I ended up adding a small delay in the sends
to fix this... not very "real time" :) Not noticable while moving one
control like a fader but noticable if performing music.
--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

_______________________________________________Linux-audio-dev
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Len Ovens]

In general I too am attracted to UDP -- but for MIDI performance transmission,
0.001% loss is still far too much, because that means one note in 1,000 might be
held and never released, causing massive encruditation to the moment :-) This is
because every time I press a key there's a MIDI signal for the press, and a
separate one for the release, and if the release is lost, we can have massive
unpleasantry. And a song can easily have thousands of notes. Some of my tests
over the years actually included this behavior!  

All note offs must be received for good performance. I agree.

I have read a lot about OSC. It has seemed to me that it would have to be an
option, given that it seems to have been designed from the beginning to run over
IP, and otherwise to sidestep all of the well-known MIDI limitations. But
whenever I have dug into it in the past, I have found myself quite profoundly
confused by the massive flexibility.  Recently I ran into OSC2MIDI, and if my

OSC has no "standard" for performance transmition except MIDI via OSC
which ends up having all the same problems as MIDI alone. It would of
course be possible to send messages that were note with length... but that
would mean a delay at least as long as the note was played because the
message can not be sent untill note off.

understanding of what OSC is is correct, OSC2MIDI should theoretically be able to
do the job if it is on both ends of the stream, correct? I'll do a bit of testing
of this, see if I can figure out a bit of toolchain design, but input of
experienced persons is much desired.

I personally don't see how that would help. It sounds like translating an
english email to french to send it and then translating back to english on
the receiving end. It is UDP in both cases. Unless I am missing something.

I will also look at the repos for MIDI over RTP. Sounds like it's being used in
production now for loss-tolerant control surfaces though, and not performance
transmission, correct?

It is designed for performance as well or even first. It is a journelled
setup that sends both the performance and a journel. The journel allows
missing packets to be noted and replaced. It tries to be smart about what
it recreates. For example, a note off is always recreated even if it ends
up late. A note on that shows up after it's note off will not. So it is
the better than tcp, where a note may sound obviously out of time due to a
retry. rtpmidi is what apple coreaudio uses as it's midi transport. A
properly "advertised" rtmidi port will show up in core audio just like any
other midi port. It is however, true that some of the linux
implementations have gotten it working but have never completed the
journeling part of things (maybe because it worked for them well enough
without) and so for them it is no better than ipmidi (which also tends to
be quite good in a local network context). The transport part of the code
is the easiest and the journel part would take work... at least that is my
guess as the reason so many are partly done.

Tcp with timing information and post analysis could do the same
thing, deciding not to use late note on events. With the speed of networks
inceasing and faster processing at both ends, tcp may be fast enough. A
lot depends on how busy the network is... what other traffic is present. I
have had both good and bad exeriences with udp both on wifi and localhost.
Even using localhost, it seems too many udp packets at a time seems to
result in packet loss. (I say packet loss, but it is possible the
receiving OSC lib ran out of buffer too).


--
Len Ovens
www.ovenwerks.net

[Paul Davis]

On Sat, Sep 1, 2018 at 10:07 PM, Len Ovens <***@ovenwerks.net> wrote:

[ etc. etc. etc. ]

i wonder if sctp (the transport protocol used for web sockets) might be
better for this sort of thing than either tcp or udp or raw ip ...

[Christoph Kuhr]

Hi Paul,

do you have some useful links or could name the RFCs for the
retransmission part of SCTP?

Perhaps it might be an idea to combine RTP-MIDI with TCSP?

BR,
Ck

Post by Paul Davis
[ etc. etc. etc. ]
i wonder if sctp (the transport protocol used for web sockets) might be
better for this sort of thing than either tcp or udp or raw ip ...
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Ralf Mardorf]

[Active Sensing]

You said that you "need lossless JACK MIDI networking", but not why you
need networking at all. You might have a good reason, I'm just curious.
For what purpose do you need an _additional_ network?

Btw. I have no experiences with MIDI over an additional network, but
regarding the missing note-off issue, a standard MIDI "network" (= MIDI
not over an additional network) has got the "Active Sense" (0xFE)
message, that could workaround some (not all) missing note-off issues.

[Hanspeter Portner]

Post by Len Ovens

In general I too am attracted to UDP -- but for MIDI performance transmission,
0.001% loss is still far too much, because that means one note in 1,000 might be
held and never released, causing massive encruditation to the moment :-) This is
because every time I press a key there's a MIDI signal for the press, and a
separate one for the release, and if the release is lost, we can have massive
unpleasantry. And a song can easily have thousands of notes. Some of my tests
over the years actually included this behavior!  

All note offs must be received for good performance. I agree.

I have read a lot about OSC. It has seemed to me that it would have to be an
option, given that it seems to have been designed from the beginning to run over
IP, and otherwise to sidestep all of the well-known MIDI limitations. But
whenever I have dug into it in the past, I have found myself quite profoundly
confused by the massive flexibility.  Recently I ran into OSC2MIDI, and if my

OSC has no "standard" for performance transmition except MIDI via OSC which ends
up having all the same problems as MIDI alone. It would of course be possible to
send messages that were note with length... but that would mean a delay at least
as long as the note was played because the message can not be sent untill note off.

Yet another idea:

Instead of implementing RTP-MIDI (and its hacky journal), I've had a very good
experience with translating (stateless) MIDI to a (stateful) protocol for UDP
communication.

TUIO [1] builds on top of OSC and has been designed to gracefully recover from
packet loss. Don't be intimidated by the spec, you'll only need a really minimal
subset of it (easily implementable) to accomplish your goal.

[1] https://www.tuio.org/?tuio20

Post by Len Ovens

understanding of what OSC is is correct, OSC2MIDI should theoretically be able to
do the job if it is on both ends of the stream, correct? I'll do a bit of testing
of this, see if I can figure out a bit of toolchain design, but input of
experienced persons is much desired.

I personally don't see how that would help. It sounds like translating an
english email to french to send it and then translating back to english on the
receiving end. It is UDP in both cases. Unless I am missing something.

I will also look at the repos for MIDI over RTP. Sounds like it's being used in
production now for loss-tolerant control surfaces though, and not performance
transmission, correct?

It is designed for performance as well or even first. It is a journelled setup
that sends both the performance and a journel. The journel allows missing
packets to be noted and replaced. It tries to be smart about what it recreates.
For example, a note off is always recreated even if it ends up late. A note on
that shows up after it's note off will not. So it is the better than tcp, where
a note may sound obviously out of time due to a retry. rtpmidi is what apple
coreaudio uses as it's midi transport. A properly "advertised" rtmidi port will
show up in core audio just like any other midi port. It is however, true that
some of the linux implementations have gotten it working but have never
completed the journeling part of things (maybe because it worked for them well
enough without) and so for them it is no better than ipmidi (which also tends to
be quite good in a local network context). The transport part of the code is the
easiest and the journel part would take work... at least that is my guess as the
reason so many are partly done.
Tcp with timing information and post analysis could do the same thing, deciding
not to use late note on events. With the speed of networks inceasing and faster
processing at both ends, tcp may be fast enough. A lot depends on how busy the
network is... what other traffic is present. I have had both good and bad
exeriences with udp both on wifi and localhost. Even using localhost, it seems
too many udp packets at a time seems to result in packet loss. (I say packet
loss, but it is possible the receiving OSC lib ran out of buffer too).

[Christoph Kuhr]

Post by Hanspeter Portner
TUIO [1] builds on top of OSC and has been designed to gracefully recover from
packet loss. Don't be intimidated by the spec, you'll only need a really minimal
subset of it (easily implementable) to accomplish your goal.
[1] https://www.tuio.org/?tuio20

Is there any documentation about the error correction capabilities? I
didn't find any in the specs. Or is this only "documented" as code?

BR,
Ck

[Hanspeter Portner]

Post by Hanspeter Portner
TUIO [1] builds on top of OSC and has been designed to gracefully recover from
packet loss. Don't be intimidated by the spec, you'll only need a really minimal
subset of it (easily implementable) to accomplish your goal.
[1] https://www.tuio.org/?tuio20

Is there any documentation about the error correction capabilities? I didn't
find any in the specs. Or is this only "documented" as code?

It's all in the spec, e.g. this should get you going:

/Message Structure/Global Messages/ALV (alive message)

[Christoph Kuhr]

If I understand it correctly, any message bundle can be acknowledged by
an ALV Message. And if some message in the bundle gets lost, it can be
reconstructed from the ALV message?

If so, what happens if the ALV message is lost?

BR,
Ck

Post by Hanspeter Portner

Post by Hanspeter Portner
TUIO [1] builds on top of OSC and has been designed to gracefully recover from
packet loss. Don't be intimidated by the spec, you'll only need a really minimal
subset of it (easily implementable) to accomplish your goal.
[1] https://www.tuio.org/?tuio20

Is there any documentation about the error correction capabilities? I didn't
find any in the specs. Or is this only "documented" as code?

/Message Structure/Global Messages/ALV (alive message)
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Ralf Mardorf]

to sidestep all of the well-known MIDI limitations

Without doubts MIDI has got well-known limitations, but nowadays a bad
implementation of the MIDI standard often gets confused with the MIDI
standard, so it's better to clearly point out a limitation instead of
overgeneralizeing the issues.

[Spencer Jackson]

Recently I ran into OSC2MIDI, and if my understanding of what OSC is is

correct, OSC2MIDI should theoretically be able to do the job if it is on
both ends of the stream, correct? I'll do a bit of testing of this, see if
I can figure out a bit of toolchain design, but input of experienced
persons is much desired.

I am working on compiling OSC2MIDI right now, does not appear trivial,

we'll see :-)

I wrote OSC2MIDI with such a use case in mind as well, though it was not my
primary one so it's not very tested (I just wanted to use android devices
to control linux synths via osc->midi). I'm happy to help you through
compiling it, I hoped that it was trivial. Please let me know whatever
issues you come across (perhaps in a github issue so we don't create too
much noise in this list).

I had not realized that TCP could produce timing errors, I do understand

that now, I remember that being a challenge in the early development of
streaming audio. I wonder if OSC2MIDI can use OSC time tag data to handle:
http://opensoundcontrol.org/spec-1_0

That is a great idea, but unfortunately I have many more ideas than time of
late, so any implementation soon is quite unlikely. I would be happy to
assist/advise anyone interested in contributing this though.

On UPD/TCP IIRC liblo the OSC library I used, offers support for TCP
connections as well as UDP, and I believe all you need to do is supply an
argument like "-a osc.tcp://localhost:8000"
I would like to explore using something like UDT (
https://git.dorkbox.com/dorkbox/UDT) for transmitting OSC, but that seems
even less likely to receive time and attention soon as it would require
using an alternative to liblo.

Best of luck,
_Spencer

In general I too am attracted to UDP -- but for MIDI performance
transmission, 0.001% loss is still far too much, because that means one
note in 1,000 might be held and never released, causing massive
encruditation to the moment :-) This is because every time I press a key
there's a MIDI signal for the press, and a separate one for the release,
and if the release is lost, we can have massive unpleasantry. And a song
can easily have thousands of notes. Some of my tests over the years
actually included this behavior!
So it's either TCP only, or it's UDP with complete error correction. UDP
with complete error correction is how NFS over UDP has been working for
ages, so that is clearly an option, but it is also not exactly trivial
programmatically :-)
I have read a lot about OSC. It has seemed to me that it would have to be
an option, given that it seems to have been designed from the beginning to
run over IP, and otherwise to sidestep all of the well-known MIDI
limitations. But whenever I have dug into it in the past, I have found
myself quite profoundly confused by the massive flexibility. Recently I
ran into OSC2MIDI, and if my understanding of what OSC is is correct,
OSC2MIDI should theoretically be able to do the job if it is on both ends
of the stream, correct? I'll do a bit of testing of this, see if I can
figure out a bit of toolchain design, but input of experienced persons is
much desired.
I will also look at the repos for MIDI over RTP. Sounds like it's being
used in production now for loss-tolerant control surfaces though, and not
performance transmission, correct?
I had not realized that TCP could produce timing errors, I do understand
that now, I remember that being a challenge in the early development of
http://opensoundcontrol.org/spec-1_0
I am working on compiling OSC2MIDI right now, does not appear trivial,
we'll see :-)
J.E.B.
Hey Len,
thanks for the insight.
I never used OSC this way so far.
I also did not know that there are existing RFCs for MIDI over RTP, which is very nice!
So, yeah, lets do that.
I will take a closer look at the code repos you posted. I definitly want
to give this a try!
But I am rather busy at the moment, so don't expect too fast progress in this matter ;-)
BR,
Ck
Mittwoch, 29 August 2018, 09:00nachm. +01:00 von Len Ovens

I would always prefer a UDP based solutions, because TCP can really

mess up the

timing. UDP packetloss usually is below 1%. The bigger problem in this

case are

WIFI connections, scrambled packet orders and jitter.
Are there any objections to using Open Sound Control based solutions?
To me it makes more sence, because it is an IP-based protocol (32 bit) in
contrast to MIDI, which is designed for 8 bit serial interfaces.

OSC being lossless has not been my experience. The problem I have had is
the OSC messages are generally one message per packet which means that a
large group of messages can overwhelm udp quite easily. OSC does allow for
using bundles of messages to be performed at the same time, however MIDI
to OSC cannot really determine a group of events that happen at the same
time because of it's (slow) serial nature.
Do note that the osc message "stormes" I have had trouble with are bigger
than what MIDI was designed to handle in realtime (10 events from 10
fingers). I am talking about refreshing a control surface with at least 8
strips with each strip having 20 or so events. So well over 100 events.
When I tried to use bundles, I found that no control surfaces created or
understood bundled messages. I ended up adding a small delay in the sends
to fix this... not very "real time" :) Not noticable while moving one
control like a fader but noticable if performing music.
--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev
--
*Hear us at ponderworthy.com <http://ponderworthy.com> -- CDs and MP3
available!*
*Music of compassion; fire, and life!!!*
_______________________________________________
Linux-audio-dev mailing list
https://lists.linuxaudio.org/listinfo/linux-audio-dev

[Alexandre Ratchov]

I had not realized that TCP could produce timing errors, I do understand that
now, I remember that being a challenge in the early development of streaming
http://opensoundcontrol.org/spec-1_0
I am working on compiling OSC2MIDI right now, does not appear trivial, we'll
see :-)

I throw few thoughts as I use a lot midi-over-tcp:

On switched ethernet, latency is excelent and the physical link is
reliable. If the network is not busy there are no packet drops and no
delays caused by tcp retransmissions. I've never seen retransmission
in normal conditions.

If the network is busy, the host and/or the routers could be
configured to prioritize and reserve bandwidth for the midi traffic
and avoid retransmissions.

If you use udp or alike, you'll have to detect packet loss (otherwise
note-off events may be lost and certain notes would sound forever),
and retransmit packets by yourself, which is basically what tcp does
for us.

HTH

-- Alexandre

[Jonathan Brickman]

Alexandre, what library/libraries/code base are you using for
MIDI-over-TCP? Can you give a URL or two?

J.E.B.

Post by Jonathan E. Brickman

I had not realized that TCP could produce timing errors, I do understand

that

now, I remember that being a challenge in the early development of

streaming

http://opensoundcontrol.org/spec-1_0
I am working on compiling OSC2MIDI right now, does not appear trivial,

we'll

see :-)

On switched ethernet, latency is excelent and the physical link is
reliable. If the network is not busy there are no packet drops and no
delays caused by tcp retransmissions. I've never seen retransmission
in normal conditions.
If the network is busy, the host and/or the routers could be
configured to prioritize and reserve bandwidth for the midi traffic
and avoid retransmissions.
If you use udp or alike, you'll have to detect packet loss (otherwise
note-off events may be lost and certain notes would sound forever),
and retransmit packets by yourself, which is basically what tcp does
for us.
HTH
-- Alexandre

[Alexandre Ratchov]

Alexandre, what library/libraries/code base are you using for MIDI-over-TCP? 
Can you give a URL or two?

Hi,

I'm using the OpenBSD native midi api, I'm one of its maintainers. An
entry point to the documentation is here (skip the audio-specific
parts in the first link):

http://man.openbsd.org/sndio
http://man.openbsd.org/mio_open

There's a portable version that works on Linux here:

http://www.sndio.org

AFAIU, this is not what you want, but you could look at the code and
see how TCP sockets are used.

[Jonathan E. Brickman]

On Wed, Sep 05, 2018 at 03:05:44PM -0500, Jonathan Brickman
wrote:Alexandre, what library/libraries/code base are you using for
MIDI-over-TCP? Can you give a URL or two?
Hi,
I'm using the OpenBSD native midi api, I'm one of its maintainers.
Anentry point to the documentation is here (skip the audio-

http://man.openbsd.org/sndiohttp://man.openbsd.org/mio_open id="-x-evo-selection-start-marker">

http://www.sndio.org
AFAIU, this is not what you want, but you could look at the code
andsee how TCP sockets are used.

Wow. Studying. Many thanks for all of your efforts!!!

[Ralf Mardorf]

MIDI was designed to handle in realtime (10 events from 10 fingers)

That is incorrect, MIDI was designed for sequencer usage, too, so MIDI
provides 16 channels ;). While I only can play 6 channels in real-time
using my guitar synth, even my C64 and Atari ST could play 16 channels
and more (btw. with way less MIDI jitter than any Linux PC can do).
Depending on the usage, we can _not_ use one MIDI connection (one MIDI
cable) for all 16 channels, but sometimes it works, let alone that from
the beginning MIDI also was desgined for usage with several MIDI IOs,
IOW for usage with x * 16 channels. Btw. regarding some data, e.g. pitch
bend messages, not only my guitar synth allows to send "reduced"
MIDI data. From keyboards we e.g. know that after touch often isn't
used, but send. MIDI sometimes require thinking about the setup, to
avoid issues, this is even true for an anlog audio setup or synth
connected by a CV/gate "network". MIDI still requires to think about
what we want to achieve. The MIDI standard isn't made to fit the needs
of braindead consumers. If we think a little bit, we even could use
SysEx non-real-time MIDI data in real-time, without experiencing any
problem.

[Jonathan E. Brickman]

On Wed, 29 Aug 2018 13:00:07 -0700 (PDT), Len Ovens wrote:MIDI was
designed to handle in realtime (10 events from 10 fingers)
That is incorrect, MIDI was designed for sequencer usage, too, so
MIDIprovides 16 channels ;). While I only can play 6 channels in
real-timeusing my guitar synth, even my C64 and Atari ST could play
16 channelsand more (btw. with way less MIDI jitter than any Linux PC
can do).

Indeed, MIDI's 31.25 kbps gives (because of its 8+2 bit protocol) a
rough upper cap of 1500+ datacommands (including notes and timing
blips...) per second, notes being one byte per command, one more for
value. And even if we use the old (and lately often obsolete) 50%
rule, that's still 750+ items per second.

It would certainly be nice to blow all of those numbers away by two or
three orders of magnitude! And it would be gorgeous to have MIDI data
simply pervade an IP stage network, or an IP instrument network, or one
multi-instrument box through localhost, or a stack-of-Raspberry-Pis
network, or a creative combo. I don't like the idea of using CAT5e
generally on stage, because MIDI DINs are just so much more durable,
but of course CAT5e cables are inexpensive, and if the journaling
works, we could use CAT5e simply as a fallback where wifi, lifi, or
xifi [not sure IP-over-xray will happen :] is not practical.

In the last day or two I have been playing with the Mido library's
documentation examples, and just now found much more apparently
practical examples:

https://github.com/olemb/mido/tree/master/examples

including what looks like two actual JACK<-->RTP-MIDI bridges in the
'ports' and 'sockets' subsections. Will be studying. Seeking much
input :-)

[Len Ovens]

Indeed, MIDI's 31.25 kbps gives (because of its 8+2 bit protocol) a rough upper
cap of 1500+ datacommands (including notes and timing blips...) per second, notes
being one byte per command, one more for value. And even if we use the old (and
lately often obsolete) 50% rule, that's still 750+ items per second.  

A note on or note off is three bytes, chanel/command, note and velocity.
Running status (first byte doesn't change from event to event) allows a
second note on in the same chanel to omit the first byte. This is why some
controllers send note off as a note on with velocity 0. Using note on and
note off means note, release, note is 9 bytes rather than 7 bytes for note
on on_with_0_velocity, on. Anyway, 1k is about the highest one can expect
on a per event basis. "Realtime" events are single byte and patch events
are two bytes. rpn and nrpn events are a minimum of 9 - 12 bytes for the
first one sent but may be a little as 3 for a next value.... though good
practice pretty much demands sending the whole 12 bytes every time.

Jack always converts incoming midi to full three byte events. I do not
know if it sends using running status to hardware devices.

All midi "mixing" requires converting to full events as well as queuing
bytes event at a time.

It would certainly be nice to blow all of those numbers away by two or three
orders of magnitude! And it would be gorgeous to have MIDI data simply pervade an
IP stage network, or an IP instrument network, or one multi-instrument box
through localhost, or a stack-of-Raspberry-Pis network, or a creative combo. I
don't like the idea of using CAT5e generally on stage, because MIDI DINs are just

There are high use/cycle cat connectors and cables designed for this kind
of use. Take a look at almost anyone who sells network snakes or stage boxes.
Most of these cables are 100 foot cables :) but I am sure shorter cables
(or longer) can be had. Yes this would mean adding the matching connectors
on each of the boxes you wanted to connect.

In the last day or two I have been playing with the Mido library's documentation
https://github.com/olemb/mido/tree/master/examples
including what looks like two actual JACK<-->RTP-MIDI bridges in the 'ports' and
'sockets' subsections. Will be studying. Seeking much input :-)

It would be interesting to know what the throughput and latency is with
that setup. I have never thought of python as being particularly great for
real time applications. However, something that works is a great start.
The road from python to c or c++ is not too bumpy depending on the
librarys used.

--
Len Ovens
www.ovenwerks.net

[Jonathan E. Brickman]

Post by Jonathan E. Brickman
In the last day or two I have been playing with the Mido library's

documentation> examples, and just now found much more apparently
practical examples:> >
https://github.com/olemb/mido/tree/master/examples>; > including what
looks like two actual JACK<-->RTP-MIDI bridges in the 'ports' and>
'sockets' subsections. Will be studying. Seeking much input :-)
It would be interesting to know what the throughput and latency is
with that setup. I have never thought of python as being particularly
great for real time applications. However, something that works is a
great start. The road from python to c or c++ is not too bumpy
depending on the librarys used.

Agreed! Am getting there, but am running into a bobble which I am
hoping you or someone else will understand :-)

Have assembled the two files below, midi2tcp.py and tcp2midi.py; you
may find them here. When run thus, in separate xterms:

python2 tcpmidi.py localhost:44440
python2 midi2tcp.py localhost:44440

the second connects with the first via TCP, and they both sit quietly,
waiting. In Catia, I connect RtMidiIn-Client to UM-ONE (USB MIDI
interface to my keyboard), everything still sits quietly. Then I press
one key on the keyboard once, and both sides report note_on and
note_off over and over again, without stopping until I terminate
them! Have gone over the Mido docs and sample code several times, have
tried setting 'message' to None after transmit and after receive on
both sides, have also recoded to use accept(), no change. Have not
found a way to handle this besides terminating and restarting the TCP
connection at every message. Ideas anyone?

Interestingly, the frequency of the current unintended behavior is a
very steady 12 messages per 1/100 of a second (1200 messages per
second), so I'll think the Mido library is probably incorporating
timing of some sort to semi-mimic hardware MIDI and/or prevent overuse
of hardware; time will tell.

[Jonathan E. Brickman]

...am running into a bobble which I am hoping you or someone else
will understand :-)
Have assembled the two files below, midi2tcp.py and tcp2midi.py; you
python2 tcpmidi.py localhost:44440
python2 midi2tcp.py localhost:44440
the second connects with the first via TCP, and they both sit
quietly, waiting. In Catia, I connect RtMidiIn-Client to UM-ONE (USB
MIDI interface to my keyboard), everything still sits quietly. Then
I press one key on the keyboard once, and both sides report note_on
and note_off over and over again, without stopping until I terminate
them! Have gone over the Mido docs and sample code several times,
have tried setting 'message' to None after transmit and after receive
on both sides, have also recoded to use accept(), no change. Have
not found a way to handle this besides terminating and restarting the
TCP connection at every message. Ideas anyone?

Renamed them to 'midi2ip.py' and 'ip2midi.py' for accuracy:
https://github.com/ponderworthy/midi2ip
Once the current odd behavior is resolved, I had a next step in a
dream: the very simplest method to ensure serial behavior is probably
something from the very old modem days, where everything one typed was
echoed back from the receiver for verification. At the extremely low
(modernly) bandwidth of MIDI this is more than practical, and it
ensures sufficient reliability: if the sender does not get its
identical acknowledgement in perhaps 50ms, it sends probably a SysEx
representing 'retry', waits for the retry confirmation, tries that two
more times, and then tries three times to do a clean MIDI reset, and
then three times to do a MIDI panic. Or something like that, need to
incorporate socket checking in there too. But as simple as possible,
let's make the future stage hardware as inexpensive as we can :-) 50ms
is very large, but intentionally so, this is detection of failure not
jitter, it means the setup will still work with major jitter in some
marginal circumstances. When circumstances get worse, we might want to
look at running this over SSH or some such.

[Ralf Mardorf]

MIDI was designed to handle in realtime (10 events from 10 fingers)

PS: Even if we reduce MIDI to one channel for real-time playing without
usage of e.g. the nose as an eleventh finger, at least usage of pedals
is included. The amount of data send by just one pedal easily exceeds
what a human could do with ten fingers on black and white keys. A
keyboarder could use one hand to control a joystick (e.g. pitch bend and
modulation at the same time) and two feet to control two pedals and at
the same time use 5 fingers to play black and white keys with after
touch. The MIDI standard allows to do this.

[Will J Godfrey]

On Sun, 2 Sep 2018 07:52:48 +0200

Post by Ralf Mardorf

MIDI was designed to handle in realtime (10 events from 10 fingers)

PS: Even if we reduce MIDI to one channel for real-time playing without
usage of e.g. the nose as an eleventh finger, at least usage of pedals
is included. The amount of data send by just one pedal easily exceeds
what a human could do with ten fingers on black and white keys. A
keyboarder could use one hand to control a joystick (e.g. pitch bend and
modulation at the same time) and two feet to control two pedals and at
the same time use 5 fingers to play black and white keys with after
touch. The MIDI standard allows to do this.

As a matter or interest, the only time I've had missing noteoffs with
standard MIDI was when I had only a single MIDI port, and daisy-chained a sound
canvas and two keyboards (both sending active sensing). One for the
keyboards also had a pedal attached. Having said that I always used good quality
short cables.

These days, I have a 4 port USB MIDI unit, and run the same hardware along with
a substantial controller and one pedal + two foot switches - narry a problem,
and no noticeable jitter with live playing.

Just my 2d {old money}

[Paul Davis]

Post by Will J Godfrey
As a matter or interest, the only time I've had missing noteoffs with
standard MIDI was when I had only a single MIDI port, and daisy-chained a sound
canvas and two keyboards (both sending active sensing). One for the
keyboards also had a pedal attached. Having said that I always used good quality
short cables.

a couple of days ago, while working on MIDI Clock support in ardour, i was
trying to figure out the origin of some missing Clock (0xf8) messages. they
would arrive every 830 samples plus or minus about 30 samples,. but every
once in a while, the gap would be twice that.

long story cut short: just changing from using my MOTU Ultralite AVB for
MIDI I/O to a Midisport 2x2 fixed the problem. No more missing Clock
messages.

could be relevant to stories like the one above. there's no good reason for
this, but it is how things are.

[Ralf Mardorf]

Post by Paul Davis

Post by Will J Godfrey
As a matter or interest, the only time I've had missing noteoffs with
standard MIDI was when I had only a single MIDI port, and daisy-chained a sound
canvas and two keyboards (both sending active sensing). One for the
keyboards also had a pedal attached. Having said that I always used good quality
short cables.

a couple of days ago, while working on MIDI Clock support in ardour, i
was trying to figure out the origin of some missing Clock (0xf8)
messages. they would arrive every 830 samples plus or minus about 30
samples,. but every once in a while, the gap would be twice that.
long story cut short: just changing from using my MOTU Ultralite AVB
for MIDI I/O to a Midisport 2x2 fixed the problem. No more missing
Clock messages.
could be relevant to stories like the one above. there's no good
reason for this, but it is how things are.

Such an issue could be related to the used opto-coupler and diode or by
a trim potentiometer for sensitivity that isn't set up properly. I
adjusted the trim potentiometer of my DIY MIDI throu box build in the
80s to work with old gear, but nowadays USB devices not always provide
enough current to drive the MIDI throu box with this adjustment. I heard
of issues related of slow opto-couplers or diodes to fix slope issues,
but never experienced this myself.