MIDI Sysex "packet loss" via Translator Pro?

Hi ,

The environment:
A synth with USB connection.
A synth editor software.
Translator Pro 1.9
OS:W10

I’m trying to send soundbanks from the editor software to the synth.
Everything works perfectly, until I’m sending the presets from the
editor software directly to synth’s USB port.

Unfortunately I must use Translator’s virtual MIDI ports inside the DAW.
The problem begins here.
Presets from number 13 to 66 aren’t arriving to the synthesizer’s USB MIDI port.
1-12 ok, 67-99 ok

One soundbank contains 99 presets.
Each preset is a 1178 bytes long separate sysex message.
The captured MIDI log contains :
The first 12 presets until 0x0B “F0 01 2C 02 00 0B 00”,
The following preset is the 67. 0x44 “F0 01 2C 02 00 44 00”
All above is ok. The last one is 99. 0x62 “F0 01 2C 02 00 62 00”

3903502 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 00 00
3903502 - MIDI OUT [Pro 2] : F0 01 2C 02 00 00 00
3903513 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 01 00
3903513 - MIDI OUT [Pro 2] : F0 01 2C 02 00 01 00
3903630 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 02 00
3903630 - MIDI OUT [Pro 2] : F0 01 2C 02 00 02 00
3903746 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 03 00
3903746 - MIDI OUT [Pro 2] : F0 01 2C 02 00 03 00
3903860 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 04 00
3903860 - MIDI OUT [Pro 2] : F0 01 2C 02 00 04 00
3903977 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 05 00
3903977 - MIDI OUT [Pro 2] : F0 01 2C 02 00 05 00
3904093 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 06 00
3904093 - MIDI OUT [Pro 2] : F0 01 2C 02 00 06 00
3904208 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 07 00
3904208 - MIDI OUT [Pro 2] : F0 01 2C 02 00 07 00
3904325 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 08 00
3904325 - MIDI OUT [Pro 2] : F0 01 2C 02 00 08 00
3904441 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 09 00
3904441 - MIDI OUT [Pro 2] : F0 01 2C 02 00 09 00
3904558 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 0A 00
3904558 - MIDI OUT [Pro 2] : F0 01 2C 02 00 0A 00
3904675 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 0B 00
3904675 - MIDI OUT [Pro 2] : F0 01 2C 02 00 0B 00
3905611 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 44 00
3905611 - MIDI OUT [Pro 2] : F0 01 2C 02 00 44 00
3905623 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 45 00
3905623 - MIDI OUT [Pro 2] : F0 01 2C 02 00 45 00
.
.
.
3908876 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 61 00
3908876 - MIDI OUT [Pro 2] : F0 01 2C 02 00 61 00
3908992 - MIDI IN [Bome MIDI Translator 2 Virtual In]: F0 01 2C 02 00 62 00
3908992 - MIDI OUT [Pro 2] : F0 01 2C 02 00 62 00

My question is , where is the problem ?
Inside the vst synth editor ? The missing sysex messages aren’t visible in the routed port’s log [Bome MIDI Translator 2 Virtual In] , but it works perfectly without Translator’s routing )
Inside translator ? A small initial buffer size or delay or something similar could cause this, but I didn’t find any regarding settings in the documentation.

Thanks,
steveJ

Hi,

All SysEx messages need to end with F7. This is part of the MIDI 1.0 standard. Maybe that is the issue?

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

Hi Steve,

Thank you for the quick answer.

I just cut the length of the log lines down, because of the better reading.
Every passed line is ends with F7.

OK, I would probably need to look at the project file to get the full picture.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

I just made a new project with a simple routing for reproduction.

Of course if you still need, I can send the bmtp also privately.

Are you still dropping SysEX? If so some synths require a delay of around 100ms between SysEx messages.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

Yes, the problem is same , presets interval 13-66 completely dropped.
1-12 are ok, 66-99 are also ok.

Everything works perfect if Translator is shut down.

The missing interval isn’t visible in the Translator log.
So I think Translator couldn’t able to process all of the incoming sysexes.

If you have a thru path set and still have translators with swallow on. Then the translators are blocking the thru path. I suspect there is a problem with a translator, however I cannot tell if I can’t look at them. If you don’t want them public, you can send via email.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

The bmptp contians signatures, so I sent it to you privately.

OK I misunderstood. You don’t have any translators, just MIDI thru paths.
In this case, I would need to see the actual fully length sysex that do not work.
Is there a way you can capture this into a sysex file or text file. I can try to duplicate by sending with SendSX or sendmidi to see if anything is getting dropped.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

PS. The signature actually doesn’t mean anything, it is added by MT pro for all projects that you create. Any time you change the content, a new signature is created. No one can steal anything with it.

Please rename it syx.

init.log (113.9 KB)

It is the original sysex, unfortunately I don’t have idea how to monitor Translator’s already opened ports, with an external application.

I measured two times:
Sending the 100 presets from editor to synth’s USB directly takes 12sec. (all ok)
“Sending” the 100 presets from editor via Translator’s virtual port to synth’s USB takes 2.5 sec. (not ok)

Nothing changed in the editor, no buffer , no delay , only the port.

OK. and thanks for your patience.

Here is what I did.

I sent with SendSX at full speed and captured back in SendSX.
Path SendSX → loopMIDI → MTPro - BMT1 - SendSX
Sent 116662 bytes received 115445 bytes so indeed data was missing

Then I put SendSX at slowest throttle setting and did it again. No bytes missing.

Next I put back at full speed but turned of logging in MT Pro - No bytes missing.

So conclusion is that logging will slow down the data stream and bytes will be lost. Turn of the MIDI log window and you should be fine.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

Unfortunately the problem is same with turned off logging.

I’ve checked the vst editor’s asm code, and it sends every sysex packet correctly to Translator.
What could be the reason , why Translator couldn’t handle them ?

It is understandable for me if an external instrument requires to slow down the midi data flow.
But this is a different case , because the instrument handles the sysexes correctly.

Somehow the synth doesn’t allow to the sender application to pump out every messages in no time.
(12 sec, what I measured)

Unfortunately Translator allows this extreme fast data transfer via it’s virtual MIDI port, but unable to handle it correctly.
(2 sec transfer, missing sysex messages)

When you turned off logging, did you just uncheck the boxes or close the log window? If you only unchecked the boxes, try closing the log window completely.

Steve Caldwell
Bome Customer Care


Also available for paid consulting services: bome@sniz.biz

No, I closed the log window totally with the “x”.

It seems to me a buffering or buffer handling bug in Translator.

Hi SteveJ, this is an intriguing issue!
First of all, MT Pro is tailored to be a real time system, so it does have buffers on incoming, engine, and outgoing to prevent large build-ups, infinitely growing memory usage, and symptoms like back pressure. However, a buffer overflow will be logged to the log window. So for now, I assume that MT Pro does not drop MIDI messages.

Now let’s have a look at timing:
From your log, I can see that the first 10 messages take 939ms (3904441 - 3903502). So, one single message takes 94ms.

7 years ago, we ran some informal tests on a standard PC and measured that MT Pro is capable of processing more than 1 million messages per second, i.e. 1000 MIDI messages per millisecond. So we can exclude this as a reason to drop messages, too.

Extrapolating from the message rate, the full set of 99 messages would have taken approx. 9.3s.
But we can see in the log that it only took approx. 5.5s from first received message until last message (3908992 - 3903502). There is also a pause between message 0B and 44 (highlighted in the log) of approx. 1s. So the active sending time is 4.5s. At the given message rate, that’s good for 48 messages (4500 / 94). That’s in line with the 45 messages actually received (1…12 + 67…99).

This indicates to me that the sending side (the VST) has actually only sent those 45 messages. Or that the messages are dropped somewhere between VST and MT Pro. Note that virtual MIDI ports always pass through Windows’ MIDI stack.