Error 15 MIDI Input Buffer Overflow

Hi Gabriel,

Solution 1 is probably your best bet but you would need another global variable. You simply set the variable to 1 and put a rule in each translator bound for the APC MINI as follows:

if ga==1 the exit rules, skip outgoing action
When the APC MINI is ready for more rules you set the global variable (ga in this example) back to 0 and then
the translator will fire on the next input.

As far as global variables are concerned. How many states are you tracking with each button. Every global variable is a 32 bit signed integer so there may be a bit mapping technique you can use to take more advantage or the 32 to bits available for a single global variable. I can track 64 button on/off states with just 2 global variables, each button representing a single bit of the global variable. If you want to track 3-4 states it would take 2 bits. For 5-8 states (colors) it would take 3 bits. So with a single variable with 8 color states, you could track 10 buttons with a single variable. If interested, I can send you an example of how this is done,

The other way you could disable input without using a global variable is to put all APC MINI incoming messages under a separate preset, and the a translator to disable/enable that preset as needed. If you need to enable/disable multiple presets, you would need to use a single timer event that fire multiple translator, each disabling/enabling a different preset.

We may also need to look at the APC MINI manual. I know on the Novation products you can turn on/off LED’s with Sysex messages instead of note messages. If APC MINI supports this, maybe it would be more reliable, than sending multiple note messages. I’m not sure if APC MINI supports this however.

Steve

Hi Steve,
For solution 1, I can’t think of a way to tell when the APC would be ready to accept more MIDI messages. Are you thinking of using a timer (as shown below)? Else how would I know when it’s safe to send data to the APC?

(ga=0 when MT is launched)
In Translator 1) Start a timer in response to the a fader movement.
In Translator 2) If ga=0 Send an LED message back to APC, but block output if ga=1.
In Translator 2) Set ga=1 if ga=0 on entering translator.
When timer elapses reset the global (ga=0)

When the timer expires, reset the global to 0 – at which point, Translator 2 allows output.

I think the above would work, but I have a concern. As the fader moves through its entire range, it starts the timer for each increment of the fader… so 127 times. To be effective, the timer probably needs to have a period of at least 100ms. Wouldn’t that cause a major bottleneck?… 127 timers all running at the same time – or maybe many hundreds of timers when I move several faders at the same time. Or does the same timer merely get reset each time it is started if it’s already running?

As for using the bit mapping to represent more than one LED, I’ve looked into that, and, in fact I began to do that a while back, but it would have required a lot of re-writing, and I’m loathe to do it. Instead of doing it, I chose to use a 2nd instance of MT to take on some of the load, (and provide extra variables). Using the 2nd MT seems not to be a problem, since I encounter the problem eve when running only one instance.

Disabling and reenabling presets whose function is to send MIDI to the APC is an interesting twist on the previous strategy. I’ll see if I can test it successfully, but wouldn’t it pose the same problem as strategy 1? How would I know when it’s safe to receive MIDI again so I can start transmitting data to the APC?

As for Sysex, AKAI is notorious for not being helpful to people seeking more esoteric info, and there’s nothing in the tiny manual that defines sysex for the Mini.
Thanks,
Gabriel

Hi, your timer strategy looks good. You wouldn’t launch 127 timers, it would just be the same timer restarting.
I would set it up so that any MIDI output sets the variable to 1 and then launches the timer.. You could use
raw incoming message of pp qq rr and then not use them in the rules. That would cover any 3 byte incoming message from the APC MINI. I suspect a setting of 10msec would give it enough time to get ready for the next incoming message. Keep in mind you may lose the message coming from Ableton Live if it reacts quickly but if you are controlling the LED’s yourself, you should be able to manage the return traffic in MT Pro. If you need reliable messages back from Ableton Live while timer is in place, we would need some sort queuing system which of course would require more global variables. Of course Ableton would never send back fader data so maybe best to limit return input only when moving faders and not pushing buttons.

As far as preset enabling, disabling. You would still use a timer but the you would disable the preset in one translator and start a timer in the other using the same incoming MIDI trigger, When the timer completes, it would then re-enable the preset.

Too bad about AKAI not providing that type of input. Many manufactures seem to have hidden features that would be very valuable if they documented them. Yesterday I found out quite by accident how to turn on LED’s on my Samson Conspiracy controller. I found this when testing whether it would crash like the APC MINI.

Yes, bit mapping although effective can be a bear to set up and sometimes I even go back and try to figure out how I did things. Replacing single digit in a variable is easy if you are taking it from 0 to 1. It is a simple OR operation.
However to take it from 1 to 10 you have to XOR the initial variable, then AND the bit and then XOR the original value (if I remember), to get it back with the change. Also sometimes remember which bits are used for what can be challenging especially if you are dealing with multiple bits for an LED (IE 3 bits for RGB).

Take care.

Steve

timers can be launched as often as you like, they’re designed to work that way.

Why don’t you throttle the data sent to the AKAI?
1) use a global variable (I know you’re short on them) to store the value of ss
2) instead of sending the long string, start timer “LED Update”, one-shot, delay 20ms
3) in Incoming Timer “LED Update” translator, send the long string with the global variable instead of ss

Like that, the LED update will only be sent 30ms after the fader stops moving. It’ll also update if the fader is moved so slowly that it fires less than 30ms in succession.

If you want the LED’s to continuously update even while you’re quickly moving faders, there’s a solution for that, too. But the above is more elegant.

Hi Florian. I didn’t suggest this due to his lack for global variables. Also Ableton may send either short or long MIDI messages in bursts so we never know what pattern to look for unless it is just a 3 byte message but if they send multiple 3 byte messages in bursts, you would need variables for them all if you have to queue them all.
I suppose if I got really agressive, I could look at the Ableton Python script itself so I could either determine a pattern of what it will send or modify the script to something that is more predicable by MT Pro. I could probably even put in a delay for returning data to the controller every time Ableton got a MIDI message and queue up the messages there in the Python script. I’m not up to the challenge of this complexity, however at this time.
Steve

Steve and Florian,
Thanks to both of you for all the attention you’re giving to this issue.

Live doesn’t play a role in the problem. The error occurs even when Live isn’t running. Beyond that, when I do have Live running, MT sits between Live and all my controllers, so any message from Live is analyzed by a translator in MT, and only messages I want to use make it past that translator. (The translator is called “Control Track Enable with Feedback from Ableton (to eliminate early audition of masked clips”, and is near the end of my project). I was able to decipher the messages Live was sending by using Midiox to monitor them.

The challenge I see is to determine when the APC Mini can accept MIDI without crashing (presumably when the APC faders are not moving). I think you are both suggesting similar strategies using a timer – either to block input from the APC or to block output to the APC once MT receives a fader movement. And since I now know that the timer would be restarted rather than to start a whole slew of timers, I feel comfortable taking this approach.
I do something similar in order to throttle MIDI sends to MT instance 2. I should have known the timer was being restarted because I can see that timer being sent out many many times, but only being acted upon once.

It doesn’t matter if the LED states are updated with significant delay. I use the LEDs only once in a while, when I want to see which clips are playing.

I believe I may have freed up a few globals in recent tinkering with the code, and if I still don’t have enough, I should be able to offload some more routines to MT instance 2, thereby freeing up enough globals.

I’ll leave this thread as not resolved so far and update as soon as I’ve proven that we have a solution.
I have a good feeling about this now.
Thanks,
Gabriel

Great, keep us posted. We are here to help if needed.

Testing is going well so far.

I’ve limited the testing to two APC Mini’s. In MT Pro, I’ve disabled any translators that output to LEDs in response to fader movement. Live is also running. Button presses do still output to LEDs.

Among other actions, a button press may send a MIDI string that turns off 8 LEDs and turns one of them back on…. so 9 commands sent as a single output from MT. Something new I’ve learned is that I can do this while scrubbing several faders vigorously.

Because I can do that, I believe the APC only hangs when it receives MIDI while a fader is moving. Sending and receiving at the same time seems to be ok as long as the sending is being done by pressing buttons. I’ve “squashed down” on as many buttons simultaneously as I can and at the same time scrubbed as many faders as I can grab onto without causing a hang.

Before I did this experiment, I tried the strategy of using a timer to disable MIDI outputs to the APC if a fader was moving. I think I wasn’t careful about making sure every potential output situation had been treated because I could still cause the hang, but I’m going about things much more methodically now.

My next step will be to enable – one at a time – MIDI outs to the APC’s but include the throttling we discussed.

I’ll let you know how things go.

Gabriel

Hi, maybe we should also open up a ticket with Akai Pro about this problem? Since both of us can duplicate it, apparently it is not a problem to a specific controller but to all APC minis. Maybe they can come out with a firmware fix?

I called AKAI today, trying to get support for this issue, but it was like crashing into a brick wall. I was told that the Mini was developed to interface directly with Live, and that if there was 3rd party software in between the two, they wouldn’t consider any firmware changes. They also said that they didn’t have any information about sysex for the Mini.

This was their support line, not a direct connection with the developers of he Mini, but they told me there was no way to get in contact with those developers. I’ve had a similar interaction with them in the past about a different issue with the Mini.

Then I tried Ableton support, where, at least, I had a sympathetic ear. I asked if they might have sysex info for the Mini, since they were involved in its development. Sympathy notwithstanding, they were unable to give me that information. Not really sure if they didn’t have it or were under obligation not to divulge it. Of course, again, I was dealing with someone quite far from the engineers involved.

So, I’ll try, now, to re-instate LED feedback, a step at a time, to see if I can isolate the cause of the problem more specifically, and then build in some kind of throttling or a timing change, in order to allow me to control the LEDs without hanging the Mini.

Since it doesn’t matter if it takes a few seconds to get the visual feedback I need, it might work to delay the output to the Mini (though there’s always a chance that I’d resume moving a fader just at the time that I’m updating the LEDs).

Or maybe just sending the LED messages one LED at a time rather than all 9 in the output of one translator as a single string would do the trick. I’ll give it a try also.
Thanks for all your help.
Gabriel

I also opened a support ticket with Akaipro but used their support forum and pointed them to this post for details.

Hi Steve and Florian,

My second try at solving the Error 15 issue seems to be very promising. It’s very much along the lines that both of you have been describing.

Originally, if I scrubbed a fader quickly, and update the LEDs as shown below the APC eventually freezes.

  • MT Pro Input from AKAI APC track volume fader: B1 pp qq
  • Response from MT Pro (output to Live track volume fader): B1 pp qq
  • and
  • Response from MT Pro (output to APC) for updating LEDs:
  • — 90 52 00 90 53 00 90 54 00 90 55 00 90 56 00 90 57 00 90 58 00 90 59 00 90 ss 01
  • — Where ss is the LED I want to turn on. All LEDs are turned off before turning ss on

Not so elegant, I know, but I didn\'t have enough globals to remember LED positions for all 20 tracks and didn\'t have the appetite for trying a bit mapped solution.

I think the problem may stem both from the fact that messages are sent back to the APC repeatedly during scrubbing (127 times for each stroke up and another 127 times for each stroke down). as well as from our original conjecture - that it might be due to the fact that the APC fader is moving while it is receiving LED related MIDI.

The solution seems to be:

  • MT Pro Input from AKAI APC track volume fader: B1 pp qq
  • Response from MT Pro (output to Live track volume fader): B1 pp qq
  • and
  • Response from MT Pro: Timer 1 OUT (initial delay 20ms)
  • Response when Timer 1 elapses (out to APC)
  • — 90 52 00 90 53 00 90 54 00 90 55 00 90 56 00 90 57 00 90 58 00 90 59 00 90 ss 01
  • — Where ss is the LED I want to turn on. All LEDs are turned off before turning ss on

The timer only produces output to the LEDs once (if the fader is moved fairly quickly) or just a few times for a typical, slower, movement of the fader. I think both of you had pointed out to me that the MIDI stream would be dramatically reduced… and it is.

The same fader movement produces several other outputs, some of which sent LED messages to the APC’s. I’ve taken steps to throttle the MIDI flow there too.

Anyway, I’ve rewritten the code for one of the APCs and it seems to be preventing freezing. I’ll test more extensively tomorrow, and if it really does work, I’ll rewrite for all 8 APCs and let you know how it goes.

Thanks,

Gabriel

crossing my fingers!

Funny…. I almost used that exact phrase in my posting!

Thanks for doing that. It would be great if that approach bears fruit.

GOT IT!!!!

Thanks for all the help.

The problem seems to have been that the APC Mini can’t handle MIDI data updating its LEDs if it comes too quickly. I was trying to send that data to the Mini at speeds that could exceed 40,000 Bytes per second if I grabbed 7 or 8 faders and quickly slid them down to attenuate volumes or effects. This easily over-ran the MIDI top speed of 3125 Bytes per second), so I presume that the Mini was receiving data at about that top speed.

There seems to be no provision in the APC for that situation, probably because it would never occur if used directly with Ableton Live (its intended use). Live couldn’t spit out that much MIDI data by itself. So I guess is that the input buffer provided in the Mini is tiny.

The solution was to stagger the feedback to the APC Mini’s. The speed at which the LEDs update is not an issue. LED state just told me what clip was playing or what VST effect was active. So… I now use a timer for each of the 72 faders on the 8 APC Mini’s. If a fader is moved, it starts the timer, which has a period of 30ms. When the timer elapses it triggers the LED output messages going to the Mini’s. The data is sent in bursts of about 10 instructions each time the timer elapses (so 30 bytes). Additionally, the timer resets itself constantly throughout the motion of a fader because the fader output is happens more frequently than every 30ms. So the APC actually might not receive any LED messages until the faders reach their final destination… and then only one burst of 10 messages.

Meanwhile, the MIDI messages going to Live (and not to the APCs) is sent without this kind of gating, so musicality is not effected by delays in reaction.

I also eliminated some of the LED messages upon rethinking my strategy for LED updates. The number of messages per increment of the fader movement was originally 18. Now it’s 10, with no real loss in functionality.

I’m so relieved. And my trust in MT Pro has once again been bolstered. What a great tool, and what amazing support. I can finally turn my attention to playing music. It’s been a long road to developing my system, but it wouldn’t have been possible without MT. And that’s not an exaggeration.

Thanks,

Gabriel

Hi Gabriel,

I’m so glad you were able to workaround the problem with the APC MINI. You have quite a large project. I don’t think I’ve ever seen one so large and complex. I’m glad the Bome and MIDIBuddy software could help. I agree, Bome MT Pro is the greatest. Can’t wait to see version 2 when it comes out. Hopefully, me helping out here will help Florian focus more on new product development. I’m so happy to be here!

Take care,

Steve

Hi Steve,
LOL… Probably the reason that you’ve never seen such a large, complex project is that anyone crazy enough to create it would probably have a lot more programming chops than I do – so they wouldn’t be asking for help on the forum.
Everything still seems to be running reliably now … one day later. So, thanks again for the advice.
BTW, MIDIBuddy is certainly the icing on the cake. I use it mainly to link a trackball to the Beat Repeat effect in Live.
Bye for now,
Gabriel

Hi Gabriel,
very impressive mapping indeed.
I’ve added this post to the advanced mappings index: