I’m wondering or it’s possible to redirect an incoming MIDI message to commit a mouse click at specific coordinate on the screen.
I’m trying to work around to control certain on-screen UI elements which has no MIDI control capability. There are 2 things I would like to accomplish:
Mouse click on an UI element on the screen (on/off) or coordinates relative to within the target application window.
Like previous, but now also to control a slider UI element on screen (value 0-127) and map it to a knob
Bome MIDI translator allows for mouse actions at specific screen coordinates, however you will need to tell it the position of the UI elements for the most part. If you are lucky and are:
running n Windows platform and
the application individually exposes the UI elements. (Most Java Apps do not)
Then you could use injected outgoing actions to access the elements without knowing their screen location.
This tutorial, shows how I use injected mouse action with Corel Video Studio.
Usually sliders are a little more difficult because the position of the slider moves around, however if your application can move sliders with a mouse scroll wheel, sometimes getting close enough to the UI slider element is good enough to make it work. If you can make the knob a relative encoder type, it is usually even better as we do not need to worry about the current position of the knob.
What platform (Windows or Mac) are you running on and what is the application?
Steve Caldwell
Bome Customer Care
Also available for paid consulting services: bome@sniz.biz
I can’t believe I never noticed in the outgoing action there was this nice little option hidden in a dropdown box called “mouse”. I selected it and found a candy store worthy of options to pick from (except it’s display based only, no option for application window name based). But this is fine for now.
The mousewheel feature comes in handy: I can make a translation to move the cursor into place, add a 2nd translation that takes input from an endless encoder on my Xone K2 and translate into mousewheel scrolling at that coordinate as the application accepts that as input to control the UI. Endless encoder also solves the problem of positioning.
I think I got that part working now. Now off to solve other 999 problems.
I’m glad I could help. I’m not sure if RekordBox handles injected actions or not. If it does, then you will not need to worry about the application window. Maybe I’ll give it a try if I get a chance.
Steve Caldwell
Bome Customer Care
Also available for paid consulting services: bome@sniz.biz
I have it doing it what I want for now. But now I’m having a little issue with the housekeeping logic inside the project: xonek2-rb6.zip (11.7 KB)
It’s in the preset: controller.layer.green.btn70.hold (also the mousewheel scroll thing)
which is triggered by translation “press btn70 → activate layer.green.btn70.hold” and “release btn70 → deactivate layer.green.btn70.hold” inside preset: controller.layer.green
I’m experimenting with the logic on detecting button press & hold, and button release with the “controller.layer.*.btn??.hold” presets.
The CSV file is the MIDI mapping data for RekordBox. Unfortunately MIDI mapping in RB is hidden behind a paywall (subscription or hardware unlock).
It appears that you might want to de-activate controller.layer.green.btn71.hold when you release button 71. I don’t see where you actually disable it. Maybe timer x1 is supposed to do that. I’m not sure I can’t debug your entire project file so maybe tell me what you are getting that doesn’t look right (show the log file) and the way, you want it to behave.
Steve Caldwell
Bome Customer Care
Also available for paid consulting services: bome@sniz.biz
Ok, after a thorough inspection I found there were plenty of flaws in my initial design implementation. The way I did some things was because I was making presets and translations listening on all devices. Which was causing loopback problems, forcing me into some unorthodox solutions.
I now reassigned the translations to be listening on the physical device or the virtual device (signals from the application) as it should be. This solved a dozen problems I was facing before.