Mouse Move with MIDI Notes

Bome Products
,
1

Hi Steve! Thanks for the quick response and for the test patch!

I think you understood me backwards — the patch you made actually emulates the exact problem I’m trying to solve! When I try to play certain games with my aerophone that require mouse movement to control the in-game camera, the cursor will get stuck on the edge of the screen much like the patch you designed — this prevents the in-game camera from moving in that direction any further. The reason is because the game is looking for mouse motion information specifically, not mouse cursor position. Moving the cursor via MT Pro does provide that motion information right up until the cursor hits the edge of the screen.

Re-centering the cursor on the screen is a no-go as well — that would center the camera in game as well, which would defeat the purpose.

I would understand also if this simply isn’t something that MT Pro can do at the moment — just trying to see what my options are! Please let me know if I need to better explain anything, and thanks for all of your help.

2

Ah, in that case, you might need to have a different outgoing action to move your camera than mouse movement. We are indeed sending movement (not absolute) outgoing actions but maybe they are not being interpreted by the game if you go out of bounds.

Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
3

Just wondering. When you get to the end of one side, can you do a jump (direct position) to the other side of the screen so that you can continue on with mouse movement actions.

So when it gets to the the right boundary:

  1. Jump to the left boundary
  2. Continue moving the mouse right.

etc.

4

Hello,

I stumbled upon this thread and I’m having this same exact issue, except I’m trying to use a piano as my controller to play games with.

I’ve been trying to find a way to key bind the in game camera to move in one direction continuously when I press a piano note. If I just use the mouse movement option that’s translated in pixels, let’s say 50 pixels up, it only jumps up 50 pixels instantaneously AND only once. I would have to keep spamming the piano note to continuously move 50 pixels up (and even that sometimes doesn’t always work.)

Is there any way for me to set up my piano so that it can continuously move the in-game camera in the direction I choose? Not mouse position, but mouse movement delta (change in position).

5

Hi, I’m moving this to a new topic because it has nothing to do with a wind instrument.

Please find the attached project file which moves the mouse as you play notes.

Translator 0.0 handles starting a timer. We use the global variable gb to determine the direction of the mouse based on the incoming note number.
0-Up
1-Down
2-Left
3-Right
We use the variable ga to flag that the timer is running

Translator 0.1 stops the timer and sets ga to 0 thereby stopping mouse movement

Translator 0.2 uses the global variable sets the global variables gx and gy based on the movement. There are a few rules that perform other actions. We set the timer delay to determine how often it fires, and we set a multiplier for the precision in pixels. We use the global variable zz to put it in debug mode (Log messages sent but no outgoing action if zz=1) otherwise we move the mouse.

Here are the rules of translator 0.2

if ga==0 then exit rules, skip Outgoing Action

// delay
ww=50
// precision
rr=10

if gb==0 then tt=rr*-1
if gb==1 then tt=rr
if gb==2 then qq=rr*-1
if gb==3 then qq=rr

//debug flag Log but don't execute
zz=0

if zz==0 then Goto "Done"

if gb==0 then Log "Up %tt%"
if gb==1 then Log "Down %tt%"
if gb==2 then Log "Left %qq%"
if gb==3 then Log "Right %qq%"
exit rules, skip Outgoing Action

Label "Done"

And the project file.
mouse-move-with-notes.bmtp (1.9 KB)

Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
6

Hello!

As helpful as Steve has been trying to help solve this issue, I think there is a fundamental misunderstanding of what we are trying to do. I have done more research into this and Bome MIDI Translator is unable to perform what we are looking for at this time.

There is one bit of software that DOES in fact translate MIDI-to-Mouse Delta information (Universal Control Remapper - GitHub - Snoothy/UCR: Universal Control Remapper [Alpha]). It can also translate MIDI directly to Xinput/DS4 Input, which is something Bome cannot do. That said, it is still in Alpha and the GUI leaves much to be desired, but it might help you out with what you need.

7

The project file that I posted didn’t work for you? I tested it and it worked for me although I certainly was not in a game. Just moving the mouse on my screen? Did the mouse move? Did you try notes 0-3? If it didn’t work, maybe you can turn on the log window with MIDI in and outgoing checked and share it with me.

Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
8

I think I see. In your case , there is no cursor. You are not trying to move the mouse cursor location. You are trying to send low level mouse motion commands which is quite different. You may be able to do this on Windows with AutoHotkey. I think these are lower level HID commands that monitor scan codes directly as opposed to mouse cursor movement commands.

Steve

9

That is correct. Once we click inside the game, the mouse cursor disappears, and our mouse movement then controls the in-game camera.

What I have attempted to do was to have the outgoing source mouse movement set to, for example, 50 pixels / -50 pixels for each different direction. Sometimes it works, sometimes it doesn’t, but when it does work it’s unfortunately too inefficient to have to spam a key to constantly move 50 pixels up or down, etc. when what we want is for the in-game camera to move smoothly in the direction we press.

As Dr. Doot had mentioned, it seems that Bome MIDI Translator cannot perform what we are looking for at the moment. Hopefully this is something that can be included in a future update! More and more people are using various MIDI controllers to play videogames with for streaming/content creation.

Thank you, Steve, for always responding quickly and for helping out as best as you can! I will take a look into AutoHotkey, as well as the software Dr.Doot shared.

10

Can you make the camera move with any keystrokes?

12

Could you tell me a game that you use that we could test with?
Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
13

For a lot of games, not really. Some games may have the option to change your camera movement bindings from mouse movement to arrow keys, for example, within the game itself. In those cases, then I can use BMT to do its thing with keystrokes for the arrow keys.

Not every game has the option to rebind the camera movement though, forcing you to use mouse movement. And since mouse movement isn’t a keystroke, I will try to utilize the Mouse Movement option under Outgoing, where you have to type in how many pixels you want the in-game camera to move, like 50 pixels for example. But that means moving the camera around 50 pixels at a time, and having to press the button a bunch of times to move the camera. And actually for some reason, the camera doesn’t always move when I press it either.

The most recent game I tried doing this with was a game called Palworld, which just released on Steam this past friday.

14

If it’s okay to post links to clips, I can show you two examples, one where the camera works the way I want it to and one where it doesn’t.

In this first clip, the game allows me to change my keybindings of the camera. You’ll notice how the camera moves smoothly whenever I press and hold the keys, specifically the notes C4, Eb4, E4 and G4 (for left, up, down and right respectively). This is from a game called ‘Elden Ring’:

This second clip is from a game called ‘Only Up!’ Here you will see that I press the G4 key to move my camera to the right. When it does move, it jumps to the right by 50 pixels (or whatever I had it set to at the time, I don’t remember off hand) but then sometimes pressing the G4 key doesn’t even move the camera at all, even though I was pressing it repeatedly. You’ll see that after I fall, my hand goes to my mouse to move the camera smoothly to see where I was going to be falling to, since the piano keybinding was not really working at that moment.

16

Yes, it must be a low level driver the monitors the HID codes and doesn’t use the standard mouse interface.
I tried with the Steam game ‘Raft’ and got nothing at all. It would recognize my real mouse but neither the Bome MIDI Translator mouse movement or my network attached mouse.

We will check into it further.

17

Thank you for taking the time to look into it! I hope we can find a solution!

18

This may take a while to get worked out with Bome MIDI Translator Pro. I would suggest looking into a small hardware programmable macro keyboard that can emulate a mouse.

Be careful as not all programmable macro keyboards can also emulate a mouse. Many can just handle keystrokes, so you might need to shop around. Even at that, it would not convert MIDI notes to mouse movements as these keyboards don’t take MIDI input. This is a pretty narrow market so not sure what is out there. Also, some gaming software companies don’t like this as it may give some gamers unfair advantage of the mortal users that use mice.

Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
1 Like
19

Hey Elementality! Fancy seeing you here!

I might have a solution for you here, but you might not like it — since Only Up! is a game running the Unreal Engine, you should actually be able to edit the initial configuration file for the game so that you can manually map the camera movement in game to specific keyboard strokes. Yep, that does mean you’re basically learning how to write a mod for the game — this is what I had to learn how to do to get Lies of P working with Bome MIDI Translator, which is the game that started this whole thread in the first place.

It’s a bit complicated and off-topic to explain here, but if you wanna talk about it, just hit me up.

  • Dr. D
1 Like
20

Hey Steve! Just wanted to follow up and see if there are any updates or progress made on this issue!

21

Not that I’m aware of.

Steve Caldwell
Bome Customer Care

Also available for paid consulting services: bome@sniz.biz
25

Hi everyone, I’m new here, I speak Portuguese and I’m using a translator, so please excuse me if I say or misunderstand something.

About using Bome in games, I’ve managed to make an accelerator for my expression pedal.

First I tried the approach of increments with mouse movements, but I had the problem of positioning on the screen, like my friends did… but it would be the best approach because of the sensitivity that the mouse can make the Game Read.

I got a satisfactory result using MouseWell, where I programmed the repetition speed inversely proportional to the velocity of the midi input, that is, the higher vv the lower tt, I’m also regulating the delta of the increment based on vv… the goal is to make the cart move slowly and steadily when vv is low and increasing, always with constant inputs, as vv increases (simulating the movement of a game controller’s RT button).

I’m making fine adjustments to mitigate the punches (I don’t know how to express this in English, lol, but I’ll try)… when it’s very slow you can notice the input lags… the inputs, it’s not very smooth, but it’s satisfactory for now.

With the mouse movement I didn’t do the interactions with tt, I left the repetitions fixed and the increment only in vv. While it’s on the screen the result was wonderful, even more so when you program vv*vv. or vv^vv and divide afterwards (to simulate a logarithmic potentiometer input).

I use 5 monitors, at first I thought that the game only captured the inputs while the mouse was on the game screen, but I believe it’s because of the entire screen, since my screen is quite large I was able to visualize that it would work well… I’m thinking here about automating the positioning on the screen based on the increments, but I don’t know yet how to do this ‘sum’, and I also don’t know if the game will understand this positioning movement as the same mouse movement that I’m using to accelerate the cart.

SteveC, it’s great to see your explanations. Although I don’t understand much yet, I can see that I can do a lot of things. I’ll answer another topic later about my controller, LEDs, Raw, Mackie, etc… to ask for help.

PS. I didn’t put my codes here because I’m still embarrassed lol, they’re not as elegant as yours.