Feature Request - QWERTY “musical typing” keyboard support

[samuelfine]

Description

I often use AudioKit apps on my iPad with my Magic Keyboard connected, and would love to be able to use those keys as a pseudo-keyboard instead of requiring touch input. Given the proliferation of Bluetooth keyboards and iPadOS support for external keyboards, I can imagine other use-cases where a MIDI controller might not be feasible, but a QWERTY keyboard is more readily available.

Proposed Solution

Many existing DAWs offer “musical typing” interfaces that offer tried and true solutions for this concept. Logic and GarageBand, for example:

I would be delighted with something even as straightforward as (1) mapping the A through ’ row to white notes, with the appropriate keys from the Q through \ row as black notes, and (2) using Z and X to raise or lower the root octave. Existing text labeling offered by this library could offer a simple way to display which physical key is mapped to an on-screen key.

Given the wide variety of keyboards offered by this library, I can imagine there would be many edge-cases to consider. For example, it may be confusing to a user if an on-screen keyboard has more keys than can be represented by a row of physical keys. (One possible UX solution: “centering” the QWERTY key mapping if there are more keys on-screen than physical keys.) Some of the less traditional keyboards (guitar, vertical) might not make sense with a QWERTY mapping, though this could be left up to individual developers. If the user can always expect A through ’ to represent C[rootOctave] through F[rootOctave+1], perhaps there doesn’t need to be a 1:1 representation?

Describe Alternatives You've Considered

I do own many dedicated MIDI controllers, and those work wonderfully when I’m at a desk or have a larger work area. For more casual exploration, however, these aren’t as useful.

I’ve also considered developing my own app, based on this library, to translate keyboard input to MIDI note out. I’ve decided against this for the time being, for two reasons. One, I’m not a Swift developer. :) But, more importantly, I believe this functionality might be of use to many/most apps that utilize this keyboard library, so rather than requiring an additional app - AUv3 or otherwise - to translate from QWERTY to MIDI, I would rather see AudioKit (and third-party!) apps be able to take advantage of the functionality “out-of-the-box”.

Additional Context

I am quite aware of the significant effort required for even small features like this request. If there is a way I could sponsor this functionality directly, please let me know!

[aure]

The last I checked the magic keyboard was only registering keypresses, not key releases, which made this less useful, but I'll have another look.

[samuelfine]

Thanks for the speedy reply! Looks like as of 2020, this restriction may have been lifted: https://forum.audiob.us/discussion/38759/would-it-be-possible-to-use-the-new-ipad-magic-keyboard-as-a-midi-controller

“It’s an iOS limit. And one which was lifted in iOS 13.4. Now, both key down and key up events are available to developers.”

For what it’s worth, I’d be thrilled to see generic QWERTY keyboard functionality, regardless of specific support for Magic Keyboard. I often use my iPad and AudioKit apps as an audio output, sending to a DAW on my macOS desktop; Universal Control (https://support.apple.com/guide/ipad/a-keyboard-mouse-trackpad-mac-ipad-ipad3924226c/ipados) allows me to share my desktop’s keyboard and mouse with my iPad wirelessly, so this sort of functionality would also be useful in a situation like that.