Implementing a virtual keyboard in Elm

[cbrake]

How practical is it to implement a virtual keyboard for an embedded (special purpose) touchscreen device in Elm (ideally elm-ui)? Often this keyboard is integrated into the browser or platform, but for some embedded devices, it would be nice to bypass all this and do it all in Elm. Then from the platform, all you need is a single browser running fullscreen which can eliminate a lot of platform complexity.

Some questions come to mind:

• showing/hiding the keyboard when clicking on an editable text field
• not sure how this would work with scrolling. When the keyboard is implemented by the device platform (say a phone), then the browser window resizes. If the keyboard is inside the browser window, not sure how this would work – could it be a popup below the current field?
• can the keys be injected into the browser, or would they need to be handled in Elm?
• accessibility – these products are not general purpose (like a phone), so likely won’t have built-in screen readers, etc, but there may be other issues.

Just getting started, so appreciate any thoughts – especially if there are reasons this is not a good idea.

Again, the motivation is that implementing/customizing this stuff at the platform level can be fairly painful, so if we are doing a ground-up Elm app, is that a better and more flexible option? I looked around in elm packages a little and did not see anything existing.

[lydell]

I built an onscreen keyboard for a “kiosk” in a store like 5 years ago. The kiosk was running a browser in fullscreen. The site used jQuery and React. Unfortunately the kiosk thing is not around anymore.

I listened for focus events on the whole document. When a text field was focused, I showed the keyboard. When it was blurred, I removed the keyboard.

The keyboard was fixed to the bottom of the screen, but I made sure to add space for it at the bottom when open so it was possible to type in fields very close to the bottom of the page.

When opening it, I think I had to do some DOM measurements to make sure the scroll of the page was alright. For example, the keyboard must not cover the input you’re trying to type into. I also animated the keyboard sliding in, so it felt more like a typical smartphone keyboard.

I don’t remember the details on what happened when you pressed a key in the keyboard. Maybe I just did input.value += "A" or whatever, but I guess I have to trigger some keyboard events as well? I emulated backspace as well. Don’t remember if there were arrow keys.

The keyboard view was the thing done in React. Was easier to render that way, with all states like uppercasing letters on the keys when shift was locked (tap to shift, not hold to shift).

This site was a regular web site that we added a kiosk mode to. So the virtual keyboard had to work with an already existing site, with a bunch on inputs in more places than we could count. The good thing about this approach was that the keyboard was pretty decoupled from the rest of the app but still worked nicely thanks to the semantics of <input> elements. I haven’t thought about the “Elm way” of doing this, but it feels like the same approach could work well with Elm too: Do some pretty general, side-effect:y JavaScript for events, hiding and showing, and manipulating inputs, but render the keyboard and handle its internal state in Elm.

I also remember looking at some off-the-shelf JavaScript packages for a generic virtual keyboard. In our case they didn’t work (don’t remember why), but they were good inspiration.

[grigorious]

I also implemented one years ago, we had to cover multiple layouts ANSI, ISO de / fr / uk / cro … I remember putting the keys to be rendered in arrays for each row of the keyboard - so a user could switch if he/she wanted to.