How to customize the keyboard layout on LineageOS 18.1?

[VaZso 1,998]

1 hour ago, Sean McCreary said:

Now, it is certainly possible that this clever feature causes some problems with custom key mappings. @VaZso, could you post the custom key map that didn't work properly so we can do some debugging?

Nothing really special, it tries to use fn + mostly non-alphanumeric characters, so USA layout of [];'\/-= are all need to be used together with a modifier.

The solution in keyboard driver uses another layer of emulated keypresses with the help of an array containing these keys.
(So it can also say you pressed ctrl+c when fn+k was pressed and alt+j when fn+a was pressed or even ctrl+alt+z when you press fn+r where fn is the slant arrow.)
This array was filled with really mixed AltGr, Shift or none additions of the base keypresses.

As there are no modifiers applied or if there is, it has a really mixed AltGr/Shift combinations, that makes impossible to use it together with custom key mappings.

It would need a separate key mapping which is consequent to use FN, AltGr, Meta or anything but added to all characters.

That is why I have started my own modification...

Edit:
Custom keymap (if you are speaking about keymap input of kernel module) works because it allows overwriting the main character tables.

Custom keyboard mapping by .kcm does not work without adding a custom keymap into the kernel module and somehow the system does not pick up the keymap from file either.

Edited August 29, 2021 by VaZso

[Sean McCreary 328]

8 minutes ago, VaZso said:

Nothing really special, it tries to use fn + mostly non-alphanumeric characters, so USA layout of [];'\/-= are all need to be used together with a modifier.

The solution in keyboard driver uses another layer of emulated keypresses with the help of an array containing these keys.
(So it can also say you pressed ctrl+c when fn+k was pressed and alt+j when fn+a was pressed or even ctrl+alt+z when you press fn+r where fn is the slant arrow.)
This array was filled with really mixed AltGr, Shift or none additions of the base keypresses.

As there are no modifiers applied or if there is, it has a really mixed AltGr/Shift combinations, that makes impossible to use it together with custom key mappings.

It would need a separate key mapping which is consequent to use FN, AltGr, Meta or anything but added to all characters.

That is why I have started my own modification...

We can't really debug the problems without a test case that demonstrates them. However, it seems like the current custom keymap ought to support the uses you describe. Specifically, you can combine keycodes with any of the flag bits to make the driver issue whatever modifiers you need:

On 2/25/2020 at 1:32 PM, tdm said:

[Y]ou need to know the physical key numbers and the Linux key codes.  Reference the keyboard driver here for the physical key numbers and the Linux header here for the key codes.  You remap keys by writing to the file "/sys/bus/i2c/drivers/aw9523b/6-0058/keymap" in a format that I mentioned earlier: "key:code:fn-code", where "key" is the physical key number in decimal, "code" is the regular Linux key code in hex, and "fn-code" is the slant-arrow Linux key code logically OR-ed with any modifiers (0x8000 for shift, 0x4000 for ctrl, 0x2000 for alt).  So let's make KEY_FN+KEY_1 send KEY_F1.  The "1" key is physical key 57, KEY_1 is 2 (0x02), and KEY_F1 is 59 (0x3b).  Note you will need to be root to do this (adb root, adb shell):

 

echo "57:02:3b" > "/sys/bus/i2c/drivers/aw9523b/6-0058/keymap"

 

To send a shifted F1, you would use "57:02:803b".

[Note the location for the custom map file has changed to '/data/system/keyboard/keymap' since Feb 2020.]

I haven't tested whether the driver will correctly handle multiple flag bits set (e.g. for CTRL+ALT+Z), but we should be able to make that work too.

[VaZso 1,998]

4 minutes ago, Sean McCreary said:

I haven't tested whether the driver will correctly handle multiple flag bits set (e.g. for CTRL+ALT+Z), but we should be able to make that work too.

As far as I can see, it supports multiple flags (but currently FN is not among them) and somehow it does not automatically load custom keymap, it can only loaded through the kernel module interface.

Also, it needs root access.

[Sean McCreary 328]

Just now, VaZso said:

As far as I can see, it supports multiple flags (but currently FN is not among them) and somehow it does not automatically load custom keymap, it can only loaded through the kernel module interface.

Also, it needs root access.

Yes, I see KF_FN is defined as 0x0400 but there is no code to send KEY_FN based on g_logical_modifiers. BTW, keycode 464 isn't handled properly by input_report_key(), so we would probably have to use a keycode < 255 and remap it in the Android .kl file to make that work.

The settings app should gain a new 'toggle' setting when a file is placed at '/data/system/keyboard/keymap'. You then need to enable the custom keymap in settings for it to be loaded into the kernel.

LineageOS provides root access via 'adb root; adb shell', or with the optional 'su' add-on package.

[Sean McCreary 328]

1 hour ago, Sean McCreary said:

The settings app should gain a new 'toggle' setting when a file is placed at '/data/system/keyboard/keymap'. You then need to enable the custom keymap in settings for it to be loaded into the kernel.

There does seem to be a bug in the settings app that is preventing this from working 😞 I will try to figure out why.

[EskeRahn 5,460]

5 hours ago, Sean McCreary said:

...or with the optional 'su' add-on package.

I know that the 'addonsu' works with 16.0, is there also one for 18.1 ?

[EskeRahn 5,460]

On 8/24/2021 at 5:27 PM, Slion said:

However some of it gave me an idea.
It looks like ATM we have like two profiles QWERTY and QWERTZ.
I wonder if we could modify the driver to add more profiles QWERTY and QWERTZ variant that would be compatibly with stock so that FinQwerty and other mods would work the same. I would probably also add another profile that's improved on stock.

(Very late to last weeks many post in this thread)

I think this looks like a PERFECT solution with two optimized build in layout-interpretation, but ALSO to have one additional mirroring stock. I bet it will be very little overhead code, but will be very helpful for various international users to be able to benefit from the work of @Anssi Hannula and others without having this to be redone in LOS-variants.

[Slion 1,201]

5 hours ago, Sean McCreary said:

There does seem to be a bug in the settings app that is preventing this from working 😞 I will try to figure out why.

Yes as it is you need to overwrite the Kernel copy of the keymap for it to work. That would be awesome if you could fix that.

[EskeRahn 5,460]

On 8/25/2021 at 10:24 AM, Slion said:

I have a feeling @VaZso is trying to get qwertz keymap to work with a qwerty keyboard and that somehow won't work as he would expect but that's actually not a bug. You are saying it works fine on your qwertz so I think we are good there.

I disagree, if QWERTZ print-layout only work some languages, it sure should be considered a bug.

The smart idea by @tdm to have a setting selecting the print-layout, independent of the language selection, is a MUCH smarter solution than what we got on stock. Look at all the 'doubles' @Anssi Hannula made to make things work, e.g. Danish for qwertY print and Danish for qwertZ print.
The idea by @tdm to convert the keyboard to a standard one, and then have standard android languages on top is clever, as it means that any language known by Android should work out of the box.
And the point of hardcoding some of these was to add missing physical keys as 'logical' ones.

Sure this sets some limitations. If the ? is generated by say Alt+something, then we could not do Alt+? , And THAT (I believe) was the reason for swapping Fn and † , as there are TWO independent †, but only one key with "Alt" print, so pressing both †+something would thus can give what Alt+? would.

And I really like the idea of adjusting the above to only catching these few specific key-combos, and otherwise leave the modifiers.

Optimally we would have a user friendly way with an app that allowed anyone to remap the keys as they please with some nice gui-interface. But if I understood @Anssi Hannula right, android does not offer 'soft-reload' of layouts, Only apk's offering layouts (Not a limit on rooted devices, with write access to the directories)

[EskeRahn 5,460]

On 8/25/2021 at 11:15 AM, Slion said:

Do you recall which keys are actually hardwired?

Shift and ctrl

[EskeRahn 5,460]

(Phew that took a while to get through all the posts the last week here 😅)

[Slion 1,201]

12 hours ago, Sean McCreary said:

This is normally because META+ALT acts as CAPSLOCK in Android, but does not turn on the LED embedded in the CAPSLOCK key. If your shift state is reversed, try pressing the 'F logo' and 'Alt' key next to it together.

Not sure that was my problem. Do we use META at all in the current keyboard? Can't reproduce it.

[EskeRahn 5,460]

13 minutes ago, Slion said:

Not sure that was my problem. Do we use META at all in the current keyboard? Can't reproduce it.

Try pressing the Logo+Alt , it will toggle the Caps mode

[Sean McCreary 328]

2 hours ago, Slion said:

Not sure that was my problem. Do we use META at all in the current keyboard? Can't reproduce it.

Android uses meta for shortcuts, e.g. meta+B to switch to the web browser, and meta+E to switch to the email app.

Also, META+ALT only works as CAPSLOCK in some contexts. For example, it will cause the shift state to be 'reversed' compared to the CAPSLOCK key LED in termux.

Edited August 29, 2021 by Sean McCreary
Add note about context for META+ALT

[Sean McCreary 328]

5 hours ago, EskeRahn said:

I know that the 'addonsu' works with 16.0, is there also one for 18.1 ?

Yes, you're right. The 'su' extra package was discontinued after LineageOS 16.0 😕 I don't use it, so I haven't been paying attention...

[claude0001 1,341]

1 hour ago, Sean McCreary said:

The 'su' extra package was discontinued after LineageOS 16.0

Sadly yes. As far as I know, that happened because changes in Android security mechanisms made porting AddonSU to LOS 17 non-trivial at best. One of the reasons I am camping on LOS 16 ...

[Sean McCreary 328]

5 minutes ago, claude0001 said:

Sadly yes. As far as I know, that happened because changes in Android security mechanisms made porting AddonSU to LOS 17 non-trivial at best. One of the reasons I am camping on LOS 16 ...

I'm curious about how you use 'su' or root access, if you're willing to describe it. For my part I haven't needed anything more than 'adb root; adb shell' for occasional administrative (mostly development) work in many years.

[claude0001 1,341]

4 hours ago, Sean McCreary said:

I'm curious about how you use 'su' or root access, if you're willing to describe it. For my part I haven't needed anything more than 'adb root; adb shell' for occasional administrative (mostly development) work in many years.

Well, even within Android there are use cases that require root management. E.g. file managers. However, the main reason is that I run a GNU/Linux chroot in parallel to Lineage that requires root access for all kinds of things.

Last, I simply like the idea that I have full control of the system from userland if I want or need it. I consider my Pro1 to be more of a computer than an appliance. So I expect to have the same possibilities as admin than on my PCs. 🙂

Of course, Magisk could do all of that. I like AddonSU because it is low-profile, nicely integrated in LOS, and an "official" solution.

Edited August 29, 2021 by claude0001

[Hook 3,020]

1 hour ago, claude0001 said:

Of course, Magisk could do all of that. I like AddonSU because it is low-profile, nicely integrated in LOS

I know that isn't your only reason, but for others, if you sideload Magisk as a zip, it pretty much works as AddonSU did.  Gives you root, doesn't require Magisk Manager (although it does install it—I ignore it), doesn't require any bootloader dance.  It just gives root to the apps you grant root access to. Acts just like AddonSU for me.

With Magisk 23.0, you have toe rename the apk to zip to flash it.  And this install method is allegedly depreciated, but it works fine.

[claude0001 1,341]

2 hours ago, Hook said:

Gives you root, doesn't require Magisk Manager (although it does install it—I ignore it), doesn't require any bootloader dance.  It just gives root to the apps you grant root access to.

That sounds good. I wonder why there is no "light" version of Magisk as spritual successor of AddonSU. I really do not need all of that advanced stuff related to "hiding" the fact that I am rooted. Know what: I am proud of running a rooted device!

I hate all that FUD spread by Apple and Google around rooted phones (despite them knowing better, of course). Guess what: I have root access to my desktop PC, my laptops and my (Internet exposed) home server. Of course that does not mean that *all* software on them runs with root privileges! And, yes, of course, my bank allows me to make transactions from my "rooted" Thinkpad! Why should we look at smartphones any differently? Wtf is wrong with this world?! 🙂  

Edited August 29, 2021 by claude0001

[Rob. S. 1,661]

55 minutes ago, claude0001 said:

And, yes, of course, my bank allows me to make transactions from my "rooted" Thinkpad! Why should we look at smartphones any differently?

Indeed. It gets even weirder when not only a rooted, up-to-date LineageOS, but even an unrooted, up-to-date LineageOS with the latest security fixes is seen as a security issue by many banking apps, while an ancient Android 7 phone that never received any fixes is deemed secure enough for those apps to not complain...

55 minutes ago, claude0001 said:

Wtf is wrong with this world?!

Don't get me started on this 😉

Edited August 29, 2021 by Rob. S.

[VaZso 1,998]

I have made a lot of keyboard-related changes on the code, now I will try to summarize them.

Also, I have tried to reach a state what probably all of us may find useful and also original working can be kept in case someone does not need additional features at all.

Driver-related changes

I have initialized some additional virtual nodes of keyboard driver's kernel module where driver's behaviour can be changed.
Some additional settings were necessary to make them accessible to system UI.
Also, I have borrowed some of the ideas like what @Sean McCreary has already done.

The additional virtual nodes are as follows:

• load_fn_array
  It can be selected whether the layout-specific codes are to be loaded or not (so the special behaviour of fn-key)
• exposed_key_fn_l
• exposed_key_fn_r
• exposed_key_fn_fx
  It can be selected if the keys above should behave like they were or provide a special keycode instead.
  They can be set to any values what the driver will pass to input_report_key() function in gpio_keys_gpio_report_event().
  
  If load_fn_array (above) is set, fn keys will only have special functions where fn array does not define something else.
  Also, if fn_l and/or fn_r keys are in default working state, it will present KEY_FN function as usual and they can be defined to expose KEY_FN directly and separately, so it will also work if load_fn_array(above) is disabled.
  That way it will behave very similarly as stock behaviour, also Finqwerty may be installed I think (but it should).
  
  The code review where I have borrowed the naming convention of fn_l and fn_r was mentioned the two fn keys may not be handled by userspace applications, but this way the driver handles it, so it really has value.

UI-related changes

At advanced keyboard settings, the following new functions are included:

• Enable special FN functions
  This switch enables or disables loading of special FN table (load_fn_array interface).
• FN + arrows to PAGE keys and INSERT to DELETE
  This function was hardcoded in keyboard layouts and they are still hardcoded, but that is not the right place for them.
  Later (if the change is acceptable) also QWERTY/QWERTZ table may be revised and these parts can be removed there.
  However, by disabling load_fn_array and enabling exposed_key_fn functions, these modified keys are still can be applied.
• Expose FN_L key to applications
• Expose FN_R key to applications
• Expose FX key to applications
  Here are the same list was added.
  I am unsure if all of them are usable by other applications, but currently these are:
  • Disabled (that is the normal state of the button)
  • FN
  • AltGr
  • Alt
  • Left Meta
  • Right Meta
  • Left FN (the special code)
  • Right FN (the special code)
• Custom keymap
  Now it does not disable alternative settings only because a file was copied in place.
  Automatic custom loading still does not work when checked and not even at boot time.
  Still has to be investigated.
  It finds the keymap (if it has appropriate permissions / system user) and it seems to try loading it, but I don't know what is the cause it does not apply.
  The layout changes anyway, but at first because it sends "QWERTY"/"QWERTZ" setting again which re-initializes the keyboard driver.

Other thoughts

I may upload the code back to GitHub but refer to @Slion and the Gerrit system, initiate a pull request there may not has real meaning (apart of the code comparing).

Still I haven't looked into Gerrit but build system seems to have support for using it (okay, for a reason), so I will try to push it there.

Currently I am really tired as I worked a lot on it yesterday and today and now I feel it has reached a state where it seems to work well and I also hope so. 🙂

I will upload the image file somewhere and kindly ask you to test it, I think in about an hour, I have to relax a bit and I am really tired.

It may happen it still has problems (also I have been doing it overnight yesterday and currently midnight is over) and also I would like to mention I am not an Android and not even a Java programmer but I have tried to use the same methods which are already in use and I have also tried to keep the very same design rules, but it may happen I made it wrong somewhere - I don't know how strict they are anyway.
I don't think there are huge problems with it, however, it may happen there is a better solution for example to setting up an initial on state of "Enable special FN functions" setting - it used to be turned on but not at UI after first boot until it was toggled which I have solved how I could...

So, looking forward your feedbacks especially when someone may install / test it and if you may have suggestions, please share it with me.

I will come back with installation files later but I need some time to do something else and even to have a bath. 🙂

Special thanks

Oh, and thanks for my friend who borrowed his Pro1 for me to use it as a development unit temporarily.
He stopped using his device a few months ago because he felt enough of stock OS, then maybe about one or two weeks ago he has asked me to install LineageOS on it which leaded me to install LineageOS on my own device which leaded some problems for me on keyboard driver which leaded me here.

After about a week of use by him as a secondary device, he felt the installation had to be happen earlier and he would like to continue using it as a secondary device...

...but this work only has real value if it can be included in official LineageOS, so this is not the very end for now.

Here are two photos of settings menu:

If you have reached the very end of this very long message...
...what do you think about this modification?

[claude0001 1,341]

1 hour ago, Rob. S. said:

1 hour ago, claude0001 said:

Wtf is wrong with this world?!

Don't get me started on this 😉

After thinking about it, I see now that that rhetoric question was not very compassionate.

That millions of children suffer from hunger is indeed unbelievable.

That Apple and Google feist on user rights may be deplorable too, but it is also not that inevitable, after all ...

Edited August 29, 2021 by claude0001

[Rob. S. 1,661]

11 minutes ago, claude0001 said:

After thinking about it, I see now that that rhetoric question was not very compassionate.

That millions of children suffer from hunger is indeed unbelievable.

That Apple and Google feist on user rights may be deplorable too, but it is also not that inevitable, after all ...

Don't worry, I don't think your criticism regarding users' rights would have made anyone infer that you wouldn't care about world hunger.

Maybe even less so if there might be agreement about both issues, different as they may be, having common causes.

Edited August 29, 2021 by Rob. S.

[VaZso 1,998]

I have uploaded temporarily the update file here.

@EskeRahn, @Slion and others who may have chance for it - could you please try it and tell me if you find anything which could be improved?

Edit:
Naturally, additional layouts (key arrangement) can also be applied more easily.

Edited August 29, 2021 by VaZso