Passing some Unicode characters to Linux Qt applications

[PCXT]

Hello

My configuration:
- Debian 10 Buster, Qt 5.11.3 (from distribution packages), X11-based desktop.

I have a problem with Qt (probably with Qt) and I just don't know where to start looking for its culprit. First of all, I decided to make a small program in which I need to insert characters to other applications. Generally, the window is like on-screen keyboard and pressing a button will cause character being sent to currently open application.
The code I have for this action is a known approach, XLib and Xtest to feed re-defined keycodes:


void MainWindow::push_char(char character)
{
Display * disp; //current X11 display
disp=XOpenDisplay(nullptr); //TODO: fail gracefully when abused in console.
KeySym * xkeysyms = nullptr;
int temporaryKeyCode=0;
int keycodes_start=0;
int keycodes_end=0;
int keysyms_per_keycode=0;
XDisplayKeycodes(disp,&keycodes_start,&keycodes_end); //obtain keycode range for current display.
xkeysyms=XGetKeyboardMapping(disp,keycodes_start,k eycodes_end-keycodes_start,&keysyms_per_keycode); //get keysyms start-end

for (int i=keycodes_start; i<=keycodes_end; i++) //Iterate through keycode list to obtain free one
{
bool empty_keycode=true;
for (int j=0; j<keysyms_per_keycode; j++) //check all keysyms to be zero
{
int keysym_index = (i - keycodes_start) * keysyms_per_keycode + j; //obtain keysym position
if(xkeysyms[keysym_index] != 0) //if the key is not empty...
{
empty_keycode=false; //nope... next one
break;
}
}
if (empty_keycode) //use this code for our character.
{
temporaryKeyCode=i;
break;
}
}
XFree(xkeysyms); //this finding and freeing must be done every time, not at the program boot-up,
XFlush(disp); //as user may change X settings on the fly, including key tables.
usleep(100*1000); //0.1 delay

//Prepare for character send
KeySym sym = XStringToKeysym("U01F92A"); //Uxxxx links unknown UTF char to key even when it's not in any definition. As in xkbcommon-keysyms.h
// KeySym sym = XStringToKeysym("U0142"); //THIS WORKS., the above does not in all applications.

if (sym == NoSymbol) //This should rarely happen.
{
sym=XStringToKeysym("x"); //have anything here or throw error.
}

KeySym symlist[2]={sym, sym}; //create a keysym list for shifted and unshifted chr
XChangeKeyboardMapping(disp,temporaryKeyCode,2,sym list,1); //assign the list to the mapping
XFlush(disp);
KeyCode code=temporaryKeyCode; //create code out of temporary

XTestFakeKeyEvent(disp, code, true, 0); //Press...
XFlush(disp);
usleep(90 * 1000);
XTestFakeKeyEvent(disp, code, false, 0); //Release...
XFlush(disp);

symlist[0]=0;
symlist[1]=0;
XChangeKeyboardMapping(disp,temporaryKeyCode,2,sym list,1); //revert the mapping
usleep(90 * 1000);

XCloseDisplay(disp); //End routine
return;
}

In the code, I have two test lines for making a keysym of UTF character: First one makes it from "U0142", second is "U01F92A" (or U1F92A - an emoji which, at least in my system, shows a smiley).
The routine (currently argument is not processed) works, but not for all characters and not for all applications. Things I observed:
- When characters are like Uxxxx, 2 hex bytes after U, it's OK for all applications I tested: Based on GTK, Qt, Qt Creator itself too, WxWidgets and even an old ?Qt3-based? TDE.
- When characters are longer, like Uxxxxxx - 3 hex bytes after U, it works for GTK, WxWidgets, TDE, but not Qt applications, even not in Qt Creator itself.

This is a strange behavior and for me it looks like there may be some problem with library configuration in my set-up?
When instead of my code I use xdotool (xdotool type "?") I have a similar behavior, that's why I think about misconfiguration.

Thanks in advance.
MW

[d_stranz]

In the code you posted, I don't see any references to Qt at all, other than that this method is in a MainWindow class. Where is the Qt connection?

[PCXT]

There are two things:
1. I'm doing it from Qt application, but it has smaller meaning.
2. It cannot insert character to any Qt5 application - even to itself. That's why I'm thinking about Qt misconfiguration.
The last character I can pass, if U+FFFD, all bigger code points fail (I think the last is U+FFFF, but it's not character by standard).

[d_stranz]

Have your read the Qt documentation (https://doc.qt.io/qt-5/unicode.html) concerning Unicode? Your problem may be in how you are handling the characters either as input to QString or as output to a Qt-based widget for display.

There are references to "16-bit Unicode", which could be the problem.

[PCXT]

Thanks for the information about Unicode in Qt! I found that another, not related problem I had was caused by casting the characters wrong and now I can process these characters well inside my application (as QStrings), so I worked it around using X selection buffer (lower characters are typed as above, higher - copied through the buffer, good that I can stuff ctrl-ins there).

The problem seems to be not especially in my application's code (which currently is a single form and a button calling the routine), but it may be in Qt configuration in my system, as any X11 "keyboard-simulating" program doing it the similar way as shown in my first post cannot send higher Unicode characters specifically to Qt5-based applications (?or sends and Qt applications can't process them?), even to programs which are from repository, not compiled by me. That's why I suspect some define has been switched wrong during Qt build (I have Qt from Debian repository, so that's why I'm a bit dubious about its build configuration, I had a similar problem before with Qt4's QSerialPort).

So my short reproduction using Linux, xdotool and two applications: In the terminal:

sleep 1; xdotool type "?"

During one second of delay, I switch to another application.
So, it types the smiley into Wx or GTK applications, but not into Qt5 applications, like Qt Creator.

But when I put a lower UTF character, let's try with the last one (replacement sign, U+FFFD):

sleep 1; xdotool type "?"

It is typed into application in both cases.

EDIT:
P.S.
OK, for some reason the emoji I pasted here got replaced with ?, so it may be here or in the forum. The first ? I put in xdotool was U+1F60A character, the second ? was U+FFFD character.