Unicode on (Win32) console

[Greisby]

Hi,

I'm trying to output unicode in a (Win32) console application, without success until now.
I read on a thread that I should use a QTextStream for this. I tried but did not get any output.

Here my code:

int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);

QTextStream qStdOut(stdout, QIODevice::WriteOnly);
// qStdOut.setCodec("UTF-16");
QString unicodeString(QChar(0x9788));
qStdOut << QString("QTextStream: ") << unicodeString << QChar('\n');
std::cout << "cout: " << (char*)unicodeString.utf16() << std::endl;
printf("printf: %ls\n", unicodeString.utf16());

return a.exec();
}


The output I get is:

cout: êù
printf:

The standard cout/printf don't output unicode, and the QTextStream nothing at all... no "QTextStream: ☼"
Is there something I missed?

[wysota]

Does the Windows console support UTF-16? Try writing to a file instead of stdout and check if you get correct output.

[jacek]

no "QTextStream: ☼"
Is there something I missed?
Add qStdOut.flush().

[Greisby]

Thanks for your answers.

1. Yes the console supports unicode. I created an unicode text file «test.txt» and displayed it (type test.txt). The unicode chars are correctly displayed.
2. thanks, I got a bit further with the flush.

I now have:

QTextStream: ?
cout: êù
printf:

I tried uncommenting the setCodec("UTF-16") and the outputs becomes:

Q T e x t S t r e a m : êù
cout: êù
printf:

Seems there is still something missing...

[wysota]

Did you try saving that character you want to display from QTextStream to a file?

Based on the fact that you get two characters on the console instead of expected single character, maybe you should assume that in doesn't use UTF-16 but some other encoding (like UTF-8).

[jacek]

I tried uncommenting the setCodec("UTF-16") and the outputs becomes[...]
If you won't set the encoding using setCodec(), QTextStream will use QTextCodec::codecForLocale(), which probably doesn't support all Unicode characters.

[Greisby]

After googling a lot, I saw that lot of people have problems displaying unicode on the console. But I didn't found a solution.

I read that :
1. the console font must be set to lucida TT.
2. the code page must be forced to 1252 (or 65001 for utf-8).


int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QString unicodeString(QChar(0x9788));
QTextStream qStdOut(stdout, QIODevice::WriteOnly);
DWORD dwWritten;

// 1st try: replace oem with ascii
#ifdef _WIN32
std::cout << "Switch input to Ascii CodePage (1252): " << (::SetConsoleCP(::GetACP())?"ok":"fail") << std::endl;
std::cout << "Switch output to Ascii CodePage (1252): " << (::SetConsoleOutputCP(::GetACP())?"ok":"fail") << std::endl;
std::cout << "Current input CodePage: " << (unsigned int)::GetConsoleCP() << std::endl;
std::cout << "Current output CodePage: " << (unsigned int)::GetConsoleOutputCP() << std::endl;
#endif
qStdOut << QString("QTextStream: ") << unicodeString << QChar('\n');
qStdOut.flush();
qStdOut.setCodec("UTF-16");
qStdOut << QString("QTextStream: ") << unicodeString << QChar('\n');
qStdOut.flush();
std::cout << "cout: " << (char*)unicodeString.utf16() << std::endl;
std::cout << "cout: " << (char*)(unicodeString.toUtf8().constData()) << std::endl;
std::wcout << L"wcout: " << (wchar_t*)unicodeString.utf16() << std::endl;
std::wcout << L"wcout: " << (char*)(unicodeString.toUtf8().constData()) << std::endl;
printf("printf: %ls\n", unicodeString.utf16());
wprintf(L"wprintf: %ls\n", unicodeString.utf16());
#ifdef _WIN32
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), L"WriteConsoleW: ", 15, &dwWritten, NULL);
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), (unicodeString + '\n').utf16(), unicodeString.length()+1, &dwWritten, NULL);
#endif

// 2nd try : set CP to utf-16
#ifdef _WIN32
std::cout << "\nSet input CP to ucs2: " << (::SetConsoleCP(1200)?"ok":"fail") << std::endl;
std::cout << "Set output CP to ucs2: " << (::SetConsoleOutputCP(1200)?"ok":"fail") << std::endl;
std::cout << "Current input codepage: " << (unsigned int)::GetConsoleCP() << std::endl;
std::cout << "Current output codepage: " << (unsigned int)::GetConsoleOutputCP() << std::endl;
#endif
qStdOut << QString("QTextStream: ") << unicodeString << QChar('\n');
qStdOut.flush();
std::cout << "cout: " << (char*)unicodeString.utf16() << std::endl;
std::wcout << L"wcout: " << (wchar_t*)unicodeString.utf16() << std::endl;
printf("printf: %ls\n", unicodeString.utf16());
wprintf(L"wprintf: %ls\n", unicodeString.utf16());
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), L"WriteConsoleW: ", 15, &dwWritten, NULL);
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), (unicodeString + '\n').utf16(), unicodeString.length()+1, &dwWritten, NULL);

// 3rd try : set CP to utf-8
#ifdef _WIN32
std::cout << "\nSet input CP to utf-8: " << (::SetConsoleCP(65001)?"ok":"fail") << std::endl;
std::cout << "Set output CP to utf-8: " << (::SetConsoleOutputCP(65001)?"ok":"fail") << std::endl;
std::cout << "Current input codepage: " << (unsigned int)::GetConsoleCP() << std::endl;
std::cout << "Current output codepage: " << (unsigned int)::GetConsoleOutputCP() << std::endl;
#endif
qStdOut.setCodec("UTF-8");
qStdOut << QString("QTextStream: ") << unicodeString << QChar('\n');
qStdOut.flush();
std::cout << "cout: " << (char*)unicodeString.toUtf8().constData() << std::endl;
std::wcout << L"wcout: " << (char*)unicodeString.toUtf8().constData() << std::endl;
printf("printf: %ls\n", unicodeString.toUtf8().constData());
wprintf(L"wprintf: %ls\n", unicodeString.toUtf8().constData());
#ifdef _WIN32
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), L"WriteConsoleW: ", 15, &dwWritten, NULL);
WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), (unicodeString + '\n').utf16(), unicodeString.length()+1, &dwWritten, NULL);
#endif

return a.exec();
}


The resulting output is awfull...

Switch input to Ascii CodePage (1252): ok
Switch output to Ascii CodePage (1252): ok
Current input CodePage: 1252
Current output CodePage: 1252
QTextStream: ?
Q T e x t S t r e a m : ˆ—
cout: ˆ—
cout: 鞈
wcout: printf:
wprintf: WriteConsoleW: 鞈

Set input CP to ucs2: fail
Set output CP to ucs2: fail
Current input codepage: 1252
Current output codepage: 1252
Q T e x t S t r e a m : ˆ—
cout: ˆ—
printf:
wprintf: WriteConsoleW: 鞈

Set input CP to utf-8: ok
Set output CP to utf-8: ok
Current input codepage: 65001
Current output codepage: 65001
QTextStream: 鞈
cout: printf:
wprintf: WriteConsoleW: 鞈

It must be possible since the type command do that.
But I didn't found and renounce (sob).

[Greisby]

Damn. The wordpad converted the symbol to a 8 bit char. That's why the «type» command worked.
I made another file with "日本", and there it isn't displayed either.
I must conclude the the windows console is shit and definitly cannot display unicode.

[Greisby]

After a good night of sleep, I rechecked my last posts and realized that putting the console code page to utf-8 seems to work.


Set input CP to utf-8: ok
Set output CP to utf-8: ok
Current input codepage: 65001
Current output codepage: 65001
QTextStream: &#38792
cout: printf:
wprintf: WriteConsoleW: &#38792


&#38792 = QChar(0x9788)

The problem is the console font. It is not unicode, and I found no way to change it...
A copy/paste of the console output to a Wordpad works.

[MaikoID]

I've the same problem, I cant print my native language pt-br in windows console because I can't print these characters: á é Ã* ó ú ç ...