Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

**nickw2066** · 4th November 2012, 01:53

Hi,

I'm using: PyQt4, OS X 10.7.5

In an editor generated by QWebView I am unable to display five-digit hex unicode code points properly.

For example, in the editor, the character "ð’€€" (U+12000, a Cuneiform sign, using the Neo-Assyrian font at: http://www.hethport.uni-wuerzburg.de/cuneifont/) displays both a "ð’€€" and a square box which is shaded on the left side and has the right side of an "A" on its right side.

When I put a space (" ") in front of the Cuneiform sign the box disappears, but an empty space (which can't be deleted) appears after the Cuneiform sign.

Other five-digit hex symbols (e.g. "ð“€€", U+13000, using the Aegyptus font from: http://users.teilar.gr/~g1951d/) also have the same problem.

Four-digit hex symbols (e.g. "áˆ€", U+1200) seem to work fine, without any of these issues.
Mac's TextEdit has no troubles displaying the five-digit hex unicode symbols.

Could the issue be QWebView's handling of Unicode symbols outside the four-digit hex range?

Thanks,

Nick Webb

**ChrisW67** · 6th November 2012, 00:02

This code snippet:

Qt Code:

Switch view

QString cuneiform = QString::fromUtf8("\xF0\x92\x80\x80");
// OR
// uint character = 0x00012000;
// QString cuneiform = QString::fromUcs4(&character, 1);
// OR 
// ushort character[] = {0xD808,0xDC00};
// QString cuneiform = QString::fromUtf16(character, 2);
qDebug() << cuneiform;
 
QWebView w;
w.setHtml(QString("<html><body>++%1++</body></html>").arg(cuneiform));
w.show();

QString cuneiform = QString::fromUtf8("\xF0\x92\x80\x80");
// OR
// uint character = 0x00012000;
// QString cuneiform = QString::fromUcs4(&character, 1);
// OR 
// ushort character[] = {0xD808,0xDC00};
// QString cuneiform = QString::fromUtf16(character, 2);
qDebug() << cuneiform;

QWebView w;
w.setHtml(QString("<html><body>++%1++</body></html>").arg(cuneiform));
w.show();

To copy to clipboard, switch view to plain text mode

results in the correct glyph in my Linux console and QWebView.

What is the encoding of the HTML file and how are you loading it?

**nickw2066** · 7th November 2012, 20:58

The following displays ++<two empty boxes>++ in a small window for me:

Qt Code:

Switch view

import sys
 
from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView
 
# Constants
CUNE_STR = u"\U00012000"
HTML = r"<html><body>++%1++</body></html>"
 
# Main
 
qStr = QString(HTML).arg(CUNE_STR)
 
app = QApplication(sys.argv)
 
widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()
 
webView = QWebView(widget);
webView.setHtml(qStr);
webView.show()
 
sys.exit(app.exec_())

import sys

from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView

# Constants
CUNE_STR = u"\U00012000"
HTML = r"<html><body>++%1++</body></html>"

# Main

qStr = QString(HTML).arg(CUNE_STR)

app = QApplication(sys.argv)

widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()

webView = QWebView(widget);
webView.setHtml(qStr);
webView.show()
 
sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode

**ChrisW67** · 7th November 2012, 22:01

Works fine here. What happens if you add:

Qt Code:

Switch view

print type(CUNE_STR), CUNE_STR.encode('utf-8')

print type(CUNE_STR), CUNE_STR.encode('utf-8')

To copy to clipboard, switch view to plain text mode

Assuming you have a console that handles UTF-8 and has access to the font you should see "<type 'unicode'>" and the correct glyph.

As experiments you could try:

Qt Code:

Switch view

HTML = u"<html><body>++%1++</body></html>"

HTML = u"<html><body>++%1++</body></html>"

To copy to clipboard, switch view to plain text mode

to force Unicode encoding of the HTML. Also try putting the cuneiform glyph directly into the unicode HTML avoiding the possible mangling that QString::arg() might inflict.

There are some rather unclear rules that apply to Python unicode to QString transforms:
http://www.riverbankcomputing.com/st...l/gotchas.html

**nickw2066** · 8th November 2012, 06:45

Firstly, thanks for your help.

The following produces "<type 'unicode'>" and the correct glyph:

Qt Code:

Switch view

print type(CUNE_STR), CUNE_STR.encode('utf-8')

print type(CUNE_STR), CUNE_STR.encode('utf-8')

To copy to clipboard, switch view to plain text mode

This does not fix my problem:

Qt Code:

Switch view

HTML = u"<html><body>++%1++</body></html>"

HTML = u"<html><body>++%1++</body></html>"

To copy to clipboard, switch view to plain text mode

Finally, putting the cuneiform glyph directly into the unicode HTML does not fix my problem either.

I'm using Python 2.7.2 and I can't find anything on http://www.riverbankcomputing.com/st...l/gotchas.html that helps solve my problem.

**ChrisW67** · 8th November 2012, 10:14

OK. We know the Python string is OK. If you:

Qt Code:

Switch view

print qStr.toUtf8()

print qStr.toUtf8()

To copy to clipboard, switch view to plain text mode

I will guess that the result is not good.

Perhaps this has a different effect:

Qt Code:

Switch view

qStr = QString(HTML).arg(QString(CUNE_STR))

qStr = QString(HTML).arg(QString(CUNE_STR))

To copy to clipboard, switch view to plain text mode

by forcing the CUNE_STR explicitly through the QString constructor

**nickw2066** · 8th November 2012, 19:16

If I use:

Qt Code:

Switch view

HTML = u"<html><body>++\U00012000++</body></html>"
qStr = QString(HTML)
print qStr.toUtf8()

HTML = u"<html><body>++\U00012000++</body></html>"
qStr = QString(HTML)
print qStr.toUtf8()

To copy to clipboard, switch view to plain text mode

then the HTML string with the correct glyph in it is printed.

If I use:

Qt Code:

Switch view

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012
 
@author: nick
'''
 
import sys
 
from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView
 
# Constants
CUNE_STR = u"\U00012000"
HTML = u"<html><body>++%1++</body></html>"
 
# Main
 
qStr = QString(HTML).arg(QString(CUNE_STR))
 
app = QApplication(sys.argv)
 
widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()
 
webView = QWebView(widget);
webView.setHtml(qStr);
webView.show()
 
sys.exit(app.exec_())

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012

@author: nick
'''

import sys

from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView

# Constants
CUNE_STR = u"\U00012000"
HTML = u"<html><body>++%1++</body></html>"

# Main

qStr = QString(HTML).arg(QString(CUNE_STR))

app = QApplication(sys.argv)

widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()

webView = QWebView(widget);
webView.setHtml(qStr);
webView.show()
 
sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode

then the same problem occurs: a window pops up with: ++<two empty boxes>++.

**ChrisW67** · 8th November 2012, 19:40

Aarrgghh!! It looks like the Python-esque QString::arg() is mangling your character.

Perhaps you could use Python methods to put the strings together then do a single conversion into QString.

**nickw2066** · 8th November 2012, 20:13

I don't think it's a string-combining problem, as even this:

Qt Code:

Switch view

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012
 
@author: nick
'''
 
import sys
 
from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView
 
# Constants
CUNE_STR = u"\U00012000"
HTML = u"<html><body>++\U00012000++</body></html>"
 
# Main
 
#qStr = QString(HTML).arg(QString(CUNE_STR))
 
app = QApplication(sys.argv)
 
widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()
 
webView = QWebView(widget);
webView.setHtml(HTML);
webView.show()
 
sys.exit(app.exec_())

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012

@author: nick
'''

import sys

from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView

# Constants
CUNE_STR = u"\U00012000"
HTML = u"<html><body>++\U00012000++</body></html>"

# Main

#qStr = QString(HTML).arg(QString(CUNE_STR))

app = QApplication(sys.argv)

widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()

webView = QWebView(widget);
webView.setHtml(HTML);
webView.show()
 
sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode

where I make no use of QString, still only produces a box with: "++<two empty boxes>++"

Considering unicode characters < 0xFFFF, I'm getting some strange behaviour. For example:

Qt Code:

Switch view

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012
 
@author: nick
'''
 
import sys
 
#from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView
 
# Constants
#CUNE_STR = u"\U00012000"
HTML = u"<html><body>++Ø€Ø€\u1250++</body></html>"
 
# Main
 
#qStr = QString(HTML).arg(QString(CUNE_STR))
 
app = QApplication(sys.argv)
 
widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()
 
webView = QWebView(widget);
webView.setHtml(HTML);
webView.show()
 
sys.exit(app.exec_())

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
'''
Created on 7 November 2012

@author: nick
'''

import sys

#from PyQt4.QtCore import QString
from PyQt4.QtGui import QApplication, QWidget
from PyQt4.QtWebKit import QWebView

# Constants
#CUNE_STR = u"\U00012000"
HTML = u"<html><body>++Ø€Ø€\u1250++</body></html>"

# Main

#qStr = QString(HTML).arg(QString(CUNE_STR))

app = QApplication(sys.argv)

widget = QWidget()
 
widget.resize(320, 240)
widget.setWindowTitle("Hello, World!")
widget.show()

webView = QWebView(widget);
webView.setHtml(HTML);
webView.show()
 
sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode

displays the correct character in the window, but if I change the HTML string to:

Qt Code:

Switch view

HTML = u"<html><body>++Ø€Ø€\u1200++</body></html>"

HTML = u"<html><body>++Ø€Ø€\u1200++</body></html>"

To copy to clipboard, switch view to plain text mode

only "++<empty box>++" displays (EDIT: My system seems to have the font to display this glyph because it displays correctly in Mac's Character Viewer).

**nickw2066** · 15th November 2012, 02:23

**bump**

Anyone have any idea about the \u1250 vs. \u1200 problem described immediately above?

Thread: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Thread Tools

Search Thread

Display

Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Re: Does QWebView support Unicode characters > 0xFFFF (for Cuneiform fonts)

Similar Threads

Help needed to convert unicode characters

How to show Unicode characters on MS Windows?

QString to unicode characters

Unicode/ASCII characters in QTextStream

Insertion of unicode characters into database oracle through pro c

Bookmarks

Bookmarks

Posting Permissions