+ Reply to Thread
Results 1 to 4 of 4

Thread: Poor asynchronous performance in PyQT

  1. #1
    Join Date
    Oct 2017
    Posts
    3
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11

    Default Poor asynchronous performance in PyQT

    Recently I really got into async stuff in Python. Unfortunately, it seems like Python's stock asynchronous capabilities do not mesh well with Qt's QEventLoop.

    Boiled down test program: receive stuff from a client on a socket and insert rows into an QAbstractTableModel that's connected to a QTableView. Receive a message, create a new row in a table -- real simple.

    Threaded test code uses socketserver module. Async code uses asyncio.Server, while Quamash is used to "bridge the gap" between Python's native async and QEventLoop (it should be noted that replacing Quamash with a bare-bones solution has no effect on the program. uvloop doesn't help either).
    All Qt-relevant code is shared, so there's no difference in views or models.

    Threaded program has no problem receiving and displaying tens of thousands messages per second. Asynchronous program can barely handle 150 messages per second.
    The peculiar thing is that the size of the window has a great effect on the async program. If I make it full screen, it drops to 70 messages per second. If I make it the smallest it can be, then it reaches 350 messages per second. If I hide the table view completely, the program becomes practically as fast as the threaded one.


    Is this how Qt normally behaves in asynchronous Python code? If not, I would really appreciate some explanation of why it could be acting like this.

    But if you think I should not bother with QEventLoop stuff, then I see 2 options:
    • Rip out all the async stuff and just write synchronous threaded code like a normal person? This is a feasible option, because I don't really need async in this project, I just wanted to have it because I like it and it has its benefits.
    • Fork async server stuff out into a thread and keep the Qt stuff synchronous? I tried this, it's maximum jank, but at least it's as fast as I need it to be. Could probably be a reasonable solution with some architectural changes to the project.

    Any advice is welcome, since there's barely any information on this topic.

    P.S. I didn't bother to include any code, sorry. Tell me if you feel like it's necessary. It'll take a bit of time, though.
    Last edited by busimus; 12th October 2017 at 18:02.

  2. #2
    Join Date
    Oct 2017
    Posts
    3
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11

    Default Re: Poor asynchronous performance in PyQT

    Further testing has shown that, unsurprisingly, having an async server running is irrelevant to the performance issue. Even inserting rows and sleeping asynchronously is enough to show awful performance.

    So I prepared the simplest example code:
    Qt Code:
    1. import asyncio
    2. import signal
    3.  
    4. from PyQt5.QtCore import QAbstractTableModel, QModelIndex, Qt
    5. from PyQt5.QtWidgets import QApplication, QMainWindow
    6. from PyQt5 import QtWidgets
    7.  
    8.  
    9. INVALID_INDEX = QModelIndex()
    10.  
    11. QUAMASH = False
    12.  
    13.  
    14. class TableModel(QAbstractTableModel):
    15.  
    16. def __init__(self, parent):
    17. super().__init__(parent)
    18. self.parent = parent
    19. self.records = []
    20. self.count = 0
    21. self.columns = ['column']
    22.  
    23. def columnCount(self, index):
    24. if not index.isValid():
    25. return len(self.columns)
    26. else:
    27. return 0
    28.  
    29. def rowCount(self, index):
    30. if index.isValid():
    31. return 0
    32. else:
    33. return len(self.records)
    34.  
    35. def data(self, index, role=Qt.DisplayRole):
    36. result = None
    37. if index.isValid():
    38. record = self.records[index.row()]
    39. if role == Qt.DisplayRole:
    40. return record
    41. return result
    42.  
    43. def headerData(self, section, orientation, role):
    44. result = None
    45. if orientation == Qt.Horizontal and role == Qt.DisplayRole:
    46. result = self.columns[section]
    47. return result
    48.  
    49. def add_row(self, record):
    50. pos = len(self.records)
    51. self.beginInsertRows(INVALID_INDEX, pos, pos)
    52. self.records.append(record)
    53. self.endInsertRows()
    54. self.count += 1
    55.  
    56. def clear(self):
    57. self.records.clear()
    58.  
    59.  
    60. class MainWindow(QMainWindow):
    61.  
    62. def __init__(self, loop):
    63. self.loop = loop
    64. super().__init__()
    65.  
    66. self.stop_signal = self.loop.create_future()
    67. self.table_shown = True
    68.  
    69. self.setupUi()
    70. self.loop.create_task(monitor(self.table_model, self))
    71. self.loop.create_task(self.loop_add_row())
    72.  
    73. def setupUi(self):
    74. self.resize(800, 600)
    75. self.centralwidget = QtWidgets.QWidget(self)
    76. self.verticalLayout = QtWidgets.QVBoxLayout(self.centralwidget)
    77. self.logTable = QtWidgets.QTableView(self.centralwidget)
    78. sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Expanding, QtWidgets.QSizePolicy.Expanding)
    79. sizePolicy.setVerticalStretch(69)
    80. self.logTable.setSizePolicy(sizePolicy)
    81. self.verticalLayout.addWidget(self.logTable)
    82. spacerItem = QtWidgets.QSpacerItem(20, 10, QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Expanding)
    83. self.verticalLayout.addItem(spacerItem)
    84. self.hideButton = QtWidgets.QPushButton(self.centralwidget)
    85. self.hideButton.setText("Hide or unhide table")
    86. self.verticalLayout.addWidget(self.hideButton)
    87. self.setCentralWidget(self.centralwidget)
    88. self.statusbar = QtWidgets.QStatusBar(self)
    89. self.setStatusBar(self.statusbar)
    90.  
    91. self.table_model = TableModel(self)
    92.  
    93. self.logTable.setModel(self.table_model)
    94. self.logTable.verticalScrollBar().rangeChanged.connect(self.logTable.scrollToBottom)
    95.  
    96. self.hideButton.clicked.connect(self.toggle_table_shown)
    97.  
    98. self.setWindowTitle('QT TESTING WINDOW')
    99.  
    100. self.show()
    101.  
    102. def toggle_table_shown(self):
    103. if self.table_shown:
    104. self.logTable.hide()
    105. else:
    106. self.logTable.show()
    107. self.table_shown = not self.table_shown
    108.  
    109. async def loop_add_row(self):
    110. c = 0
    111. while True:
    112. self.table_model.add_row(f"row {c}")
    113. c += 1
    114. done, pending = await asyncio.wait([asyncio.sleep(1e-6), self.stop_signal],
    115. return_when=asyncio.FIRST_COMPLETED)
    116. if self.stop_signal in done:
    117. return
    118.  
    119. def closeEvent(self, event):
    120. self.shutdown()
    121. event.accept()
    122.  
    123. def shutdown(self, signal=None):
    124. try:
    125. print("stopping")
    126. self.stop_signal.set_result('shutdown')
    127. self.close()
    128. except asyncio.InvalidStateError:
    129. raise SystemExit
    130.  
    131.  
    132. async def monitor(model, window):
    133. while True:
    134. await asyncio.sleep(1)
    135. status = f"{model.count} rows/s"
    136. print(status)
    137. window.statusBar().showMessage(status)
    138. model.count = 0
    139.  
    140.  
    141. async def process_events(qapp):
    142. while True:
    143. await asyncio.sleep(0)
    144. qapp.processEvents()
    145.  
    146.  
    147. if __name__ == "__main__":
    148.  
    149. app = QApplication([])
    150.  
    151. if QUAMASH:
    152. from quamash import QEventLoop
    153. loop = QEventLoop(app)
    154. asyncio.set_event_loop(loop)
    155. else:
    156. loop = asyncio.get_event_loop()
    157.  
    158. main = MainWindow(loop)
    159. loop.add_signal_handler(signal.SIGINT, main.shutdown, None)
    160.  
    161. if QUAMASH:
    162. with loop:
    163. loop.run_forever()
    164. else:
    165. loop.run_until_complete(process_events(app))
    166.  
    167. app.closeAllWindows()
    To copy to clipboard, switch view to plain text mode 

    This code creates a window with a table and starts filling it with rows of strings through a task.
    Row insertion speed is shown in the status bar, as well as printed. You can press the button to hide the table, if you want to see how fast the code is actually supposed to go.
    Tested with Python 3.6.2 and PyQt5.9. Quamash is not required, although if you have it installed it can be enabled by setting QUAMASH variable at the beginning of the file to True.

  3. #3
    Join Date
    Oct 2017
    Posts
    1
    Thanked 1 Time in 1 Post
    Qt products
    Qt5
    Platforms
    Unix/X11

    Default Re: Poor asynchronous performance in PyQT

    The proccessEvents takes a couple of milliseconds per call when a row has been added and it looks like this 'starves' the asyncio loop. When the time to sleep is non-zero the asyncio loop gets some room to breath and the performance goes from 200 to 6000 rows/s (14000 when table is hidden) on my system. The longer the asyncio sleep the less Qt overhead and the more efficient.

    Qt Code:
    1. async def process_events(qapp):
    2. while True:
    3. await asyncio.sleep(0.005)
    4. qapp.processEvents()
    To copy to clipboard, switch view to plain text mode 


    Added after 42 minutes:


    Adding the rows in chunks also helps a lot:

    Qt Code:
    1. async def loop_add_row(self):
    2. c = 0
    3. while not self.stop_signal:
    4. for i in range(1000):
    5. self.table_model.add_row(f"row {c}")
    6. c += 1
    7. await asyncio.sleep(0)
    To copy to clipboard, switch view to plain text mode 

    It gives 84000 rows/s or 178000 when table is hidden (with the Quamash loop).

    PS: stop_signal is turned into a boolean here, it's no longer a future.
    Last edited by erdewit; 14th October 2017 at 12:30. Reason: doublure

  4. The following user says thank you to erdewit for this useful post:

    busimus (14th October 2017)

  5. #4
    Join Date
    Oct 2017
    Posts
    3
    Thanks
    1
    Qt products
    Qt5
    Platforms
    Unix/X11

    Default Re: Poor asynchronous performance in PyQT

    Thank you, erdewit.
    Your fix indeed worked for the test program, but not for my actual project. No idea why, as it's pretty much the same code (in places where it matters, at least). Strange and irritating, I'll be investigating.

    And yeah, chunking rows would help, but I don't think it makes a lot of sense for the original purpose (receiving rows from the network). Could be a good enough workaround if I don't find the root cause of the issue I'm facing currently.
    Last edited by busimus; 14th October 2017 at 16:10.

+ Reply to Thread

Similar Threads

  1. DashLine drawing poor performance
    By jecaro in forum Qt Programming
    Replies: 0
    Last Post: 11th August 2010, 14:46
  2. QTableView - resizeRowsToContents() poor performance
    By antarctic in forum Qt Programming
    Replies: 2
    Last Post: 11th December 2009, 14:13
  3. Poor OpenGL performance
    By rakkar in forum Newbie
    Replies: 1
    Last Post: 3rd September 2009, 20:51
  4. Qt4 poor printing performance
    By seneca in forum Qt Programming
    Replies: 4
    Last Post: 22nd January 2009, 15:23
  5. Poor performance with Qt 4.3 and Microsoft SQL Server
    By Korgen in forum Qt Programming
    Replies: 2
    Last Post: 23rd November 2007, 11:28

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts