QGraphicsScene/QGraphicsView performance problems

[bnilsson]

Mine is Linux/x86.

Were you able to build the application?

Can't you just provide some small application that would generate such example datafile? They don't have to make sense and they don't have to be larger than a megabyte. It's just a test app.

The example datafile supplided, "testexp_c.j51" should enable you to thest the function, but maybe not show the problem. I don't have a J51 generator ready, but I can try to make one. It would not take long, I hope.

It'd be best if you provided something small. It's easier to debug then.

I can strip it down further if you want. Should I?

BTW. I just had a quick look at the code you provided and I have some comments:
- I'm not sure if manipulating the update mode the way you do is a good idea, you should set the update mode once and never touch it again

If I turn off update permantly it doesn't scroll. If I turn it on a large file will be drawn twice or more, taking maybe 10-30s extra before the user can zoom in to any region of interest.
Do you have any suggestion on how to handle this?

- I'm not sure if beginning by reimplementing the main drawing routine is a good idea. I'd start with the base class implementation first, it's probably trying to be smarter than your implementation

I dont' quite uderstand. Please expalin.
This is my very forst Qt application, I took the outline from a basic example (a text editor) described in the documentation. My plan was to clean it up later, and evenually make a multidocument application. But I got stuck before I got to that point.

- why do you use the data stream if you only read raw data and seek through the file? Operate on the file directly instead

Sure. In one of my later versions I make a linked list from the data analyzed.

- try not to fall of the scale with your dimensions, 10nm = 10E-8, pretty close to 32bit precision.

10-8 compared to 1 meter, yes. In the drawing unity means 1 micrometer, so the precision is no problem here.

[wysota]

Were you able to build the application?

I had no data so I didn't even try.

I can strip it down further if you want. Should I?

You could get rid of the loading routines and just generate polygons and rectangles in the application.

Do you have any suggestion on how to handle this?

Yes, find out why it draws twice.

I dont' quite uderstand. Please expalin.

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Edit:
The application works just fine with the provided data. It uses about 15MB of RAM which I assume is quite fine.

[bnilsson]

Here is a stripped down version with no loader and a limited set of redundant functions. I did not want to change the UI part so there are some empty action handling methods, please ignore them.

It draws an area set by "chipsize", divided into fields of size "fieldsize", whuch are in turn divided into subfields of size "subfieldsize", and in each subfield there is one rectangle and one polygon. Chip, field and subfield boundaries are drawn in blue color, while the rectangles and polygons are drawn in black.

Chipsize can range from 50,000 microns up to 125,000 microns, the field size is 800 microns, and subfield size is 100 microns. I kept this sceme to be able to relate to my real needs.

Here, 125,000 micron chipsize is beyond the capacity of any of my machines (1 Mac G5 1GB RAM, and 1 amd64 1GB RAM) while 50,000 micron chipsize can be managed with some patience.
Note that Qt4.3.3 has real problems when zooming in on the amd64, the Qt4.4.0 snapshot is much better. They really must have done something there.

Some logs:
Chipsize 50000.000 um, 3844 fields, 246016 subfields, 246016 rects, 246016 polys
Chipsize 100000.000 um, 15625 fields, 1000000 subfields, 1000000 rects, 1000000 polys
Chipsize 125000.000 um, 24336 fields, 1557504 subfields, 1557504 rects, 1557504 polys

[wysota]

Ok, I finally managed to take a look at the app. The example you gave us occupies about 200MB of memory, which is not that bad for 0.5M items ( ~400B per item). The application is horribly slow, but I see some ways to speed it up. I don't know if this will improve the speed significantly, but it's worth to try. I'm going to get rid of your polygon items first.

Edit: Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code. But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon. As you have only four points, there is no point in creating a path from them - you should draw them manualy and return a very tiny bounding rectangle. Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points. It should make a huge difference. And it's not a matter of memory, it's consumption is fine.

[Gopala Krishna]

Ok, I finally managed to take a look at the app. The example you gave us occupies about 200MB of memory, which is not that bad for 0.5M items ( ~400B per item). The application is horribly slow, but I see some ways to speed it up. I don't know if this will improve the speed significantly, but it's worth to try. I'm going to get rid of your polygon items first.

Edit: Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code. But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon. As you have only four points, there is no point in creating a path from them - you should draw them manualy and return a very tiny bounding rectangle. Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points. It should make a huge difference. And it's not a matter of memory, it's consumption is fine.

Very true.. Level of detail really makes lot of difference.. The chip demo which qt provides is really a nice demo on how graphicsview applications can be optimized atleast to some extent.

Actually your app didn't run on my system(1Gb ram, 3GHz, pentium 4 HT) until i changed chip size to 5k.
But to be frank enough, I personally feel gv framework is overkill. Infact your app didn't run(50k) even when i turned of all possible bells and whistles. Also you don't need those unnecessary memory strain as well.
If you are sure that you don't need to select or move items here and there, definitely implementing a widget is best option. Remember you always don't have to draw everything as drawing everything at < .25 scale is hardly visible.

[bnilsson]

First, I want to thank you for your engagement in this. This is my VERY first Qt project, and surely must have missed and misunderstood a lot of things.

Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code.

You have all of my relevant code in the example. What is missing?

But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon.

Here it seems I have missed something really important. I was unaware of pos().

Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points.

Again. I was unaware of levelofDetail, I will study it. However, the description in Assistant is a bit cryptic, so I will have to test different settings. Will a higher value improve?

Gopala:
But to be frank enough, I personally feel gv framework is overkill.

I tend to have the same opinion. But the gv framework is so convenient! But on the other hand, it is the only thing I have looked at so far. Is it as easy to implement scroll and zoom for a regular widget?

pherthyl:
This may be useful to you.

http://labs.trolltech.com/blogs/2008...graphicsitems/

I definitely see this on Linux, but for some reason not on my MacOSX. On Linux, displaying the full chip took time, but was ok finally. Zooming in locked it up completely. On Linux, this was eliminated using Qt4.4.0. Or is this a note of a further improvement?

[bnilsson]

First:

From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon.

This statement puzzles me.
I just do

QVector<QPointF> Tpz(4);
<fill in the 4 points>
QPolygonF Trapezoid(Tpz);
scene->addPolygon(Trapezoid,AreaShotPen);

QVector<QPointF> Tpz(4);
<fill in the 4 points>
QPolygonF Trapezoid(Tpz);
scene->addPolygon(Trapezoid,AreaShotPen);

To copy to clipboard, switch view to plain text mode 

Where does the pos() come in?

Secondly, can you apply levelofDetail to a high-level draw such as scene->addRect();?
Or do you have to create a custom QGraphicsItem and apply it to painter->drawRect() to do that?

[Gopala Krishna]

Secondly, can you apply levelofDetail to a high-level draw such as scene->addRect();?
Or do you have to create a custom QGraphicsItem and apply it to painter->drawRect() to do that?

You have to create a custom item. Have a look at the code for demos/chip/chip.cpp for reference.

[Gopala Krishna]

BTW can you explain the composition of a chip with a screen shot ? i think you can simplify the drawing by representing less items which draw sub polygons.
I mean I could see an array of rectangular items, with some polygons and rect inside.
The outside rect can be considered as one item.

[bnilsson]

This thought has occurred to me, since the pattern is organized in exposure fields and subfields. So the items could be a FieldItem with child SubFieldsItem, which in turn has child ShapeItems. The ShapeItems are the actual stuff that are drawn by my microchip exposure machine which is using the data file as input, and the fields and subfields are placement parameters.
Will this item hierarchy in three levels, if implemented, give any performance gain?
The field content is usually unique for each field, so there is no memory to be gained by repetition. However, if the field and subfield boundaries are to be drawn, these items can be re-used by offset.

[bnilsson]

Knowledge what items represent. I don't want to go through all your code and learn it by trial and error. For example I don't know if you really need 4 points to represent a trapezoid or if those trapezoids have somehow "fixed" properties, for example always a 90deg. angle - then you just need 3 points, or fixed lengths.

Ok, now I understand.
I will have the following kinds of objects to draw:

FieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout is divided into fields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. Typically 800 by 800 micrometer in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the chip.
SubFieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout in the field is divided into subfields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. The subfield size is ALWAYS smaller or equal to the field size, typically 100 by 100 microns in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the current field.
The Real shapes below are ALWAYS contained and enclosed within one subfield, and thus always smaller.
XRectangle:
Should always be drawn. The X side is longer than the Y side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.
YRectangle:
Should always be drawn. The Y side is longer than the X side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.
XTrapezoid:
Top and bottom sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,x1,x2,x3,y3, referenced from the upper left corner of the current subfield.
YTrapezoid:
Left and right sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,y1,y2,x3,y3, referenced from the upper left corner of the current subfield.
Line:
Just a line from x0,y0 to x1,y1, referenced from the upper left corner of the current subfield.

Exposure dose index:
In the input data file, each Real shape may have a tag to control the exposure dose. Here we us colors (up to 63 colors) to present this.

Hope this helps.

[bnilsson]

About LOD:

t is nothing you may set. It's a value given by the view to the item's painting routine that tells it how far away the "camera" is from the item. The further, the less details may be seen, so drawing those details becomes obsolete.

So, why discuss it?
Or is there any way I can make use of it?

[Gopala Krishna]

Ok, now I understand.
I will have the following kinds of objects to draw:

FieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout is divided into fields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. Typically 800 by 800 micrometer in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the chip.
SubFieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout in the field is divided into subfields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. The subfield size is ALWAYS smaller or equal to the field size, typically 100 by 100 microns in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the current field.
The Real shapes below are ALWAYS contained and enclosed within one subfield, and thus always smaller.
XRectangle:
Should always be drawn. The X side is longer than the Y side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.
YRectangle:
Should always be drawn. The Y side is longer than the X side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.
XTrapezoid:
Top and bottom sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,x1,x2,x3,y3, referenced from the upper left corner of the current subfield.
YTrapezoid:
Left and right sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,y1,y2,x3,y3, referenced from the upper left corner of the current subfield.
Line:
Just a line from x0,y0 to x1,y1, referenced from the upper left corner of the current subfield.

Exposure dose index:
In the input data file, each Real shape may have a tag to control the exposure dose. Here we us colors (up to 63 colors) to present this.

Hope this helps.

May be you can have only one item to represent the field and its contents and by contents i don't mean child items.

I am using the terminology "item" to represent an entity on scene and which inherits QGraphicsItem.

The field item is the item responsible to draw its subfields and contents of subfield.

Now if levelOfDetail is very less, just draw the outer boundary of field item only.
If levelOfDetail is less, but not very less, then draw subfield boundary also.
Only if levelOfDetail is greater enough so that user can see detail, draw all the shapes.
Again you can finetune more by drawing rect when level of detail is medium and polygon if levelOfDetail is sufficiently large.

And for the values of levelOfDetail, you have to experiment.

Edit:

Chipsize 50000.000 um, 3844 fields, 246016 subfields, 246016 rects, 246016 polys
Chipsize 100000.000 um, 15625 fields, 1000000 subfields, 1000000 rects, 1000000 polys
Chipsize 125000.000 um, 24336 fields, 1557504 subfields, 1557504 rects, 1557504 polys

As you can see considering worst case, you can represent all your data by 24336 field items (instead of 24336 + 1557504 + 1557504 items! ) and chip demo has 50k items!!
I think your app can be smooth enough if you design it carefully
All the best

[wysota]

FieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout is divided into fields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. Typically 800 by 800 micrometer in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the chip.

This is nice. A rectangle which is child of the scene. Coordinates them come relative to the scene.

SubFieldBoundary:
This object is 'virtual', and the drawing of this is optional, to indicate for the viewer how the pattern layout in the field is divided into subfields.
It is always rectangular (special case quadratic), the same size and shape throughout the whole drawing. The subfield size is ALWAYS smaller or equal to the field size, typically 100 by 100 microns in size. The size is predefined in the file header, and the pattern data comes as position x,y referenced from the upper left corner of the current field.

A rectangle which is child of the field item. The coordinates then come relative to the field.

XRectangle:
Should always be drawn. The X side is longer than the Y side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.

Same as above.

YRectangle:
Should always be drawn. The Y side is longer than the X side. The input data comes as x0,y0,w,h, referenced from the upper left corner of the current subfield.

Same here.

XTrapezoid:
Top and bottom sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,x1,x2,x3,y3, referenced from the upper left corner of the current subfield.

Identified as four points. Child of the subfield. Coordinates relative to the subfield.

YTrapezoid:
Left and right sides parallell. Degenerated into triangles may occur. The input data comes as x0,y0,y1,y2,x3,y3, referenced from the upper left corner of the current subfield.

Same here.

Line:
Just a line from x0,y0 to x1,y1, referenced from the upper left corner of the current subfield.

Child of subfield item.

To sum things up:
You need the following items:
* Field - representing Field and Subfield
* Rectangle - representing X and Y rectangles
* Trapezoid - representing X and Y trapezoids
* Line - representing lines, obviously

These together will make it easier to manipulate all objects. All coordinates will be relative to their parents thus easier to calculate.

[wysota]

You have all of my relevant code in the example. What is missing?

Knowledge what items represent. I don't want to go through all your code and learn it by trial and error. For example I don't know if you really need 4 points to represent a trapezoid or if those trapezoids have somehow "fixed" properties, for example always a 90deg. angle - then you just need 3 points, or fixed lengths.

However, the description in Assistant is a bit cryptic, so I will have to test different settings. Will a higher value improve?

It is nothing you may set. It's a value given by the view to the item's painting routine that tells it how far away the "camera" is from the item. The further, the less details may be seen, so drawing those details becomes obsolete.

I tend to have the same opinion. But the gv framework is so convenient!

I don't think it is an overkill. If you design the scene correctly, it will work fine. With a regular widget you'd probably want to have items as well.

Where does the pos() come in?

It's set to (0,0) by default. The way you do it makes your items very complicated, especially if you'll want later to interact with them somehow. If you have a trapezoid, make it span from (0,0) to the width and height of the trapezoid and them move it into an appropriate position using setPos(). This probably won't yield any performance gain, but will it easier to implement the ShapeItem that doesn't use paths or polygons that are really slow to draw.

Secondly, can you apply levelofDetail to a high-level draw such as scene->addRect();?

The rectangle item probably doesn't use LOD, so no. You have to implement your own item for that. It is most essential for your trapezoids. Simplify things wherever possible.

[bnilsson]

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Please define "reimplement".
I need to tell you I never took the course in C++ so sometimes I have problems with the programming lingo. I just look at examples and try to do the same, and then fill in my own stuff.

It would be great if you could explain with "more words".

[wysota]

Please define "reimplement".

"Subclass and rewrite a method which is defined in the base class and has the same signature (return value, name and arguments)".

Use QGraphicsView and QGraphicsScene instead of MyQGraphics... classes.

[bnilsson]

Ok, I think I understand.
But this removes the possibility to add new methods for the class, right?
Now I have a myQGraphicsView member pointing at the coordinate display in the main window, and I give it a value by the method setCoordDisplay(QLineEdit *in);
Please suggest how I should transfer the result form QGraphicsView::mouseMoveEvent to the coordinate display field in the main window using some alternate method.

If new methods can be added, please advice me on how the header should be written.

[wysota]

But this removes the possibility to add new methods for the class, right?

You can add new methods, just don't reimplement the ones that already exist and do something - namely drawItems().

[bnilsson]

So you mean that the problem was NOT the new class myQGraphicsView, but the fact that drawItems was re-implemented?
If so, what about mouse*Event methods?