Technical indicators over trading curves

[Seamus]

I am converting a Forex trading application from KDChart to Qwt and am wondering:
2012-11-21_14h08_18.jpg

1. How can I hide the horizontal gaps in a candlestick/bar chart (I've marked this with 'Weekend' in the screenshot provided)?
2. Which 'Qwt*Data' classes are best suited to storing calculations made from the candlesticks samples that are to be plotted over or below the chart the calculation formula need to look back (i.e a Simple Moving Average, or MACD)?
3. What is the proper way to handle the last candle in the plot, the last candle changes when the markets are open. At the moment I'm using two trading curves, one for finished bars one for open bar. I had to prepend a fake candle to plot the open bar as Qwt wasn't handling a data-set with only one sample properly.

Details about screenshot:
Left chart is KDChart using QIdentityProxyModels to calculate 5 proxy models from the one data-set.
Right chart is Qwt SVN.
They both contain the same samples and time frame.

[wysota]

How can I hide the horizontal gaps in a candlestick/bar chart (I've marked this with 'Weekend' in the screenshot provided)?

You can subclass QwtPlotCurve and draw whatever you want (and not draw whatever you want) by reimplementing draw* methods.

Which 'Qwt*Data' classes are best suited to storing calculations made from the candlesticks samples that are to be plotted over or below the chart the calculation formula need to look back (i.e a Simple Moving Average, or MACD)?

I did it this way that I had a QAbstractItemModel storing all the data and I had a subclass of QwtSeriesData<QPointF> that calculated and cached desired MA samples from the main data store.

What is the proper way to handle the last candle in the plot, the last candle changes when the markets are open. At the moment I'm using two trading curves, one for finished bars one for open bar. I had to prepend a fake candle to plot the open bar as Qwt wasn't handling a data-set with only one sample properly.

I don't really understand what you mean here. But you can again override draw* methods from QwtPlotCurve and draw what you want. I was reading open/close/min/max data from the abstract item model associated with my series data but you can also store the data directly in QwtSeriesData if you use a structure containing x and those four y values. Then if the open price is lower than 0 then just don't draw it.

[Uwe]

When using Qwt from SVN: what type of plot items are your questions about ?

The screenshots show regular curves ( QwtPlotCurve ) but your questions are about candlestick or bar charts ( QwtPlotTradingCurve/QwtPlotBarChart/QwtPlotMultiBarChart ). F.e. you are asking about how to hide horizontal gaps, what doesn't make much sense for the latter type of items.

Uwe

[Seamus]

I don't really understand what you mean here. But you can again override draw* methods from QwtPlotCurve and draw what you want. I was reading open/close/min/max data from the abstract item model associated with my series data but you can also store the data directly in QwtSeriesData if you use a structure containing x and those four y values. Then if the open price is lower than 0 then just don't draw it.

Currently my app pulls the full (closed) candles from a price server (open candles are unavailable) to mimic an open candle I create a synthetic one from the streaming price data stored in another QVector (with a size() of 1, for one candle), on a candle stick chart the open candle can change an unlimited amount of time during the charts time frame and a closed candle never changes, at the moment I call replot() when I update the last candle and replot() again when I new candle is created. The first replot() I mention only needs to redraw one candle (the last/open one), the second replot() I mention only needs to shift the chart left one candle. Am I missing a more efficient way of of doing this?

When using Qwt from SVN: what type of plot items are your questions about ?

The screenshots show regular curves ( QwtPlotCurve ) but your questions are about candlestick or bar charts ( QwtPlotTradingCurve/QwtPlotBarChart/QwtPlotMultiBarChart ). F.e. you are asking about how to hide horizontal gaps, what doesn't make much sense for the latter type of items.

Uwe

The screenshot is of QwtPlotTradingCurve (really small candles). It's actually more common to hide the horizontal gaps in a trading packages, the gaps are created when a market is closed, the screenshot provided is EURUSD which is only closed over the weekend. When the gaps are included line studies (i.e. Trend lines and channels) can't be drawn accurately. Currently when plotting a candlestick chart showing daily candles over one year the QwtPlotTradingCurve plots space for around 365 candles when it should be closer to 200 (business days in a year) for most financial instruments.

Thanks for your replies guys

[Uwe]

Am I missing a more efficient way of of doing this?

It is possible to avoid replotting the complete curve in the situation, when you update the last candle only, but as long as you don't have a performance issue I would recommend to keep the code simple.

Currently when plotting a candlestick chart showing daily candles over one year the QwtPlotTradingCurve plots space for around 365 candles when it should be closer to 200 (business days in a year) for most financial instruments.

It's not the curve - your samples ( the time values ) decide where to put the candle sticks.

F.e when you return samples with an time coordinate that represents the index of the day in the year without closed days you wouldn't have any gap. Of course you would also need to adjust the tick labels by adding the closed days to represent the real day on the scale.

Uwe

[Seamus]

It's not the curve - your samples ( the time values ) decide where to put the candle sticks.

F.e when you return samples with an time coordinate that represents the index of the day in the year without closed days you wouldn't have any gap. Of course you would also need to adjust the tick labels by adding the closed days to represent the real day on the scale.

Uwe

Okay I changed the QwtOHLCSample::time to use int/index (2012-12-03_21h05_57.jpg) instead or time_t/double (2012-12-03_21h07_44.jpg). That works fine, but all the annotating tools (hLine, plot markers. shapes..etc) get attached to xBottom which doesn't work as expected now that xBottom is not a date (x axis position have to be calculated when time frame on chart is changed).

Is their any other possibilities to hide the gaps and keep the time_t values in 'sample.time'. Could I mess-around with the transform function in the scale map or scale draw to collapse/squash the empty space. Or should I look into replacing QwtOHLCSample with something that will hold both an index and datetime?

[I think] I seen somewhere in the qwt source a mention of vertical invert (of the plotting), is this also available for a horizontal flip/invert, so index zero would be last on the right instead of first on the left (opposite to time scale), it's not really a problem it would just mean less brain work when writing technical indicators, I did write helper functions to mimic the behavior, just wondering if its built-in somewhere?

Thanks

[Uwe]

Could I mess-around with the transform function in the scale map or scale draw to collapse/squash the empty space.

Deriving from QwtTransform is an option. I wouldn't expect it to be too difficult to implement transform/invTransform with gaps: have a look at the playground/scaleengine.

While the time scaleengine from the playground examples might work when you are in ranges of months or weeks you probably have to reimplement divideScale for intervals where you are on days.

[I think] I seen somewhere in the qwt source a mention of vertical invert (of the plotting), is this also available for a horizontal flip/invert, so index zero would be last on the right instead of first on the left (opposite to time scale), it's not really a problem it would just mean less brain work when writing technical indicators, I did write helper functions to mimic the behavior, just wondering if its built-in somewhere?

setAxisScale( 0, 10 ) shows an increasing, setAxisScale( 10, 0 ) a decreasing axis. When you use autoscaling you have to enable the QwtScaleEngine::Inverted flag.

Uwe

[Seamus]

Deriving from QwtTransform is an option. I wouldn't expect it to be too difficult to implement transform/invTransform with gaps: have a look at the playground/scaleengine.

While the time scaleengine from the playground examples might work when you are in ranges of months or weeks you probably have to reimplement divideScale for intervals where you are on days.

I've decided to go with a reimplemented QwtOHLCSample that includes another variable to store the real time and I'll use QwtOHLCSample::time to store the index, I figure it would be more useful as I can re-use the annotating tools on candlestick charts that are n*tick based (instead of time).

setAxisScale( 0, 10 ) shows an increasing, setAxisScale( 10, 0 ) a decreasing axis. When you use autoscaling you have to enable the QwtScaleEngine::Inverted flag.

Setting QwtScaleEngine::Inverted and setAxisScale( 10, 0 ) works good for me, but QwtPlotRescaler doesn't respect the invert.

[Seamus]

Setting QwtScaleEngine::Inverted and setAxisScale( 10, 0 ) works good for me, but QwtPlotRescaler doesn't respect the invert.

I got QwtPlotRescaler working with an my inverted plot:

m_rescaler = new QwtPlotRescaler( canvas() );
m_rescaler->setReferenceAxis( QwtPlot::xBottom );
m_rescaler->setRescalePolicy( QwtPlotRescaler::Expanding );
m_rescaler->setEnabled(true);
m_rescaler->setExpandingDirection( QwtPlot::yRight, QwtPlotRescaler::ExpandBoth );
m_rescaler->setAspectRatio( QwtPlot::yRight, 0.0 );

m_rescaler = new QwtPlotRescaler( canvas() );
m_rescaler->setReferenceAxis( QwtPlot::xBottom );
m_rescaler->setRescalePolicy( QwtPlotRescaler::Expanding );
m_rescaler->setEnabled(true);
m_rescaler->setExpandingDirection( QwtPlot::yRight, QwtPlotRescaler::ExpandBoth );
m_rescaler->setAspectRatio( QwtPlot::yRight, 0.0 );

To copy to clipboard, switch view to plain text mode 

I only had to change

m_rescaler->setExpandingDirection( QwtPlot::xBottom, QwtPlotRescaler::ExpandDown );

m_rescaler->setExpandingDirection( QwtPlot::xBottom, QwtPlotRescaler::ExpandDown );

To copy to clipboard, switch view to plain text mode 

to

m_rescaler->setExpandingDirection( QwtPlot::yRight, QwtPlotRescaler::ExpandBoth );

m_rescaler->setExpandingDirection( QwtPlot::yRight, QwtPlotRescaler::ExpandBoth );

To copy to clipboard, switch view to plain text mode 

[Seamus]

So my candlestick plot now uses QwtPlotRescaler and QwtPlotPanner, when panning on a rescaled chart the y axis doesn't update to visible samples only (like this 2012-12-06_19h30_32.jpg) but I would like it to rescale on every pan (like this 2012-12-06_19h31_26.jpg). This is my current working code

if( !axisAutoScale(yRight) )
    {
        double min, max, begin, end;
 
        begin = qMin( axisScaleDiv(xBottom).lowerBound(), axisScaleDiv(xBottom).upperBound() );
        end = qMax( axisScaleDiv(xBottom).lowerBound(), axisScaleDiv(xBottom).upperBound() );
 
        if( begin < 0 )
            begin = 0;
        if( end < 0 )
            end = 0;
 
        if( m_samples.size() < end )
            end = m_samples.size() - 1;
 
        if(!includeWeekends())
        {
            min = m_samples.at(begin).low;
            max = m_samples.at(begin).high;
        }
        else
        {
            min = m_openCandle.at(begin).low;
            max = m_openCandle.at(begin).high;
        }
 
        for ( int i = begin + 1; i <= end; i++ )
        {
            min = qMin( min, m_samples.at(i).low );
            max = qMax( max, m_samples.at(i).high );
        }
        setAxisScale(yRight, min, max);
    }

if( !axisAutoScale(yRight) )
    {
        double min, max, begin, end;

        begin = qMin( axisScaleDiv(xBottom).lowerBound(), axisScaleDiv(xBottom).upperBound() );
        end = qMax( axisScaleDiv(xBottom).lowerBound(), axisScaleDiv(xBottom).upperBound() );

        if( begin < 0 )
            begin = 0;
        if( end < 0 )
            end = 0;

        if( m_samples.size() < end )
            end = m_samples.size() - 1;

        if(!includeWeekends())
        {
            min = m_samples.at(begin).low;
            max = m_samples.at(begin).high;
        }
        else
        {
            min = m_openCandle.at(begin).low;
            max = m_openCandle.at(begin).high;
        }

        for ( int i = begin + 1; i <= end; i++ )
        {
            min = qMin( min, m_samples.at(i).low );
            max = qMax( max, m_samples.at(i).high );
        }
        setAxisScale(yRight, min, max);
    }

To copy to clipboard, switch view to plain text mode 

This code seems to work okay, but I'm thinking this requirement is common and may be build in to Qwt already? I though it was QwtScaleEngine::Floating but that didn't have any effect.

[Uwe]

Autoscaling and a fixed aspect ratio ( = QwtPlotRescaler ) are contradictory requirements !

What would you expect to happen after panning: shrinking the y axis to the boundaries of the visible area and adjusting the range of the x axis somehow according to the aspect ratio ?

Uwe

[Seamus]

Autoscaling and a fixed aspect ratio ( = QwtPlotRescaler ) are contradictory requirements !

QwtPlotRescaler is being used to make the x axis longer or shorter keeping the candle width and only changing the amount of candles displayed (keeping the index used for the last candle in view). Making a window/plot taller has no effect on x axis but makes the candles (on y axis) taller, making a window wider show more time, and only effects y if the samples brought into view are outside of the y axis bounds, the y bounds are set to the lowest low and highest high or the visible samples.

What would you expect to happen after panning: shrinking the y axis to the boundaries of the visible area and adjusting the range of the x axis somehow according to the aspect ratio ?

QwtPlorPanner is being used to pan back and forwards in time (left and right). Adjusting y axis to high low bounds of visible samples when required.

This is the behavior I've implemented and it works, it just redraws the chart twice I want to get this down to once.

So I think I need to subclass QwtPlotRescaler and QwtPlotPanner, look up the samples highs and lows in between the x axis upper/lower bounds. Then change the plot->setAxisScale() call for y axis in QwtPlotRescaler::updateScales() and QwtPlotPanner::moveCanvas() with the visible samples y bounds.

[Uwe]

I guess overloading QwtPlotPanner::moveCanvas() is enough.

The rescaler is an event filter for resize events of the canvas, that will happen after a replot that has an effect on the layout ( f.e. because of tick label modifications ). To avoid flickering I would disable the rescaler until the panner has done its job. But as the resulting resizeEvent happens asynchronously this is not right after the replot of the panner !

Don't forget to recalculate and assign the new aspect ratio for the rescaler for the new situation after panning.

Uwe

[Seamus]

I've now got TA-Lib abstract interface integrated nicely and outputting to Qwt.

In the screenshot attached the lime triangles are output from one of TA-Libs pattern recognition functions, instead of plotting a symbol for some of the other patterns I may want to draw the candles differently. Can individual candles be targeted/styled differently, or should i use a separate [overlaid] curve for each style.
2013-01-07_14h01_04.jpg
Also, the horizontal lines currently have the labels on the canvas, I would like to have them overlapping (in a solid background box) over the labels on the axis, and have them movable [mouse drag], Is this something QwtWidgetOverlay is used for?

I would also like the candles width to be the same pixel size on any window size, I've tried numbers from the canvas like:

setAxisScale(xBottom, canvas()->contentsRect().width() / 5.0, 0);

setAxisScale(xBottom, canvas()->contentsRect().width() / 5.0, 0);

To copy to clipboard, switch view to plain text mode 

So if that code was run on a canvas with width 100px, 100 / 5 = 20 candles visible, if run on 50px canvas, 50 / 5 = 10 candles visible.
Is they any other numbers, or functions that I can use. I also tried setSymBolWidth() but the candles overlapped. Also after the initial plot I have resizing the window properly working, its just the initial setAxisScale().

Thanks

[Uwe]

Can individual candles be targeted/styled differently, or should i use a separate [overlaid] curve for each style.

You can use QwtPlotTradingCurve::UserSymbol and implement YourTradingCurve::drawUserSymbol().

After checking this hook I decided to extend the API of drawUserSymbol() and also added methods for drawing a candlestick or bar to the public API, so that they can be used as fallback. These modifications have been made in SVN trunk and will be part of Qwt 6.1.0.

Also, the horizontal lines currently have the labels on the canvas, I would like to have them overlapping (in a solid background box) over the labels on the axis, and have them movable [mouse drag], Is this something QwtWidgetOverlay is used for?

Yes, have a look at the itemeditor example.

In your case you don't hide/show the item ( = trading curve ) while dragging the symbol, instead you remove/insert the sample you are moving from the data.

I would also like the candles width to be the same pixel size on any window size, ...

You can limit the width of the candles by setMinSymbolWidth()/setMaxSymbolWidth(). When min and max are the same value the width is fixed. Of course you can't have a fixed height.

Did you try the new QwtDateScaleEngine - I would be interested in having some feedback before Qwt 6.1 final is out.

Uwe

[Seamus]

Yes, have a look at the itemeditor example.

Thats a nifty example, Is QwtWidgetOverlay now the recommended method for adding most non-curve items now?, like shapes, labels, and everything that doesn't need to be redrawn when a replot happens?

You can limit the width of the candles by setMinSymbolWidth()/setMaxSymbolWidth(). When min and max are the same value the width is fixed.

The symbol width isn't affecting the number of candles, I think I need to set a fixed interval width, which would make the candles appear to have a fixed width (I think).

Did you try the new QwtDateScaleEngine - I would be interested in having some feedback before Qwt 6.1 final is out.

I read the changelog after my last rebase, and noticed these classes. Unfortunately, I don't actually use dates in the plot any-more, as I didn't want dates without samples being shown, I use an index for x axis and pass around a reference to the samples for the labels.

I just took a look at the QwtDateScaleEngine docs it mentions "[values] are organised in steps with non constant intervals", so if I'm understanding that correctly I may be able to change back to dates, I will have a closer look.

[Uwe]

Thats a nifty example, Is QwtWidgetOverlay now the recommended method for adding most non-curve items now?, like shapes, labels, and everything that doesn't need to be redrawn when a replot happens?

No the other way round - the intention is to avoid replots. Dragging an object on the canvas is a performance requirement: when the object is behind the mouse movement grahical editing becomes unusable soon ( the problem is erasing the previous position of an object - not drawing it on its new position ).

Note that overlays are only intended for the temporary state while you drag something. They are not part of the scene or the backing store of the canvas and are not seen by classes like QwtPlotPanner or QwtPlotRenderer.

The technical reason for using an overlay is:

1. the canvas gets minimal update regions: the previous mask of the overlay only
2. these regions can be filled from the backing store of the canvas without any replot

On high end system and scenes without much content you might notice the difference, but on systems with low graphics ( f.e. the raspberry ) or when using remote desktops it might be a difference of usable vs. unaccapatble.

The symbol width isn't affecting the number of candles, I think I need to set a fixed interval width, which would make the candles appear to have a fixed width (I think). .

I'm afraid I didn't get the point, but if you want to show less candles depending on the width of the scale interval you could enable QwtPlotItem::ScaleInterest and overload:

QwtPlotItem::updateScaleDiv() or QwtSeriesData< T >::setRectOfInterest( )

QwtPlotItem::updateScaleDiv() or QwtSeriesData< T >::setRectOfInterest( )

To copy to clipboard, switch view to plain text mode 

I just took a look at the QwtDateScaleEngine docs it mentions "[values] are organised in steps with non constant intervals", so if I'm understanding that correctly I may be able to change back to dates, I will have a closer look.

Non constant intervals f.e. means that the scale engine might decide to return ticks for the beginning of each month - what is non constant, because the number of days of the months differ. For the requirement of leaving out weekends the situation is the same as with any other scale engine.

But please have a look - maybe you have an idea how to improve QwtDateScaleEngine with such an option.

Uwe

[Seamus]

Did you try the new QwtDateScaleEngine - I would be interested in having some feedback before Qwt 6.1 final is out.

I was able to change back from index based candles to dates in about five small edits. Keeping in mind that my data doesn't have a start or a real end, and weeks are just another time frame, What areas in particular would you like feedback for?

[Uwe]

Keeping in mind that my data doesn't have a start or a real end, and weeks are just another time frame, What areas in particular would you like feedback for?

QwtDateScaleEngine is a collection of several algorithms for the various units of date/time ( milliseconds -> years ). It has to decide which unit to align to for a date/time range and to find reasonable positions for the ticks. For Qt::LocalTime you have additional problems like DST or locale depending issues like the first day of a week: this all sums up to a quite complex class, that hasn't been used - or even tested - much so far.

When you have a plot where you know the date/time axis you can build your scales manually, but as soon as you need autoscaling or use some sort of navigation ( zooming, panning ... ) QwtDateScaleEngine is the only way to have reasonable scales !

Uwe

PS: QwtDateScaleEngine is not for scales that show elapsed time ( f.e. seconds since starting something ). Unfortunately I was too late for such a scale engine, but it is planned to come with Qwt 6.2.

[Seamus]

QwtDateScaleEngine is the only way to have reasonable scales !

I added in QwtDateScaleEngine, but I haven't quite got the scaling right yet so I don't have any useful comments, I think it will come in handy for the trading analytics portion the my program that I've just started with, or for any aggregate data I suppose. I am using the other Qwt date classes, they have tidied my weekly chart up nicely.

As am using the pseudo date axis (index based), I've re-implemented all the drawing tools in my program (except markers) to use QwtPlotShapeItem, now I'm adding support to them to rescale themselves when the charts time frame changes. So far I've got rectangle behaving correctly with draw implemented as

void PlotShapeItem::draw(
        QPainter *p, const QwtScaleMap &xMap, const QwtScaleMap &yMap, const QRectF &rect) const
{
    const QPointF topLeft =
            QPointF(xMap.transform(m_leftIndex), yMap.transform(boundingRect().topLeft().y()));
    const QPointF bottomRight =
            QPointF(xMap.transform(m_rightIndex), yMap.transform(boundingRect().bottomRight().y()));
 
//    QTransform t;
    p->save();
//    p->setTransform(t);
    p->fillRect(QRectF(topLeft, bottomRight), brush());
//    p->drawPath(shape());
    p->restore();
}

void PlotShapeItem::draw(
        QPainter *p, const QwtScaleMap &xMap, const QwtScaleMap &yMap, const QRectF &rect) const
{
    const QPointF topLeft =
            QPointF(xMap.transform(m_leftIndex), yMap.transform(boundingRect().topLeft().y()));
    const QPointF bottomRight =
            QPointF(xMap.transform(m_rightIndex), yMap.transform(boundingRect().bottomRight().y()));

//    QTransform t;
    p->save();
//    p->setTransform(t);
    p->fillRect(QRectF(topLeft, bottomRight), brush());
//    p->drawPath(shape());
    p->restore();
}

To copy to clipboard, switch view to plain text mode 

m_leftIndex, and m_rightIndex are values I calculate when the time frame changes, they should match up with x values. I want to add support for transforming the shape(), or QPainterPaths. Is QTransform the way to go hear?, I've tried QTransform::scale() but couldn't get anything accurate;