Using default implementation, intefaces and Qt plugins (not designer plugins)

[high_flyer]

Hi,

so I get to ask a question too :)

I need help with the following problem:
I am working on a logger application which needs to be able to parse various stream formats (from serial port) .
(The logger class is implemented as a QThread.)
The run() method implementation should not be changed by subclasses, and it calls some virtuals - this way various parsing schemes can be used, while conserving a uniform work flow and common tasks.
I now would like to have my application use Qt plugins, so that other parsing implementations will be loaded if available, as plugins.
And here is where I have the problem:
I could supply the default implementation by making a static "default" plugin, which is always "present".
But what do I do with the non default plugins, which will be dynamic? :
I don't want the other plugins to link to the default one, since that will defeat the objective of being able to separately build them, yet I want my non default plugins to inherit the default implementation.
Even if I would link to the default plugin from my non default ones, the application will have multiple definitions errors at link time.

Am I over complicating things?

Any suggestions, pointers, or "why don't just do xxxx" which I might oversaw is greatly appreciated.

[wysota]

so I get to ask a question too :)
Congrats ;)


I don't want the other plugins to link to the default one, since that will defeat the objective of being able to separately build them, yet I want my non default plugins to inherit the default implementation.
Even if I would link to the default plugin from my non default ones, the application will have multiple definitions errors at link time.
I'd say you should provide a library exporting functions your custom plugins can use to communicate with the main part of the application and then link both the plugins and the main application against that library. The library should probably (among other things) contain code for the "default" implementation. The application can then instantiate the default handler and plugins can inherit from that handler without any multiple definitions (as there is a single definition in the shared object).

[high_flyer]

Congrats
Thanks :)

I'll have to think about what you said, and see if I understand it correctly.

However the following:

... and then link both the plugins and the main application against that library
this brings me back to:

I don't want the other plugins to link to the default one, since that will defeat the objective of being able to separately build them,...

Hmm... may be I need to think about it from a different perspective...

I'll think about it and come back.

Thanks.

[tbscope]

With default plugin, do you mean the interface?
If so, you will always need to link to it.


+---------------+
| Plugin 1 |
| Default o---------------+
| | |
| Always loaded | |
+---------------+ | +--------------------+ +------------------+
+-----o Abstract interface o-----o Host application |
+---------------+ | +--------------------+ +------------------+
| Plugin 2 | |
| o---------------+
| Loaded on |
| demand |
+---------------+

In the schematic above, both plugin 1 and plugin 2 implement an interface (which can be abstract). All the plugins and the host application need to link to the interface.

[high_flyer]

Thanks.

With default plugin, do you mean the interface?
No I really mean an implementation of default behavior.
My problem is not the fact that I have to link to the interface - that is clear - rather, that I wanted each plugin to extend the default behavior of the "default" plugin.

But my problem was in part based on my "thinking mode".
I have thought a bit further, and came up with the following solution:
I will embed the "default" behavior in the application, and have the plugin interface not derive from that class, but only supply the extended/different functionality.
Then, in my application I can check for plugins, and if found create an instance and asign it to a member, on which I can then call the extended functionality.

Something like this:


//Main app
void MyDefaultClass::work()
{
//... common work for all plugins (therefore no need to be in the plugin)

if(m_pPlugin)//initialized at app startup
{
m_pPlugin->particularWork();
}
else this->paricularWork();

//... more common work
}


This may seem trivial now, but at first I thought to have all the work done in each plugin - in other words - I was thinking about a fully functional units before - and none in the application which would just call the plugins, instead of just putting the differences in the plugins, and the common work in the application.
This allows each pluin to be free of extenal dependencies, and at the same time, deliver the specific behavior.

I still need to see if on implementation detail level, this is doable, or do my plugins need stuff from the "default" class, and if so, how can I solve it...

[wysota]

You can always have plugins containing factories for your interface and such plugin can take a pointer to the "default" "plugin" and call that plugin for everything it doesn't do on its own. A kind of "decorator" design pattern.


class Extended : public BasicIface {
public:
Extended(BasicIface *basic), m_basic(basic){}
void doSomething() { m_basic->doSomething(); } // reimplemented from BasicIface
};

BasicIface would be pure abstract thus it wouldn't need any implementation. There would be a Basic : public BasicIface implementing the "default" behaviour and passed to the factory function of your real plugin infrastructure. I hope you get what I mean. If not then say so and I'll explain more.

[high_flyer]

There would be a Basic : public BasicIface implementing the "default" behaviour and passed to the factory function of your real plugin infrastructure. I hope you get what I mean. If not then say so and I'll explain more.
I am not sure, let me think about it a bit, and I'll ask if not :)
The pure virtual interface was part of the design in my last post.
This has to be the case anyway, since Qt plugins demand (fully) pure virtual interfaces.

It (what you are suggesting) might however be similar to an improvement I was (still) thinking about to my last post:
What I didn't like in my last post is the fact that the logic (the default behavior) was moved or mixed with the application, which is basically the UI.
So what I think to do now is as follows, and it might be similar to what you are saying:
I will move my default logic to a regular DLL, and have that DLL, work with plugins.
All the plugins, as said, will inherit the same interface.
I think this is the best solution I came up with for this problem so far - but if you think you have a better, do tell! :)

[wysota]

This has to be the case anyway, since Qt plugins demand (fully) pure virtual interfaces.
No, not really. You can have inline methods. And others as well but then you need to link the plugin against the implementation of the interface which defeats the purpose of calling it an interface.

What I didn't like in my last post is the fact that the logic (the default behavior) was moved or mixed with the application, which is basically the UI.
So what I think to do now is as follows, and it might be similar to what you are saying:
I will move my default logic to a regular DLL, and have that DLL, work with plugins.
All the plugins, as said, will inherit the same interface.
I think this is the best solution I came up with for this problem so far - but if you think you have a better, do tell! :)
You can do that, that's ok. Recently I'm leaning more and more towards the design Qt Creator has - with a dumb plugin infrastructure, clever plugins and linking plugins to plugins (as in dynamic linking). Then it's just a matter of exposing proper interfaces by the plugins and you can build whatever application you want on top of it by changing the set of plugins that get loaded into the app. If you want plugins for one specific thing though, it'd certainly be an overkill.

[high_flyer]

No, not really. You can have inline methods. And others as well but then you need to link the plugin against the implementation of the interface which defeats the purpose of calling it an interface.
Are you sure about that since the docs say something else:


Making an application extensible through plugins involves the following steps:

1. Define a set of interfaces (classes with only pure virtual functions) used to talk to the plugins.
2. Use the Q_DECLARE_INTERFACE() macro to tell Qt's meta-object system about the interface.
3. Use QPluginLoader in the application to load the plugins.
4. Use qobject_cast() to test whether a plugin implements a given interface.


Ok, I guess inline would work - and which was the base of my problem - implementing requires linking, which makes one plugin dependent on another, which to me is a principal (definition) conflict.


You can do that, that's ok. Recently I'm leaning more and more towards the design Qt Creator has - with a dumb plugin infrastructure, clever plugins and linking plugins to plugins (as in dynamic linking). Then it's just a matter of exposing proper interfaces by the plugins and you can build whatever application you want on top of it by changing the set of plugins that get loaded into the app. If you want plugins for one specific thing though, it'd certainly be an overkill.
Yes this may be an overkill - but watch out that nroberts wont blow you away with "small project is no excuse to non proper design" ;-)
But in truth, its also a kind of exercise for me - this is a private project, so I can take the time to play with overkill designs ;-)
Even though, I am not sure its an overl kill, since any other way seems just wrong.
This way I will have my logger, and it will always do its job (with out the need to change it), and yet, it will be expandable to new formats if such will come in the future.

Thanks for helping me jugging this in my head!

[wysota]

Are you sure about that since the docs say something else:
Yes, I'm sure. The part in brackets expands the definition of an interface not the C++ requirements for using such an architecture. Since the concept of interface doesn't exist in C++ it is emulated by the use of pure abstract classes with only pure virtual methods in them but nobody says it will suddenly stop working if you implement some of the methods.


Ok, I guess inline would work - and which was the base of my problem - implementing requires linking, which makes one plugin dependent on another, which to me is a principal (definition) conflict.
Not with inline methods. They don't require linking as they are inlined in the code fragment that calls the method not the one that defines it.


Yes this may be an overkill - but watch out that nroberts wont blow you away with "small project is no excuse to non proper design" ;-)
It's by no means improper. It's just simpler. Creator's plugin infrastructure is quite complicated with an object pool (which is really not very proper when talking about C++ principles), plugin dependency resolving, versioning and stuff like that. You don't need it if all you want is to have a single interface exposing two methods.


Even though, I am not sure its an overl kill, since any other way seems just wrong.
Then maybe I didn't explain it clear enough. Creator doesn't use a plain QPluginLoader approach, it builds upon it. You don't need the extra part unless you want your plugins to do something more than just implement one interface. Plain QPluginLoader approach is about "pulling" functionality from plugins (which are dumb) to the main app, Creator works the other way round - the plugins (which are aware of what the main app exposes) "push" functionality to the mainstream.

[high_flyer]

Not with inline methods.
Yes, as I said, I guess this would work. the rest was about implementation which is not inline.


Then maybe I didn't explain it clear enough. Creator doesn't use a plain QPluginLoader approach, it builds upon it. You don't need the extra part unless you want your plugins to do something more than just implement one interface. Plain QPluginLoader approach is about "pulling" functionality from plugins (which are dumb) to the main app, Creator works the other way round - the plugins (which are aware of what the main app exposes) "push" functionality to the mainstream.
Ok, I see what you mean now.
No, I need the "pulling" variant, the dumb plugin approach.
The plugin should not be aware of anything, the less it needs to know or depend on, the better.

I think we nailed it, I'll have a go at implementing this thing. :cool:

Thanks!

[fullmetalcoder]

I will move my default logic to a regular DLL, and have that DLL, work with plugins.
All the plugins, as said, will inherit the same interface.
I think this is the best solution I came up with for this problem so far - but if you think you have a better, do tell! :)
There is a really straightforward alternative which quite surprisingly has not been mentioned so far : full-exploitation of interfaces. Basically it means :
* create an abstract base class for plugins
* for each "chunk" of default logic you would like to expose to plugins, create another abstract class and alter the plugin abstract class so that the application can pass instances of these "logic providers" to the plugin whenever and however appropriate
* implement default logic in the application
* implement plugins independently
* profit...

This does mean you have to think very carefully about your interfaces from the start if don't want to rewrite them every now and then, thereby loosing backward/forward compat but that's generally a good thing to do anyway...

[high_flyer]

Thanks, but :

implement default logic in the application
Your suggestion is very similar to what I wrote in post #5.
And as I wrote later, I see it as bad design, to mix UI and logic, and in general to spread the logic over several modules.

It is much cleaner when each modules logic is self contained.

But thanks never the less! :)

[fullmetalcoder]

Thanks, but :

Your suggestion is very similar to what I wrote in post #5.
And as I wrote later, I see it as bad design, to mix UI and logic, and in general to spread the logic over several modules.

It is much cleaner when each modules logic is self contained.

But thanks never the less! :)
Well, nothing prevents you from implementing that default logic as different plugins, loading them first and passing them to other plugins that might need them. I inferred from your post that these chunks of logic would be some common code you wanted to factor out of the plugins per-se but I might have misunderstood.

There is a fundamental difference between my proposed design and the one discussed in post #5 : in my case the application offers to the plugins some interfaces (possibly coming from other plugins) that can be used to achieve the default behavior and the plugins decide when and how to use them, if at all, to achieve the desired behavior whereas in post #5 each plugin expose specific tasks it can do to alter the default behavior and the application decide to execute them or not at some specific points of the default codepath.

[wysota]

We have to understand the difference between a plugin and a regular library. There are two practical differences:
1) adding a new plugin doesn't require the main application to be recompiled,
2) loading of a plugin is optional while loading a linked library is mandatory (which makes their dependencies optional or mandatory as well).

An indirect consequence of the above two is that you can ship the same application binary to different clients with different sets of plugins but with the same sets of linked libraries.

It doesn't make sense to use plugins "just because you can". The sole fact that one wants to have different "handlers" doesn't require them to be implemented as plugins. If you want more than one handler, you can build them all into the application (or link as libraries) and through the configuration of the application decide which one will be used. If you always have the same set of handlers available and loading them all doesn't cause any extra stress to the application, I would consider not using plugins at all and instead just have a couple of implementations of the "plugin" interface directly in the application.

To give an example - take a look at Designer plugins for custom widgets. They are implemented as plugins because it doesn't require to rebuild Designer just to add support for new widgets. But when you use those widgets in your own applications, it's not done through plugins (code for widget classes needs to be linked to the application) although it would be possible to do so (see QFormBuilder). It just doesn't yield any benefits to have it done via plugins. It would however impose some disadvantages.

[high_flyer]

It doesn't make sense to use plugins "just because you can". The sole fact that one wants to have different "handlers" doesn't require them to be implemented as plugins. If you want more than one handler, you can build them all into the application (or link as libraries) and through the configuration of the application decide which one will be used. If you always have the same set of handlers available and loading them all doesn't cause any extra stress to the application, I would consider not using plugins at all and instead just have a couple of implementations of the "plugin" interface directly in the application.

Exactly.
Which is what brought me to the design in post #7.
The DLL is always present (the default logic that is always there), and IF plug ins are present, they will extend the default behavior.
So the DLL is not optional, but the plugins are.

[wysota]

Just decide whether it is a bigger pain to provide a plugin architecture or to recompile the application. If the former then maybe it's not worth investing in plugins.

[high_flyer]

In this specific case, its probably more or less the same pain.
But since I find it more interesting with the plugins, (and a better design) I'll go with that :-)