Wrapping Command Line Application in a Qt GUI?

[soupfly]

Hi all,

This is my first post and I'm a total Qt noob. I've been watching Void Realms videos on YouTube, going through the extensive Qt documentation, and coding away whenever I have spare time. Overall I have been very impressed with Qt.

On to my question: I've been working on making a Qt GUI wrapper for a command line application. My approach is somewhat working and I feel I should ask for some feedback. I am looking for advice regarding potential gotchas as well as best practices. Is there a tutorial on how to best wrap a command line app with Qt? Is there a gold standard example Qt program I should look at as a a reference?

For ease of explaining my situation let's suppose the command line application is for stock quotes. The user types in a stock ticker at the command line, and at some point later the stock quote is printed out (or an error happens). The stock quote application can also asynchronously generate output without first receiving input from the user (i.e. send a special alert message).

1) While the stock quote app itself is open source, I explicitly do not want to modify the source code or link to the application as a lib. The goal is to install my GUI and have it talk to the stock market app without requiring modifications to the existing stock market app in any way. I would like to wrap STDIN and STDOUT. This is because in the future I may want to wrap other stock market apps which are not open source.
2) I want the MainWindow to be completely abstracted from the underlying stock quote app. To do this I've written an IStockQuote Abstract Interface. In theory the GUI should only call functions, signals, and slots exposed by the IStockQuote interface. To this affect my MainWindow does not even have a QProcess() object. All of that has been abstracted away. This will allow more flexibility in the future when I would like to use my GUI as a frontend for other Stock Quote apps, potentially even situations where the IStockQuote is talking directly over the network instead of invoking a local command line app.
3) I have a StockQuote class which inherits from IStockQuote. The StockQuote class opens the command-line application via QProcess(). I connect the readyReadStandardOutput(), readyReadStandardError() to the parseResults() slot within the StockQuote class. parseResults() does some processing, then emits the cleaned up results to a MainWindow slot which displays the output. This technique also works great for the asynchronous output that the underlying stock quote app generates.
4) In MainWindow I have multiple windows. The ConsoleDisplay window replicates the command line app input\output. When the user types in that window I call a IStockQuote interface function that passes the input to the StockQuote class which passes it to the command line tool. Because I know I can grab the results asynchronously (#3) I don't have to poll wait here and everything works great.
5) I'm stumped on how to handle synchronous input\output from other windows in the MainWindow class. Let's suppose the command line application will display the trade volume if the user types in the trade volume command. In my MainWindow I have a TradeVolume window. When the GUI detects a new quote has come it, it automatically executes a IStockQuote->GetTradeVolume(...) call. This correctly passes the request to the underlying command line application and the command line application correctly prints the results. This information should only go to the TradeVolume window and not the ConsoleDisplay window. However I don't know how to synchronously grab this data (it may not be possible). The output of the command is passed to the StockQuote function that is connected to the readyReadStandardOutput() call. If I don't do anything it will be emitted the same as the data in #4 was and will be appended to the wrong window when this is not the desired behavior.
- I can add code in MainWindow to parse this data but that would require MainWindow having knowledge of the underlying command line application which I must avoid.
- I tried having readyReadStandardOutput() writing the output to a results buffer, and then I have the GetTradeVolume() poll() until it sees the results. This doesn't work and the program hangs. I believe this is because everything is single threaded and the readyReadStandardOutput() won't execute until I have returned from my IStockQuote->GetTradeVolume(...) call.
- I can have parseResults() emit to a different MainWindow slot for every result type I want out. This seems like it would be extremely painful and would balloon to a bunch of code. I prefer the simplicity of IStockQuote->GetTradeVolume() returning synchronously. The other issue I have with this is figuring out which TradeVolume window should get the data. The user can have multiple trade volume windows open.
- The other option is to have MainWindow create a new thread\worker\delegate\future\concurrency object (not sure which to use) per TradeVolume window. Then when I poll in GetTradeVolume() it should work. This seems like it will be a brutal amount of coding.

Please let me know where I went wrong. I appreciate any feedback you have. Please let me know if you need any clarification on the above. Thanks in advance.

[anda_skoa]

You have two options:

1) Each window has its own StockQuote object, so you have a bidirectionally uique association between window and process

2) You create a way to associate request and responses, so that any request to the a single StockQuote object can be matched to one of its responses.

Cheers,
_

[soupfly]

Thank you for the quick reply.


1) Each window has its own StockQuote object, so you have a bidirectionally uique association between window and process

Each window has a pointer to the same StockQuote object. I cannot give each window it's own unique StockQuote object due to state (I oversimplified the StockQuote app).


2) You create a way to associate request and responses, so that any request to the a single StockQuote object can be matched to one of its response

I'm attempting to do this. I parse all of the output from the command line app and then put it into a results data structure. I tried polling on the results data structure but again, do the fact that I only have a single thread the polling never completes. I will continue reading on threading.

[anda_skoa]

Each window has a pointer to the same StockQuote object. I cannot give each window it's own unique StockQuote object due to state (I oversimplified the StockQuote app).

You mean the console app can only be run once?



I'm attempting to do this. I parse all of the output from the command line app and then put it into a results data structure. I tried polling on the results data structure but again, do the fact that I only have a single thread the polling never completes.
Not sure what you mean with polling.

QProcess is, and likely any other backend you might implement will be, asynchronous, so there is not and likely won't be any reason for polling.



I will continue reading on threading.

If you ever run into a backend that is synchronous then yes, look into threading.
No point in burdening you with that now.


Cheers,
_

[ChrisW67]

While I understand that you are not currently using direct networking the QNetworkAccessManager and QNetworkReply provides a reasonable example of one way to handle separation of responses. When given a request the (typically single) QNetworkAccessManager (StockQuote) immediately returns a QNetworkReply (IStockQuoteReply with concrete subclasses StockQuoteReply, StockVolumeReply perhaps) that you connect to the handlers that should deal with the reply. The reply object is the one that issues the various signals related to the request that created it. In this way you avoid having all replies go to a single place that must then separate them.

[soupfly]

You mean the console app can only be run once?

Yes there can only be a single console app and each window in the GUI will send\recv messages to the command-line app. The console app lifetime will be the duration of the GUI application. Imagine that there are 4 windows in the GUI app: A, B, C, D. If window A requests data it should only come back to window A and not other windows. In theory I wanted to be able to write something like this for each of the functions:



// psuedo code
::GetTradeData( QString* output )
{
QProcess()->write("GetTradeData\n"); // send a command to the console app
QProcess()->read(output); // read it's response and return it to the caller
}


Sending requests to the command application seems to work great: each window can send data to the command line app via the QProcess()->write() method. Unfortunately QProcess()->read() seems to never return a buffer. I'm not sure if this is because 1) the race condition between the console application outputting data and the GUI calling read() or 2) all command line output is going to the function that I have connected the signals and slots to:



connect(myCommandLineApp, SIGNAL(readyReadStandardOutput()), this, SLOT(recvCommandOutput()));
connect(myCommandLineApp, SIGNAL(readyReadStandardError()), this, SLOT(recvCommandErrorOutput()));


That code works great for the asychronous output that's sometimes outputted by the console app.


Not sure what you mean with polling.

QProcess is, and likely any other backend you might implement will be, asynchronous, so there is not and likely won't be any reason for polling.


Here's where I'm stumped. Because I never received output from the QProcess()->read() call I instead had recvCommandOutput() insert results into a results data structure. The results output have a transaction ID that I can set when I make my query. I then attempted to poll on the results data structure like so looking for my transaction ID:



// psuedo code
::GetTradeData( QString* output )
{
QProcess()->write(transactionId + "GetTradeData\n"); // send a command to the console app

do{

results = CheckForResults( transactionId );

}while( results == NULL)

}


The issue with this is that the polling will loop forever. recvCommandOutput() is never hit once I start looping. Again I believe this is because I have a single MainWindow thread.

Reading between the lines in both of your messages I now believe I am making a mistake in attempting to make the asychronous calls appear sychronous to the GUI. While it would make the GUI code easier to write I feel like I will run into issues in the future.


While I understand that you are not currently using direct networking the QNetworkAccessManager and QNetworkReply provides a reasonable example of one way to handle separation of responses. When given a request the (typically single) QNetworkAccessManager (StockQuote) immediately returns a QNetworkReply (IStockQuoteReply with concrete subclasses StockQuoteReply, StockVolumeReply perhaps) that you connect to the handlers that should deal with the reply. The reply object is the one that issues the various signals related to the request that created it. In this way you avoid having all replies go to a single place that must then separate them.

Yes, the more I think about it this is the correct way to go. Do you know if there is something similar for command line app output? Also, how would you handle the case where there are multiple StockQuote and StockVolume windows open? Again only one window should receive the output.

Thank you both for all your help. I'm beginning to get an idea for what I have to write.

[anda_skoa]

In theory I wanted to be able to write something like this for each of the functions:

That does not really fit your application.
Your current backend is already asynchronous and any future backend has a likely chance of also being asynchronous (e.g. network based).
As a GUI you are not in a situation where you can just block and wait for response, that would result in bad user experience.



Unfortunately QProcess()->read() seems to never return a buffer.

It will if there is something to return.
Which is not likely in the pseudo code above since that would require that the data has been written to the child process, then your process having been suspended long enough for the child to process and write back before your program has had a chance to resume and call read.

Not impossible, but highly unlikely.



Because I never received output from the QProcess()->read() call I instead had recvCommandOutput() insert results into a results data structure. The results output have a transaction ID that I can set when I make my query

Good, that is what I meant with making request and response associated to each other.



The issue with this is that the polling will loop forever. recvCommandOutput() is never hit once I start looping. Again I believe this is because I have a single MainWindow thread.

It is less a matter of having a single thread but exclusively using the single thread for one purpose (looping) so it can't do anything else (like processing events for QProcess).
Basically you've deadlocked yourself.



Reading between the lines in both of your messages I now believe I am making a mistake in attempting to make the asychronous calls appear sychronous to the GUI.

Definitely. A GUI should never assume that I/O is instantaneous, especially nothing that will require communication with a different system (via network, cable, etc).
It is very rare that such communication has deterministic and guaranteed response times that would make blocking the UI a viable approach.

Luckily most I/O in Qt is handled asynchronously so you don't have to do anything special (e.g. threads) to make it non-blocking.



Do you know if there is something similar for command line app output?

No, as application output is very application specific, i.e. how the application separates output of different responses, etc.



Also, how would you handle the case where there are multiple StockQuote and StockVolume windows open?

The window just needs to know which StockQuote object to work with.
In the QNAM example it would just need to know which QNAM to work with.



Again only one window should receive the output.

Since you can associate responses to requests, you just need to pick the right "reply" handler to pass it to, like QNAM knows which QNetworkReply to pass incoming response data to.
Each window only knows its reply object, so it won't be confused by other windows' data.

Cheers,
_