Results 1 to 13 of 13

Thread: Parse Text File with Qt

  1. #1
    Join Date
    Jun 2009
    Posts
    11
    Thanks
    5

    Question Parse Text File with Qt

    Hi All,

    I need to parse the text file below and process the data using Qt. However, I do not have such an experience and do not know what is best way to handle this job in Qt.

    Here is an example of my text file:

    Frame(BoardList(Board(ID("Board1"),
    TestPointList((NO(1),
    X(1m),
    Y(1m),
    Z(0)),
    (No(2),
    X(2m),
    Y(2m),
    Z(0)),
    (No(3),
    X(2.5m),
    Y(2.5m),
    Z(0))),
    NetList((ID("Net1"),
    PinList((ID("Pin1")),
    (ID("Pin2")))),
    (ID("Net2"),
    PinList((ID("Pin3")),
    (ID("Pin4")),
    (ID("Pin5")))),
    (ID("Net3")))),
    (Board(ID("Board2"),
    ...
    )))

    This text file is used to describe the data of PCB boards. Basically, each file contains a frame, which contains a list of board. Each board contains a list of test points and a list of Nets. Each Net contains a list of Pins.

    In the example above, there are two boards. The first board "board1" contains three test points ( (1,1m,1m,0),(2,2m,2m,0) and (3, 2.5m, 2.5m, 0) ) and three nets. The first net "Net1" contains two pins ("Pin1" and "Pin2"). The second net "Net2" contains three pins ("Pin3", "Pin2", "Pin3"). The third net "Net3" contains no pin.
    The content of the second board is omitted here.

    My job is to
    1) Parse a given text file and get the data (e.g. boards, test points and nets) out of it.
    2) Process the data in the memory. For example, add/remove/modify a net or a test point.
    3) Save the updated data back into the text file.

    Any guide will be appreciated. Thanks.


    Rice

  2. #2
    Join Date
    Sep 2009
    Location
    UK
    Posts
    2,447
    Thanks
    6
    Thanked 348 Times in 333 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Parse Text File with Qt

    Although you could probably use QFile and QString for crude parsing, a proper lexical analyser would be better:

    http://dinosaur.compilertools.net/lex/index.html

  3. The following user says thank you to squidge for this useful post:

    rhf417 (17th May 2010)

  4. #3
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Parse Text File with Qt

    Given that the source file has some grammatical structure you could use the traditional lex/yacc approach to build a parser. You could also try QLALR for the Qt experience.

    http://labs.trolltech.com/page/Projects/Compilers/QLALR
    http://qt.nokia.com/developer/qtquar...alr-adventures

  5. The following user says thank you to ChrisW67 for this useful post:

    rhf417 (17th May 2010)

  6. #4
    Join Date
    Jun 2009
    Posts
    11
    Thanks
    5

    Default Re: Parse Text File with Qt

    Is there an easier way? For example, will QRegExp be capable of doing my job?
    I do not want to parse the text file from scratch (e.g. use QFile and QString). I am interested in QLALR. But it is hard to start with QLALR due to the poor documentations.
    Traditional lex/yacc is powerful. However, it requires additional learning process for a person (like me) who never use it before. I am not sure whether it is worth learning lex/yacc for this simple job I have.

    BTW: My working environment is Visual C++ 2008 and Qt4.4.

  7. #5
    Join Date
    Sep 2009
    Location
    UK
    Posts
    2,447
    Thanks
    6
    Thanked 348 Times in 333 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Parse Text File with Qt

    I don't know, from the file it seems like a flexible language, so you can't really use a regexp IMO. I'm guessing the above file isn't everything you need to support, and other files may be written out slightly differently.

    What would be better than the above file is the file format specification document; then you know what you need to support.

  8. #6
    Join Date
    Jan 2006
    Location
    Belgium
    Posts
    1,938
    Thanked 268 Times in 268 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows
    Wiki edits
    20

    Default Re: Parse Text File with Qt

    Here's an example.
    Of course, I made this in just a couple of minutes to demonstrate how you can parse this file.
    There are tons of places that can benefit from improvements, but that is up to you.
    And I only used a few elements of your file, not all, that too is up to you.

    MainWindow.h
    Qt Code:
    1. #ifndef MAINWINDOW_H
    2. #define MAINWINDOW_H
    3.  
    4. #include <QMainWindow>
    5.  
    6. namespace Ui {
    7. class MainWindow;
    8. }
    9.  
    10. struct BoardElement {
    11. QString id;
    12. };
    13.  
    14. struct FrameElement {
    15. QList<BoardElement *> boardElements;
    16. };
    17.  
    18. class MainWindow : public QMainWindow {
    19. Q_OBJECT
    20. public:
    21. MainWindow(QWidget *parent = 0);
    22. ~MainWindow();
    23.  
    24. protected:
    25. void changeEvent(QEvent *e);
    26.  
    27. private slots:
    28. void startParsing();
    29.  
    30. private:
    31. Ui::MainWindow *ui;
    32. };
    33.  
    34. #endif // MAINWINDOW_H
    To copy to clipboard, switch view to plain text mode 

    MainWindow.cpp
    Qt Code:
    1. #include "mainwindow.h"
    2. #include "ui_mainwindow.h"
    3.  
    4. #include <QFile>
    5. #include <QTextStream>
    6. #include <QDebug>
    7.  
    8. MainWindow::MainWindow(QWidget *parent) :
    9. QMainWindow(parent),
    10. ui(new Ui::MainWindow)
    11. {
    12. ui->setupUi(this);
    13.  
    14. connect(ui->pushButton, SIGNAL(clicked()), this, SLOT(startParsing()));
    15. }
    16.  
    17. MainWindow::~MainWindow()
    18. {
    19. delete ui;
    20. }
    21.  
    22. void MainWindow::changeEvent(QEvent *e)
    23. {
    24. QMainWindow::changeEvent(e);
    25. switch (e->type()) {
    26. case QEvent::LanguageChange:
    27. ui->retranslateUi(this);
    28. break;
    29. default:
    30. break;
    31. }
    32. }
    33.  
    34. void MainWindow::startParsing()
    35. {
    36. QFile file("./file.txt");
    37.  
    38. if (!file.open(QFile::ReadOnly))
    39. return;
    40.  
    41. QTextStream ts(&file);
    42.  
    43. FrameElement *frameElement;
    44. BoardElement *boardElement;
    45.  
    46. QChar ch;
    47. int depth = 0;
    48. int currentState = 0;
    49. //states tell the parser which element it is parsing.
    50. // 1 = Frame element
    51. // 2 = BoardList element
    52. // 3 = Board element
    53. // 4 = Id element
    54. // 5 = ame element (between "")
    55.  
    56. QList<int> stateList;
    57.  
    58. QString element = "";
    59. do {
    60. ts >> ch;
    61.  
    62. if (ch == '(') {
    63. ++depth;
    64.  
    65. if (element == "Frame" || element == "frame") {
    66. stateList.append(1);
    67. frameElement = new FrameElement;
    68. qDebug() << "Found a Frame element";
    69. }
    70. if (element == "BoardList" || element == "boardlist") {
    71. stateList.append(2);
    72. qDebug() << "Found a BoardList element";
    73. }
    74. if (element == "Board" || element == "board") {
    75. stateList.append(3);
    76. if (!frameElement) {
    77. qDebug() << "Parsing error: no frame element found!";
    78. return;
    79. }
    80. boardElement = new BoardElement;
    81. frameElement->boardElements.append(boardElement);
    82. qDebug() << "Found a Board element";
    83. }
    84. if (element == "Id" || element == "id" || element == "ID") {
    85. stateList.append(4);
    86. qDebug() << "Found an Id element";
    87. }
    88.  
    89.  
    90. element = "";
    91. }
    92. else if (ch == ')') {
    93. --depth;
    94.  
    95. if (element.startsWith("\"")) {
    96. if (stateList.count() < 2) {
    97. //parsing error, the list should have at least 2 items
    98. return;
    99. }
    100. int currentState = stateList.at(stateList.count() - 2); // Go back two states, the previous is the id, the one before the id
    101. // tells wich object receives the name
    102. qDebug() << "Found a name element in state " << currentState;
    103.  
    104. element = element.section("\"",1,1);
    105. if (currentState == 3) {
    106. boardElement->id = element;
    107. qDebug() << "Set the board element id to:" << boardElement->id;
    108. }
    109.  
    110. }
    111.  
    112. if (!stateList.isEmpty())
    113. stateList.removeLast();
    114. element = "";
    115. }
    116. else if (ch == '\r' || ch == '\n' || ch == ' ') {
    117. // do nothing
    118. // If you are able to use spaces in names, then create a boolean value to check if you're parsing a name.
    119. // If you do (boolean = true), do not swallow spaces but add them to the parsed string.
    120. // If you don't (boolean = false), swallow the spaces
    121. }
    122. else {
    123. element += ch;
    124. }
    125. } while (!ts.atEnd());
    126.  
    127. file.close();
    128. }
    To copy to clipboard, switch view to plain text mode 

  9. The following user says thank you to tbscope for this useful post:

    rhf417 (17th May 2010)

  10. #7
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Parse Text File with Qt

    The problem with this parser is that you expect the file to have all those elements - it is fitted to this exact file (or to be precise to the exact keywords used here). But this is not sufficient in general case.

    The language used here is predicate-based. It seems like prolog, lisp or lua (and probably dozen others that I am not familiar with). Predicates themselves are surely parsable by regexp but lexical analisys is one thing (and this could be done by QRegExp or even using code suggested by tbscope) but semantical one is a completely different thing. In the end you need to have a proper parser - either your own (be it hand-crafted or auto-generated) or from a library supporting this actual language (whatever it is).
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  11. The following user says thank you to wysota for this useful post:

    rhf417 (17th May 2010)

  12. #8
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Parse Text File with Qt

    Quote Originally Posted by rhf417 View Post
    Is there an easier way? For example, will QRegExp be capable of doing my job?
    Bits of the task certainly. However, matching whole, nested compund structures where deelimiters must be balanced rapidly becomes very difficult.
    I do not want to parse the text file from scratch (e.g. use QFile and QString). I am interested in QLALR. But it is hard to start with QLALR due to the poor documentations.
    Traditional lex/yacc is powerful. However, it requires additional learning process for a person (like me) who never use it before. I am not sure whether it is worth learning lex/yacc for this simple job I have.
    Fair call on the QLALR documentation... there's a high level of assumed knowledge in the examples that are around. You might like the better documented GOLD Parsing System.

    I think the effort of learning lex/yacc, or the theory of operation, is well worth it for the potential to help in many future projects. However, time and effort required to learn this must be offset by your circumstance. Are you parsing one file, or thousands? Are they very variable (numbers of net-lists etc.)? Is the grammar we can see only a subset of a larger possible grammar?

  13. The following user says thank you to ChrisW67 for this useful post:

    rhf417 (17th May 2010)

  14. #9
    Join Date
    Jun 2009
    Posts
    11
    Thanks
    5

    Default Re: Parse Text File with Qt

    I did some research on the parser generator these days and found a good tool - "Visual Parser++" which is easy to learn. I am working under windows and C++. I feel that Visual Parser++ is very convenient to use. I have parsed my text file with it successfully. The only drawback is that there will be no support since Sandstone does not exist anymore.

    Basically, the text file shown above is the only file I need to handle. I only need to parse one file at a time, not thousand of them. The grammar you can see here is not all, but the rest are very similar. I would like to spend more time to learn Lex/Yacc and QLALR later, but not now, I guess.

    For me, tbscope's method is the one I would like to apply to my current project. It looks quite straightforward and does not require any additional tools. Thank you very much. Thanks all for all the valuable advices.

  15. #10
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Parse Text File with Qt

    Quote Originally Posted by rhf417 View Post
    Basically, the text file shown above is the only file I need to handle. I only need to parse one file at a time, not thousand of them.
    It's not the number of files you wish to parse at a time that may be a problem. The problem is you may want to parse different files (with different contents) in general.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  16. #11
    Join Date
    Jun 2009
    Posts
    11
    Thanks
    5

    Default Re: Parse Text File with Qt

    Quote Originally Posted by wysota View Post
    It's not the number of files you wish to parse at a time that may be a problem. The problem is you may want to parse different files (with different contents) in general.

    I understand. If there are many different types of files to parse, a parser generator would be the better choice.

  17. #12
    Join Date
    Sep 2009
    Location
    UK
    Posts
    2,447
    Thanks
    6
    Thanked 348 Times in 333 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Parse Text File with Qt

    Be careful and do your research. I've fallen into this trap before - build a parser for the files I had that worked brilliantly. Then, a month later I had some more files and it only parsed one of them correctly. I didn't then have the time to fix the program properly to parse them all, so hacked it up to make it work. Do that a few times and you have a nightmare of an application to maintain or a range of application that parse different variations.

    Its so much easier to do it right from the beginning.

  18. #13
    Join Date
    Jan 2013
    Posts
    5
    Qt products
    Qt4 Qt5
    Platforms
    MacOS X Unix/X11 Windows Android

    Default Re: Parse Text File with Qt

    Does anyone know where to find an old copy of Visual Parse++? It is a great tool to teach parsing. We have been searching for it, and all we can find are URLs that point to the no longer existing Sandstone site.

Similar Threads

  1. parse items of QTreeWidget into file/QSettings
    By Mystical Groovy in forum Qt Programming
    Replies: 3
    Last Post: 8th November 2009, 16:52
  2. Best way to load and parse an HTML file ??
    By tuthmosis in forum Qt Programming
    Replies: 8
    Last Post: 23rd August 2008, 11:06
  3. Replies: 13
    Last Post: 21st June 2006, 21:22
  4. How to parse this XML file ?
    By probine in forum Qt Programming
    Replies: 7
    Last Post: 4th April 2006, 09:05
  5. Parse a pickle file
    By karye in forum Qt Programming
    Replies: 1
    Last Post: 19th January 2006, 17:02

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.