Results 1 to 10 of 10

Thread: How to write and read from binary files

  1. #1
    Join Date
    Jun 2011
    Location
    Porto Alegre, Brazil
    Posts
    482
    Thanks
    165
    Thanked 2 Times in 2 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Question How to write and read from binary files

    Hello!

    I'm planning a software that will plot a graph using Qwt or OpenGL (yet not decided) and the data of the graph would be catch by reading a .bin file, but this is the first time I deal with binary files so I'm a "little" bit ignorant on how do I proceed.

    Before writing the actual software, I'm developing a second one to help me in my process of learning how to deal with binary files. This software has two QTextEdit side-by-side and have a "instant connection" between them (i.e. if one of them is edited, the other immediately should change base on what is written in the changed QTextEdit). But one of the QTextEdit is supposed to be connected with a .bin file and originally show what is written in the file in binary code (so one of the QTextEdit will show only a set of 0000 and 1111 and so on), while the other should be its translator, i.e. should show the binary code of the first QTextEdit in ASCII characters.

    Now I already noticed that to work in this project I will use QDataStream and QFile. But the problem is that I'm not familiar with QDataStream::readBytes() and readRayData(), and nether with the write functions. So for example in the beginning, when the QFile is opened and should show in on of the QTextEdit the bin data contained in the .bin file, I don't know which function to use in the place of QTextStream::readAll().

    Could somebody help me in this project? Which function do I use to read a binary code and show them in ASCII (e.g. 01000001 shows 65), how can I give a readAll() and how can I write an ASCII character and the software translate it to binary? And how can I make sure that the user will only write "0"s or "1"s in the QTextEdit for binary data, calling a warning if the user attempts to write a non-binary caracter? (i.e. try to put an ASCII char)


    Thanks!!


    Momergil



    Note: It's something like this I want to do: http://www.roubaixinteractive.com/Pl...ry_To_Text.asp
    Last edited by Momergil; 22nd November 2011 at 14:54. Reason: one more question

  2. #2
    Join Date
    May 2011
    Posts
    239
    Thanks
    4
    Thanked 35 Times in 35 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Symbian S60

    Default Re: How to write and read from binary files

    Even though people talk about "binary files", nobody in his/her right mind would ever think of their contents as being ones and zeros, but what ever expression some data structure or number format might require. Do yourself a favor and stop thinking in ones and zeros and think like a programmer should -- at the highest abstraction level that your tool supports.
    Saving the files in some textual format would make them something like thousand times easier to debug when you change your code, data structures and file contents.

  3. #3
    Join Date
    Aug 2009
    Location
    Belgium
    Posts
    310
    Thanks
    10
    Thanked 31 Times in 25 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: How to write and read from binary files

    Hi,

    Don't do the qtextedit thing, it's just over-complicated and irrelevant.

    How do you know what is in your .bin file ? Is it generated by some datalogger or so, and do you have documentation about that ?

    First you should know how to interpret the .bin file, and then read it byte for byte and manipulate the bytes to turn them into usefull information.

    Regards,
    Marc

  4. #4
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: How to write and read from binary files

    If the files you are using as input are just being treated as a sequence of bytes then you do not need QDataStream, which is more aimed at providing an object serialisation system for Qt applications, you can just use the QIODevice interface directly.

    QString::number can convert a byte to its string representation in any base you like between 2 and 36.
    QString::toLong can do the opposite magic.

    Ultimately I am not sure how useful your experiment will be in getting to your end goal of graphing a data stream from another source.
    Last edited by ChrisW67; 23rd November 2011 at 04:31. Reason: spelling corrections

  5. #5
    Join Date
    Jun 2011
    Location
    Porto Alegre, Brazil
    Posts
    482
    Thanks
    165
    Thanked 2 Times in 2 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: How to write and read from binary files

    Hello!

    Thanks for the answers.

    Even though people talk about "binary files", nobody in his/her right mind would ever think of their contents as being ones and zeros, but what ever expression some data structure or number format might require. Do yourself a favor and stop thinking in ones and zeros and think like a programmer should -- at the highest abstraction level that your tool supports.
    Saving the files in some textual format would make them something like thousand times easier to debug when you change your code, data structures and file contents.
    Sorry, but I'm working for a industry and they choose to use binary files (I imagine for security reasons, since I read bin files are more difficult to fall under reverse engeneering), so there is nothing I can do about it; I must learn how to convert text to bin and how to read bin and transform it into ASCII.

    First you should know how to interpret the .bin file, and then read it byte for byte and manipulate the bytes to turn them into usefull information.

    Regards,
    Marc
    yep, that is what I'm trying to learn! Since my software will open a .bin file and must read it to find the data it must show, I must learn how to read .bin files, and that is my question and the purpose of my software.

    If the files you are using as input are just being treated as a sequence of bytes then you do not need QDataStream, which is more aimed at providing an object serialisation system for Qt applications, you can just use the QIODevice interface directly.

    QString::number can convert a byte to its string representation in any base you like between 2 and 36.
    QString::toLong can do the opposite magic.

    Ultimately I am not sure how useful your experiment will be in getting to your end goal of graphing a data stream from another source.
    Thanks, Chris, for the functions. I only didn't understand the explanation regarding QDataStream. What do you mean by "aimed at providing an object serialisation system for Qt applications"?



    Thanks!


    Added after 40 minutes:


    QString::number can convert a byte to its string representation in any base you like between 2 and 36.
    QString::toLong can do the opposite magic.
    Hello Chris,

    are you sure these are the functions and your explanation is correct? I was able to use QString::toLong to convert from byte to ASCII, but my efforts to use QString::number are being useless.


    Qt Code:
    1. QString text;
    2. bool ok;
    3. text = ui->Text->toPlainText().toLong(&ok,2);
    4. qDebug() << "Text to long:" << text;
    5.  
    6. ui->ASCII->setText(text);
    To copy to clipboard, switch view to plain text mode 
    Last edited by Momergil; 23rd November 2011 at 17:23.

  6. #6
    Join Date
    Aug 2009
    Location
    Belgium
    Posts
    310
    Thanks
    10
    Thanked 31 Times in 25 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: How to write and read from binary files

    Quote Originally Posted by Momergil View Post
    MVR : First you should know how to interpret the .bin file, and then read it byte for byte and manipulate the bytes to turn them into usefull information.

    yep, that is what I'm trying to learn! Since my software will open a .bin file and must read it to find the data it must show, I must learn how to read .bin files, and that is my question and the purpose of my software.
    For reading the file, use a QFile and the read() method. There are version of the read() method to put the data in a array of characters, or in a QByteArray.

    Then you must know how the information is encoded, and use this knowledge to transform the array of bytes. If there is for instance a 16-bit value in it, you must know which byte is first (LSB or MSB) and then add them accordingly (A*256 + B or A + B*256). If the data is generated by a desktop PC, you can do it dirty and just type-cast. E.g. float fTemperature = *(float *)pDataPointer

    Regards,
    Marc

  7. #7
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: How to write and read from binary files

    Sorry, but I'm working for a industry and they choose to use binary files (I imagine for security reasons, since I read bin files are more difficult to fall under reverse engeneering), so there is nothing I can do about it; I must learn how to convert text to bin and how to read bin and transform it into ASCII.
    Security reasons? Rubbish, unless the binary files are encrypted and that is unlikely. The data stream is most likely binary, i.e. not human readable, because that is a convenient, compact form for the source to emit. You need to know what the bytes in the file represent, how to group them into larger units (ints, floats, etc.), and into larger structures. Being able to dump the raw byte stream in a human readable form may help understand what is in the file, but it unlikely to be the end goal.

    For example, a (fictitious) data acquisition unit might gather samples from n inputs and output a variable length data structure like this for each sample set:
    Qt Code:
    1. struct sample {
    2. ulong byteOrder, // 32-bits value 0xFFFE0000
    3. char senderName[16], // ASCII chars only, space padding, no termination NUL
    4. ulong sequence, // 32-bits
    5. ushort n, // 16 bits
    6. double samples[n], // 64-bits per sample, IEE754 format
    7. ulong crc // CRC-32, excludes byteOrder and crc fields
    8. };
    To copy to clipboard, switch view to plain text mode 
    Your problem is how to interpret that at the receiving end.

    Thanks, Chris, for the functions. I only didn't understand the explanation regarding QDataStream. What do you mean by "aimed at providing an object serialisation system for Qt applications"?
    I mean it is a way to write binary data stream in one Qt program that can be read by another Qt program using any version of Qt (within limits set by the writer), with any native machine endianness, and reconstruct the objects (Qt or user) at the other end. That applies to simple types like int or char as well as to complex structures like QString, QList<QPair<QString,QPixmap> > etc.

    You can use it on arbitrary data streams but it may be easier not to depending on the structures involved.

    are you sure these are the functions and your explanation is correct?
    Yes
    I was able to use QString::toLong to convert from byte to ASCII
    Are you sure? Read the documentation for QString::toLong()... Hint: it doesn't return a QString.
    , but my efforts to use QString::number are being useless.
    Probably because you seem very confused.
    Read the documentation for QString::number()... Hint, it doesn't convert strings into numbers.

    Qt Code:
    1. bool ok(false);
    2. QString strBinary("00000010"); // That's 2 in 8 binary bits.
    3. // This is a human-readable STRING representation of a number. This is not what a "binary" file contains.
    4.  
    5. long result = strBinary.toLong(&ok, 2);
    6. qDebug() << result << ok;
    7. // Output: 2 true
    8. // Result is the number represented by the binary digits above, and the conversion was successful.
    9. // The output is a human-readable decimal representation of that number.
    10.  
    11. qDebug() << QString::number(result, 2);
    12. // Output: "10"
    13. // This is a human-readable STRING representing the number (minus leading zeros)
    To copy to clipboard, switch view to plain text mode 
    Last edited by ChrisW67; 24th November 2011 at 01:29.

  8. #8
    Join Date
    Jun 2011
    Location
    Porto Alegre, Brazil
    Posts
    482
    Thanks
    165
    Thanked 2 Times in 2 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: How to write and read from binary files

    Hello!

    First, thanks once again for the answers.

    marcvanriet, I'm doing something like this right now, but first (i.e. before working with encoded .bin files and so forth), it seems more simple to me to work with direct chars, i.e. what I'm doing with the two QTextEdit in the Qt software. So I write binary code in one and the correspondent ASCII should appear in the second. Later I pass that to the bin file (despite, in fact, my code for doing so is actually ready, I'm just not searously using it specially because I still didn't define the binary encoding). Anyway, I'm looking now for more direct reading.

    now about the details you mentioned, well that seriously gave me the impression I actually don't understand what a bin file is! In my simple reasearch I understood that a bin file is like a file but instead of numbers and letters, it contains series of 0 and 1 and ends with ".bin". Nothing special. So everything that I should do is to open and read it as if it was a .txt file, but instead of directly using the text, I should create a reader to divide the 0s and 1s into characters to be used. I'm wrong?

    Chris:

    Security reasons? Rubbish, unless the binary files are encrypted and that is unlikely.
    Yeah, that's it: http://www.iwriteiam.nl/Ha_HTCABFF.html

    Now, Christ, about those two functions, yep, that's what I did. I only have the impression that actually you didn't understand what I mean by reading a string; I created a confusion between the FUNCTION returning a QString (like: QString QString::xx()) and I reading one. So to make myself clearer, I will create an example:

    Lets supose that I want to read the following line of bits:

    010011010110000101110010011101000110100101101110

    What is written on it is "Martin". So I would like to have a function that reads the first 8 numbers, 01001101, which is the "M", and than show it in the QTextEdit "M". But if I write the complete serie of 0 and 1, it will show "Martin". Much like it is done in this wepage: http://www.roubaixinteractive.com/Pl...ry_To_Text.asp

    So in this sense, I must read a binary number (01001101) and show its ASCII meaning (M). Or else the contrary: I write the ASCII character (M) and it will show in the other side its binary value (01001101).

    Of course that in the actual software I would do that process! I would read the binary numbers and work them directly. But I must know what is in the given part of the bin file to know if I put here or there in the software, and to learn that I'm writing this software.

    Unfortunately I can't only use that wepage that, in a first moment, would do the job. After all, in one moment my software will have to actually convert the binaries for something more human-readable (e.g. a given "001100010011001100110000", aka 130, will be used in a method where this 130 is a value for a graphic coordinate, or a given "010001010100001101000111", aka ECG, will be displayed in a part of the interface as title for that graph). So I must know how to do that reading.


    I hope you understood =]

    Momergil

  9. #9
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: How to write and read from binary files

    I understood what you are trying to do, and you already have all the pieces to do that:
    Qt Code:
    1. #include <QtCore>
    2. #include <QDebug>
    3.  
    4. int main(int argc, char *argv[])
    5. {
    6. QCoreApplication app(argc, argv);
    7.  
    8. QString input("010011010110000101110010011101000110100101101110");
    9.  
    10. // Strip anything that is not a '0' or '1' if you want this to be even slightly reliable.
    11. // Assumes multiple of eight characters
    12.  
    13. QString output;
    14. for (int offset = 0; offset < input.length(); offset += 8) {
    15. bool ok(false);
    16. const char c = static_cast<const char>(input.mid(offset, 8).toLong(&ok, 2));
    17. if (ok)
    18. output.append(c);
    19. else
    20. qDebug() << "Conversion failed for" << input.mid(offset, 8);
    21. }
    22.  
    23. qDebug() << output;
    24. }
    To copy to clipboard, switch view to plain text mode 

    Seriously though, you shouldn't need to be inventing a tool to do the job of od on Linux and other UNIXes, or any half decent programmer's editor on Windows (e.g Notepad++ and the hex editor plugin) just to see what bytes are in the input data.

  10. #10
    Join Date
    Aug 2009
    Location
    Belgium
    Posts
    310
    Thanks
    10
    Thanked 31 Times in 25 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: How to write and read from binary files

    now about the details you mentioned, well that seriously gave me the impression I actually don't understand what a bin file is! In my simple reasearch I understood that a bin file is like a file but instead of numbers and letters, it contains series of 0 and 1 and ends with ".bin". Nothing special. So everything that I should do is to open and read it as if it was a .txt file, but instead of directly using the text, I should create a reader to divide the 0s and 1s into characters to be used. I'm wrong?
    Yes, you are wrong.

    A binary file is a series of bytes, which you could e.g. read into an array of unsigned char or a QByteArray. File extension doesn't matter.

    Other data (floats, integers, strings, ...) are encoded as a series of bytes. Only booleans could be combined to have more than 1 in a byte. You never ever try to process a binary file like a stream of 0's and 1's (unless maybe for educational purposes).

    Regards,
    Marc

Similar Threads

  1. Replies: 2
    Last Post: 2nd November 2010, 06:15
  2. Using QT to read and write files over bluetooth FTP
    By deano in forum Qt Programming
    Replies: 0
    Last Post: 15th December 2009, 00:22
  3. Replies: 1
    Last Post: 27th August 2009, 06:41
  4. How can I read Binary files with Qt
    By geo_saleh in forum Qt Programming
    Replies: 2
    Last Post: 16th August 2007, 11:37
  5. How to read/write sets of orderded numbers in binary
    By kaydknight in forum General Programming
    Replies: 3
    Last Post: 12th March 2007, 06:50

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.