Reading from /dev/hidrawX

[cszawisza]

Hi!

I want to communicate with SixAxis PS3 controller on linux OS, and I really don't know how...
The output from hidraw-dump looks like

01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 EC 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00

01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 EC 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00 
 
01 00 00 00 00 00 81 80 80 80 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 03 EF 16 
00 00 00 00 33 FD 77 01 C0 FA 01 DB 01 92 01 ED 
01 00

To copy to clipboard, switch view to plain text mode 

Blocks are sent one every Xms, I have to capture them and send (only couple values) by TCP socket.
The question is, should I use a QProcess to capture those blocks, or a QIODevice and stdin, QFile or some other class?

What I want to achieve in this application is a signal every end of block (not every bit of data).

What is the best ( good ) way to do that

[wysota]

I would probably use QSocketNotifier and QFile.

[cszawisza]

I try to use QSocketNotifier with no success.
The direct output from /dev/hidraw2 looks in terminal like

$ cat /dev/hidraw2
�{~~��w������{~~��w������{~~��w������{~~��w������{~~��w������

$ cat /dev/hidraw2
�{~~��w������{~~��w������{~~��w������{~~��w������{~~��w������

To copy to clipboard, switch view to plain text mode 

I made a simple app that reads data from /dev/hidraw and print it.

QSocketNotifier *sn;
    QFile *in;
 
MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
 
    in = new QFile ("/dev/hidraw2");
 
    if(in->open(QFile::ReadOnly)) {
        sn = new QSocketNotifier (in->handle(),QSocketNotifier::Read);
        sn->setEnabled(true);
        connect(sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
    }
}
 
void MainWindow::readyRead(){
    qDebug() << in->read(50); // frame if 50 bits long
}

QSocketNotifier *sn;
    QFile *in;

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    in = new QFile ("/dev/hidraw2");

    if(in->open(QFile::ReadOnly)) {
        sn = new QSocketNotifier (in->handle(),QSocketNotifier::Read);
        sn->setEnabled(true);
        connect(sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
    }
}

void MainWindow::readyRead(){
    qDebug() << in->read(50); // frame if 50 bits long
}

To copy to clipboard, switch view to plain text mode 

But the app performance is terrible! And the output is not the same either (it looks likt QT is reading only one of ... ~100 frames)

Whats wrong with my code?

[wysota]

I try to use QSocketNotifier with no success.
The direct output from /dev/hidraw2 looks in terminal like

$ cat /dev/hidraw2
�{~~��w������{~~��w������{~~��w������{~~��w������{~~��w������

$ cat /dev/hidraw2
�{~~��w������{~~��w������{~~��w������{~~��w������{~~��w������

To copy to clipboard, switch view to plain text mode 

Is it the output you expected?

qDebug() << in->read(50); // frame if 50 bits long

qDebug() << in->read(50); // frame if 50 bits long

To copy to clipboard, switch view to plain text mode 

You are reading 50 bytes, not 50 bits.

But the app performance is terrible! And the output is not the same either (it looks likt QT is reading only one of ... ~100 frames)

I don't know why you are expecting to always receive just one frame. An arbitrary number of frames can arrive between subsequent calls to readyRead(). The signal from the notifier means there is something to be read from the descriptor, not that there is 50 bits to be read.

[cszawisza]

qDebug() << in->read(50); // frame if 50 BYTES long

My terrible mistake! Of course bytes, not bits!

Is it the output you expected?

Yes. The raw output looks like this, and the output parsed by hidraw-dupm is like in my first post.
hidraw-dump is very simple https://github.com/aaronp24/QtSixA/b.../hidraw-dump.c I want to do something similar. But using QT

I don't know why you are expecting to always receive just one frame

No I was expecting to receive only one byte, but I can't just readAll, or readLine because the input "file" is a stream that newer ends So this "50" just says to QFile "read only 50 bytes and stop" and that is what I expected.
I chenged this to

void MainWindow::readyRead(){
    static QByteArray buf;
    buf.append(in->read(1));
 
    if (buf.size()==50){
        qDebug()<<buf;
        buf.clear();
    }
}

void MainWindow::readyRead(){
    static QByteArray buf;
    buf.append(in->read(1));

    if (buf.size()==50){
        qDebug()<<buf;
        buf.clear();
    }
}

To copy to clipboard, switch view to plain text mode 

The output is probably OK, I didn't check it because of bad performance...
In normal console (cat hidraw) data flow is constant, something like 1 frame every couple ms, but my app stops for 2s and after that print ~50 frames.

[ChrisW67]

Every time you see readyRead() your code reads only a single byte ignoring the (potentially) hundreds of other bytes that may be available. It then waits for another readyRead() signal, which will not occur until more new bytes become available, to read another single byte and ignore what else is piling up in the socket's buffer. It needs 50 readyRead() signals before you finally output the first block of 50 bytes. If it receives new data on a 50ms cycle then 2500ms will elapse before the first block pops out. See the problem now?

You need to read all the available bytes every time you get readyRead(). Then, for every complete 50 byte block in the buffer you process it and remove it from the buffer. You exit the readyRead handler with an incomplete block still in the buffer (or nothing).

Something like this:

void MainWindow::readyRead(){
    static QByteArray buf;
    buf.append(in->readAll());
 
    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

void MainWindow::readyRead(){
    static QByteArray buf;
    buf.append(in->readAll());

    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

To copy to clipboard, switch view to plain text mode 

[cszawisza]

I can't check it now, but as far as I remember /dev/hidrawX (and other devices) don't have size() (or bytesAvalible()), and readAll means read until stream end's and that’s... newer.

If it receives new data on a 50ms cycle then 2500ms will elapse before the first block pops out

After that time, not one, but couple of blocs pops out. And see my previous post where I read 50bytes form input, effect is the same

One thing that I don't understand is

If it receives new data on a 50ms cycle

Why 50ms?

[ChrisW67]

Why 50ms?

You told us the "Blocks are sent one every Xms". I picked 50 to illustrate why your code takes 2 seconds to output the first packet.

[cszawisza]

I though so, but wanted to be sure

[wysota]

I can't check it now, but as far as I remember /dev/hidrawX (and other devices) don't have size() (or bytesAvalible()), and readAll means read until stream end's and that’s... newer.

readAll() means "read whatever is available". For a sequential device it won't use size(). However readAll() will work just fine.

[cszawisza]

I've done some tests and I can say that readAll() hangs the application (reading never ends), size is always 0, bytes available behave strange... some examples:


First version of function is reading a fixed number of bytes. First in output is "bytes avalible: 0" then application takes a long break and print the rest (like it was waiting to receive 16k bytes, and then start reading)

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->read(50));
 
    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->read(50));

    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

To copy to clipboard, switch view to plain text mode 

output

bytes avalible:  0 
// LONG BREAK
"01000000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ed0101" 
bytes avalible:  16334 
"000000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ec010100" 
bytes avalible:  16284 
"0000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ed01010000"

bytes avalible:  0 
// LONG BREAK
"01000000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ed0101" 
bytes avalible:  16334 
"000000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ec010100" 
bytes avalible:  16284 
"0000000083797c790000000000000000000000000000000000000003ef160000000033fd7701de1e02f8019201ed01010000"

To copy to clipboard, switch view to plain text mode 

After that I tried to use bytesAvailable() to read all bytes in buffer at once. The result was surprising. bytesAvalible started to return only 0 value

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->read(in->bytesAvailable()));
 
    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->read(in->bytesAvailable()));

    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

To copy to clipboard, switch view to plain text mode 

output

bytes avalible:  0 
bytes avalible:  0 
bytes avalible:  0 
(...)

bytes avalible:  0 
bytes avalible:  0 
bytes avalible:  0 
(...)

To copy to clipboard, switch view to plain text mode 

Changing read() to readAll() results in application hang.

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->readAll());
 
    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

void MainWindow::readyRead(){
    static QByteArray buf;
    qDebug() <<"bytes avalible: "<< in->bytesAvailable();
    buf.append(in->readAll());

    while (buf.size() >= 50){
        qDebug() << buf.left(50).toHex() ;
        buf = buf.mid(50);
    }
}

To copy to clipboard, switch view to plain text mode 

output

bytes avalible:  0

bytes avalible:  0

To copy to clipboard, switch view to plain text mode 

The best way to read from this device seems to by reading a fixed number of bytes and then process it, but I can't wait for 16k blocks!

[wysota]

I've done some tests and I can say that readAll() hangs the application (reading never ends), size is always 0, bytes available behave strange...

How are you opening the file?

Does calling isSequential() return true? If not then you didn't open the file correctly. Remember /dev/* files are not regular files.

See QIODevice::isSequential() for more info.

[cszawisza]

isSequential returns true

in = new QFile ("/dev/hidraw2");
 
    if(in->open(QFile::ReadOnly)) {
        qDebug()<< in->isSequential();
        sn = new QSocketNotifier (in->handle(),QSocketNotifier::Read);
        sn->setEnabled(true);
        connect(sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
    }

in = new QFile ("/dev/hidraw2");

    if(in->open(QFile::ReadOnly)) {
        qDebug()<< in->isSequential();
        sn = new QSocketNotifier (in->handle(),QSocketNotifier::Read);
        sn->setEnabled(true);
        connect(sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
    }

To copy to clipboard, switch view to plain text mode 

output

true

true

To copy to clipboard, switch view to plain text mode 

[wysota]

If the file is correctly determined as sequential then calling readAll() when there is anything to be read should not block the call.

I don't know how this particular device driver works. Maybe instead of appending data, it discards it and replaces it with new data. In that case I'd suggest opening the file in unbuffered mode, it might help.

[cszawisza]

Opening file in Unbuffered mode helped in case of using read(50), readAll() still behave like earlier.
The way it works now is satisfying for me, so this topic can be closed. Thanks for your help!
But I'm wondering now why readAll() still don't work...

[wysota]

The problem is using read(50) is just plain wrong unless the device driver guarantees that the file always has just 50 bytes of content (in other words that it always returns just the current sample). This is really something driver-dependent. If you can dig up the documentation of the driver (or better yet inspect the source code) and find out that such a guarantee is there then you're safe. Otherwise don't be suprised when the code fails you tomorrow.

[ChrisW67]

Humour me and try:

#include <QtCore>
#include <fcntl.h>
 
class Reader: public QObject
{
    Q_OBJECT
    int fd;
    QSocketNotifier *notifier;
 
public:
    explicit Reader(QObject *p = 0): QObject(p) {
        fd = open("/dev/hiddraw0", O_RDONLY|O_NONBLOCK);
        if (fd >= 0) {
            notifier = new QSocketNotifier(fd, QSocketNotifier::Read, this);
            connect(notifier, SIGNAL(activated(int)), this, SLOT(handleRead()));
        }
        else
            qFatal("Could not open");
 
    }
public slots:
    void handleRead() {
        static QByteArray buffer;
 
        ssize_t count;
        char block[128];
        notifier->setEnabled(false);
        do {
            count = read(fd, block, 128);
            buffer.append(block, count);
        } while(count > 0);
        qDebug() << "Read done, buffer size" << buffer.size();
        notifier->setEnabled(true);
    }
};
 
int main(int argc, char **argv) 
{
    QCoreApplication app(argc, argv);
    Reader r;
    return app.exec();
}
#include "main.moc"

#include <QtCore>
#include <fcntl.h>

class Reader: public QObject
{
    Q_OBJECT
    int fd;
    QSocketNotifier *notifier;

public:
    explicit Reader(QObject *p = 0): QObject(p) {
        fd = open("/dev/hiddraw0", O_RDONLY|O_NONBLOCK);
        if (fd >= 0) {
            notifier = new QSocketNotifier(fd, QSocketNotifier::Read, this);
            connect(notifier, SIGNAL(activated(int)), this, SLOT(handleRead()));
        }
        else
            qFatal("Could not open");

    }
public slots:
    void handleRead() {
        static QByteArray buffer;

        ssize_t count;
        char block[128];
        notifier->setEnabled(false);
        do {
            count = read(fd, block, 128);
            buffer.append(block, count);
        } while(count > 0);
        qDebug() << "Read done, buffer size" << buffer.size();
        notifier->setEnabled(true);
    }
};

int main(int argc, char **argv) 
{
    QCoreApplication app(argc, argv);
    Reader r;
    return app.exec();
}
#include "main.moc"

To copy to clipboard, switch view to plain text mode 

[cszawisza]

Thank you ChrisW67 your method works perfect! Only thing that I must change is add #include <unistd.h> line and change the hidraw device name!!

To wysota. If you want to test readAll() on this driver you can just listen to your PC mouse (hidraw0 or hidraw1 in my case) it behaves very similar, the main difference is that frame size is not always the same, so my method will not work.
Thank you for help once again!