QAudioInput & QUdpSocket. Adding Identifier(String) to sent datagrams possible?

[Zandare]

Hi folks!

I am programming a tactical radio application that communicates via LAN. It is basically a voice communication application in which you can set a frequency and only participants who have set the same frequency, can hear and speak to this frequency. Also you can set multiple radios in my application, so that you can listen and talk to multiple frequencies. You hear every frequency you have set, but can only speak on one frequency set as active.

The microphone packages are sent by a qudpsocket to the Broadcast Address, so every participant receives every voice package. This means I have to append the frequency to every voice package sent, so that every receiving radio can filter the incoming datagrams and only playback those with the correct frequencies.

I wrote a line like QString("frequency:100000") to the socket, when the transmission starts, but when more people talk the packages get mixed up. Thats why I need to add a string to every single audio package sent. Though I don't know how, because "QAudioInput::start(socket);" takes care of all the recording and sending via udp.


I hope this states my problem sufficiently and would appreciate any advice.




#include ...

class Audio_receive : public QObject
{
Q_OBJECT

private:
Widget_radio* parent;
QAudioOutput*output;
QUdpSocket* socket;
QIODevice* device;
QTimer* timer;
public:
Audio_receive(Widget_radio* parent);
~Audio_receive();
private slots :
void receive();
void updateRx();
};

#include ...

Audio_send::Audio_send()
{
QAudioFormat format;
format.setCodec("audio/PCM");
format.setSampleRate(16000); //128000
format.setSampleSize(16);
format.setChannelCount(1);
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
input = new QAudioInput(format);

QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
if (!info.isFormatSupported(format))
format = info.nearestFormat(format);

socket = new QUdpSocket();
socket->connectToHost(QHostAddress::LocalHost, Config_manager::getInstance()->getPort());
}

Audio_send::~Audio_send()
{

}

void Audio_send::transmit(bool state)
{
if (state)
{
// Frequency Header
//QByteArray datagram("frequency:100000");
//socket->write(datagram);
input->start(socket);
}
else
input->stop();
}

[ChrisW67]

Don't connect the audio device directly to the socket. Use the QIODevice returned by start() to populate an internal buffer as data becomes available. You then manage sending from that buffer to the socket with a channel indicator, and whatever other metadata is required, at the top of each frame (I would suggest a fixed size integer channel number rather than a string).

[Zandare]

Thank you for the good advice. I am new to qt. I will read about QIodevice and let you know how it went.

[Zandare]

Finally I could spend some time with my project again. I tried to work your ideas in my class, but my transmit function is only executed for one iteration, so not all my voice data is sent. In fact, only the first package.

EDIT: I changed line 63 to: "} while (input->bytesReady() > 4076);" and now data is sent as long as I press the speak button. Though the receiving client only gets a humming noise that varies by changing the audio data size. Voice is also transmitted.. very distorted... :(
Anyway I checked the packages via wireshark and now every transmission contains my frequency string. Is it possible that the humming noise comes from that string? It is not removed yet from the package in the receiving class.

This is what I came up with:


// audio_send.h

class QUdpSocket;
class QAudioInput;
class QIODevice;

class Audio_send : QObject
{
Q_OBJECT

public:
Audio_send();
~Audio_send();
void setRecording(bool);

public slots:
void transmit();

private:
QAudioInput *input;
QUdpSocket *socket;
QIODevice* device;
};


// audio_send.cpp

Audio_send::Audio_send()
{
QAudioFormat format;
format.setCodec("audio/PCM");
format.setSampleRate(16000); //128000
format.setSampleSize(16);
format.setChannelCount(1);
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
input = new QAudioInput(format);
//input->setBufferSize(262144); //TODO: find best buffersize

QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
if (!info.isFormatSupported(format))
format = info.nearestFormat(format);

socket = new QUdpSocket();
socket->connectToHost(QHostAddress::LocalHost, Config_manager::getInstance()->getPort());
}

Audio_send::~Audio_send()
{

}

void Audio_send::transmit()
{
do
{
QByteArray buffer;

buffer.append("frequency:001000000;"); // 20 chars == 20 byte
buffer.append(device->read(qint64(236))); // == 236 byte

socket->write(buffer); // 256 byte
} while (device->bytesAvailable() > 235);
}

void Audio_send::setRecording(bool state)
{
if (state)
{
device = input->start();
connect(device, SIGNAL(readyRead()), this, SLOT(transmit()));
}
else
input->stop();
}

[ChrisW67]

What happens in transmit() when only 30 bytes are available to read?
How about when 300 bytes are ready? What happens to excess bytes? What is the lifetime of your buffer?

Start thnking about collecting data from the mic and sending data as two separate processes connected only by a shared data buffer that is persistent.

You also need to understand Why the User Datagram Protocol (UDP) is sometimes called the Unreliable datagram protocol. What happens to a UDP datagram larger than the maximum transferrable unit on your network interface? Does UDP make guranatees about arrival or order of arrival of transmitted data?

[Zandare]

Well, I believe when 30 bytes are available, nothing is sent. This only happens when a transmission is about to stop in the last package. I think not sending the very last package does not make a difference, because only some milliseconds in the end will be missing.
When there a 300 bytes and more, my understanding is that only the first 236 bytes are taken and transmitted. transmit() then waits until more than 236 bytes have been written to "device" or are available and then sends another package via udp.

I have put the buffer outside of the loop and clear it everytime its content is written to the socket. By processes do you mean threads or a signal slot connection?


void Audio_send::transmit()
{
QByteArray buffer;
while (input->bytesReady() > 500)
{

buffer.append("f:100000000;"); // 12 chars == 12 byte
buffer.append(device->read(qint64(500))); // == 4076 byte

socket->write(buffer); // == 4096 byte datagrams
buffer.clear();
}
}

I know that udp supports no handshake and is simply sending the packages "out". Package loss is not critical in my application in my view. Varying the package size results in different background noises, but everytime my voice is distorted. A package size of 256 byte should be no problem for LAN or WiFi, right?

[ChrisW67]

do { stuff; } while(cond); will always execute stuff once before it evaluates cond. If transmit() is called because 30 bytes arrive from the audio card then they are read and sent with 20 bytes on the front. If those 20 bytes are not being removed at the receiver then they are a lot of noise in short packets. It seems you have changed that logic anyway.

Regardless of whether datagrams get fragmented, which would happen with a 4096 byte datagram, the datagrams can still arrive out of order, several times, or not at all. Not highly likely on a LAN, but certainly possible. If you do not detect these conditions then garbling may result.

I do not mean threads. I would look at logic to retrieve data from the audio device and buffer it, separately from logic that unbuffers data and sends it on a fixed cycle time. The two logics are only related by the content of a shared buffer.

[Zandare]

Finally success. I have clear voice transmission via udp and can filter the packets by the string I put in the buffer. The while loop seemed to be a mistake, as packets got sent multiple times.

I will add the transmitted ip and a package counter to the string later to make sure late packets won't be played. Thank you Chris, you were a big help.

Any further suggestions are still appreciated. Here is my working class:



#include "audio_send.h"

#include "config_manager.h"

#include <QIODevice>
#include <QUdpSocket>
#include <QAudioFormat>
#include <QAudioDeviceInfo>
#include <QAudioInput>

#define BUFFERSIZE 1024

AudioSend::AudioSend()
{
QAudioFormat format;
// Set up the desired format, for example:
format.setSampleRate(16000);
format.setChannelCount(1);
format.setSampleSize(8);
format.setCodec("audio/pcm");
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);


QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
if (!info.isFormatSupported(format))
format = info.nearestFormat(format);

input = new QAudioInput(format, this);
//input->setBufferSize(262144); //TODO: find best buffersize
input->setVolume(1.0);

socket = new QUdpSocket(this);
socket->connectToHost(QHostAddress::LocalHost, ConfigManager::getInstance()->getPort());
}

AudioSend::~AudioSend()
{
delete input;
delete socket;
delete device;
}

void AudioSend::transmit()
{
qDebug("transmit()");
if (!input)
return;

qint64 len = input->bytesReady();
if (len > BUFFERSIZE)
len = BUFFERSIZE;

if (len > 0)
{
QByteArray buffer;
buffer.append("f:100000000;"); // 12 chars == 12 byte
buffer.append(device->read(len));

socket->write(buffer.constData());
}
}

void AudioSend::setRecording(bool state)
{
if (state)
{
qDebug("setRecording()");
device = input->start();
if (device != NULL)
connect(device, SIGNAL(readyRead()), this, SLOT(transmit()));
}
else
{
input->reset();
input->stop();
}
}