How to set x509 on a QSqlDatabase Connection?

[m3rlin]

Hi,

I have broken several braincells, trying to find out how QSqlDatabase establishes a TCP/IP connection with another computer/server.

What I want to do is the following:
I want to force the socket that QSqlDatabase is using to establish a TCP/IP connection with another computer to use openssl with a predefined client-key.pem, client-x509-cert.pem, and ca-cert.pem.

I am using a MySQL v5.5 database hosted on a different computer e.g. 192.168.1.25. It is configured properly to accept (open)SSL connections.
My QT4 application runs on, and is programmed for Linux (I am using Ubuntu 11.04).
My application can establish a connection with the MySQL server without any problems - though without using SSL.

I have read the MySQL C API and understand that prior to the db.open(), you need to set the mysql_ssl_set() (see also http://developer.qt.nokia.com/forums/viewthread/415)
I have played around with the code but cannot force the connection to use my specific ca, keys and certificates.
I have also read that specifying connection options like I do (db.setConnectOptions("CLIENT_SSL=1;CLIENT_IGNORE_SPACE=1");) is limited to forcing the client to use SSL, but that there is no way to define the SSL specifics?

I hope that anyone has an idea how I can get to the socket - called by QSqlDatabase, when it establishes a connection... or can tell me how to define the connection so that it uses the pems...
Thank you in advance!

[wysota]

What exactly did you try regarding mysql_ssl_set()?

[m3rlin]

wysota,

I followed the example given for the 'sqlite3' driver (see link that I posted).

I linked to the library (libmysql) in the .pro file, and I added the QSqldriver/QMySqlDriver
I declared a QVariant obj and can read the driver handle with qdebug (which returns "MYSQL*").

I tried to play with the code below, after looking into mysql.h but MYSQL stays undeclared whatever I do...?!?
In mysql.h the MYSQL refers to a connection??

This code below allows me to connect to MySQL - but without SSL.

bool p2Sql::createConnection()
{
QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL");
db.setHostName(sMySqlIpAddress);
db.setDatabaseName(sMySqlDbName);
db.setUserName(sMySqlAdminName);
db.setPassword(sMySqlPwd);
db.setConnectOptions("CLIENT_SSL=1;CLIENT_IGNORE_SPACE=1");

if (!db.open())
{
QMessageBox::critical(0, qApp->tr("Cannot open database"),
qApp->tr("Unable to establish a database connection.\n\n"
"Click Cancel to exit."), QMessageBox::Cancel);
return false;
}
return true;
}

I tried the code (below) as I mentioned, I can't get it to work...


QVariant v = db.driver()->handle();
qdebug() << v.typename(); // returns "MYSQL*

MYSQL *handle = static_cast<MYSQL *>(v.data()); //what is MYSQL?, or what is the pointer to handle?
if (handle != NULL) // handle stays undeclared even after including QSqlDriver or QMySqlDriver (which errors on the mysql.h - not found)
{
mysql_ssl_set(handle, p2pro-application-key, p2pro-application-cert, p2pro-ca-cert, NULL, NULL); // QStrings p2pro-* are set in functions in my p2Sql class.
}

I assume that somehow I can add specifc arguments to QSqlDatabase as mentioned in the MySQL C API - needless to mention is not documented in Qt Documentation.
Thanks for any hints or help...

[wysota]

Did you #include <mysql.h> ? Probably not. You need to do that, it's part of mysqlclient development package. And you need to explicitly link to mysqlclient.

[m3rlin]

wysota,

I erroneously included libmysqlclient.a. I changed that to be libmysqlclient.so.16.0.0, and I did indeed not include mysql.h from the include directory of the c-api. So the program compiles, but still no SSL.
I checked that using the code below:
const char *cipher_name;

MYSQL *handle = static_cast<MYSQL *>(v.data());
cipher_name = mysql_get_ssl_cipher(handle);
qDebug() << "The cipher name is " << cipher_name;

I will have to re-visit the GRANT definition for the account and see if REQUIRE X509 is set..

[wysota]

First check if your mysql client library is compiled with SSL support, then check if the server advertises SSL support and finally show your code that sets the key, certificate and ca data.

[m3rlin]

The client library was compiled with ssl support from the get-go, also the server advertises SSL support. I can only establish an SSL connection from the terminal, using either GRANT... REQUIRE x509, or GRANT... REQUIRE SSL. The terminal command: SHOW STATUS LIKE 'Ssl_cipher' replies with the cipher used.

When I attempt to establish a connection through my application, and using SSL, the program reports no SSL connection and no cipher used. The connection is refused. The MySQL Log file is not very clear about the error, it says connection rejected without further specifying a reason.
Do I have to continue to use the C API mysql_real_connect(...) or can I use QSqlDatabase with a command like db.open()?
I assumed that the connection was established by calling the QSqlDatabase object..

[Lesiok]

Did You read about QSqlDatabase::setConnectOptions (http://doc.trolltech.com/4.7/qsqldatabase.html#setConnectOptions) method ?

[wysota]

You can use Qt API once you properly initialize the keys with mysql_ssl_set.

[m3rlin]

@wysota, that's what I thought.
I might be experiencing problems with how the .pem files are presented in QT. I used an absolute path for key, cert, and ca-cert in mysql_ssl_set(handle, "path to key", "path to cert", "path to ca-cert", NULL, NULL);

I also tried adding the keys to QtResources and called them with ":/path/...key.pem or ...cert.pem or ca...pem". I also tried adding the cipher. MySQL still keeps refusing the connection when demanding SLL/x509.


QVariant v = db.driver()->handle();
MYSQL *handle = static_cast<MYSQL *>(v.data());
if (handle != NULL) // handle = mysql*
{
mysql_ssl_set(handle, ".ssl/p2pro-application-key.pem", ".sll/p2pro-application-cert.pem", ".ssl/p2pro-ca-cert.pem", NULL, "DHE-RSA-AES256-SHA");
}

I might need to try the mysql_real_connect() to see if the problem lies with Qt?!? Any other suggestions are welcome. Thx so far for your help.

@Lesiok:
See my code belowm especially this line:

db.setConnectOptions("CLIENT_SSL=1;CLIENT_IGNORE_SPACE=1");

[wysota]

I don't think mysql_ssl_set() expects paths. I would assume that it expects the data itself.

[m3rlin]

@wysota
I hope not MySQL reference on C API says:

20.9.3.67. mysql_ssl_set()

my_bool mysql_ssl_set(MYSQL *mysql, const char *key, const char *cert, const char *ca, const char *capath, const char *cipher)

Description

mysql_ssl_set() is used for establishing secure connections using SSL. It must be called before mysql_real_connect().
mysql_ssl_set() does nothing unless SSL support is enabled in the client library.
mysql is the connection handler returned from mysql_init(). The other parameters are specified as follows:

key is the path name to the key file.
cert is the path name to the certificate file.
ca is the path name to the certificate authority file.
capath is the path name to a directory that contains trusted SSL CA certificates in pem format.
cipher is a list of permissible ciphers to use for SSL encryption.

Any unused SSL parameters may be given as NULL.

Return Values
This function always returns 0. If SSL setup is incorrect, mysql_real_connect() returns an error when you attempt to connect.

[wysota]

Maybe the problem is that the server doesn't trust the client's certificate? Check whether the client even tries to connect to the server using SSL (use a network sniffer or check in server logs or using a firewall). And use absolute paths instead of relative ones as in your code above.

[m3rlin]

@wysota,

That would be a realistic option if I couldn't login with a terminal. Using the following command works fine however with the same certificates:

mysql --ssl-ca ca-cert.pem --ssl-cert p2pro-application-cert.pem --ssl-key p2pro-application-key.pem -u test -h 192.xxx.xxx.xxx -p

I am checking right now if the C-API mysql_real_connect() works with the same certs. If so then the problem lies with Qt. If that also doesn't work - then I will check the MySql C++ connector API. Maybe there is something else that I am overseeing...
Again thx. for your patience, help, and suggestions..

[wysota]

Qt doesn't do anything special, it just calls mysql_real_connect() :) You should really check whether your app tries to connect over ssl and fails or if it totally ignores ssl.

[m3rlin]

Here is what I tried and this DOES work with the same certificates:


#include <QtCore/QCoreApplication>
#include <cstdio>
#include "mysql.h"

int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
MYSQL mysql;
if (mysql_init(&mysql)==NULL)
{
printf("Failed to initate MySQL connection");
}
mysql_ssl_set(&mysql, "../../p2pro/src/newcerts/p2pro-application-key.pem", "../../p2pro/src/newcerts/p2pro-application-cert.pem", "../../p2pro/src/newcerts/ca-cert.pem", NULL, "DHE-RSA-AES256-SHA");
if (!mysql_real_connect( &mysql, "192.xxx.xxx.xxx", "test", "xxxxxxxx", "mysql", 3306, NULL, 0) )
{
printf("Failed to connect to MySQL: Error: %s", mysql_error(&mysql) );
}

printf("Logged on to database sucessfully");
mysql_close(&mysql);

return a.exec();
}


So in summary - I will try to use a reference (this is the only difference) - instead of mysql_ssl_set(handle.... I will use mysql_ssl_set(&handle
If that doesn't work... then I need to find out why in a gui application it doesn't work - but it does in a console app...
Oh djeee.. well tomorrow I will post my results..

[wysota]

Sorry to be a spoil sport but I think you example proves exactly nothing. You don't have any reference anywhere, you are dereferening an object, effectively turning it into a pointer, which is what you already have in your Qt app. I really advise you to take my hint and check if SSL is used or not and whether your paths are correct or not. Those are the only two points of failure here.

[m3rlin]

Indeed, the referencing doesn't matter. But I did prove the following:
1. The path to the certificates is correct.
2. The way they are used in mysql_ssl_set() is correct.
3. That the program is actually using an SSL connection:
a. because MySQL Server accepted the connection, while it was configured to ONLY accept user "test", if he could supply a valid x509.
b. because the SHOW STATUS LIKE 'Ssl_cipher' replies with the cipher used, and it ONLY does that when a SSL connection is established.
c. becuase WireShark reports SSL trafic on port 3306 :)
4. That somehow setting the QSqlDriver or QMySQlDriver handle with my code doesn't work via mysql_ssl_set().

Whom should I contact for an explanation? Where can I find documentation about the underlying structures? Maybe anyone reading this has a similar problem and found a solution? Or is it a Qt shortcoming in arguments that can be specified for QSqlDriver?

[wysota]

1. The path to the certificates is correct.
Not necessarily but that's a minor problem.

4. That somehow setting the QSqlDriver or QMySQlDriver handle with my code doesn't work via mysql_ssl_set().
There is no such handle. There is the MYSQL structure which is a MySql handle which is exactly the same what you are using in your test program.


Whom should I contact for an explanation?
What kind of explanation?


Where can I find documentation about the underlying structures?
Qt's SQL drivers are documented in Qt's docs.


Or is it a Qt shortcoming in arguments that can be specified for QSqlDriver?
Yes, and you need to work around that by calling mysql_ssl_set at the right moment (like directly after calling addDatabase()) :)

[m3rlin]

In my latest test, I now for sure know that QSqlDatabase doesn't adapt its connection to any mysql_ssl_set definitions.

I created another user test2.
I set for test2 user the following GRANT on the test database:

GRANT ALL PRIVILEGES ON test.* FOR 'test'@'192.xxx.xxx.xxx' IDENTIFIED BY 'xxxxxxxxx' REQUIRE SSL;

In my application I used:

mysql_ssl_set(handle, NULL, NULL, NULL, NULL, NULL);
and

db.setConnectOptions("CLIENT_SSL=1;CLIENT_IGNORE_SPACE=1;CLIENT_COMPRESS");

User test2 can connect with SSL (checked by WireShark)
User test2 can also connect when CLIENT_SSL=0 but then no SSL is used (checked by WireShark)

If I however demand that test2 identifies himself by specific x509 (by REQUIRE SUBJECT or ISSUER or a combination of them) then the connection is refused by MySQL Server.
In summary, I must conclude that - QSqlDatabase doesn't allow you to specify WHICH certificate, issed by what Authority must be used. If this conclusion is true then using SSL is very limited with Qt?!?

[wysota]

Can we see complete code of yours used for establishing the connection using mysql_ssl_set?

Ok, I have analyzed Qt's code regarding connecting to MySql. With current implementation you won't be able to call mysql_ssl_set between mysql_init and mysql_real_connect. You should probably report it as a bug. mysql_init() should be called earlier (probably as a result to QSqlDatabase::addDatabase()) so that one can manipulate the structure before opening the connection.

[m3rlin]

Wow beyond expectation, I like you will-never-give-up mentality! Of course I will post my code. Be it tomorrow - having guests now...
Thanks so much!

[wysota]

Before I forget... What I think you should do is that you should either subclass the MySql driver Qt has and reimplement the method for opening the database (which is a bit of work) or you should copy the source code for the plugin, rename the classes and modify the open() routine so that you have the ability to call mysql_ssl_set between mysql_init and mysql_real_connect. Then you can build the new plugin --- either as a plugin (and then deploy it in the proper directory) or as part of your application (and then call QSqlDatabase::registerSqlDriver() to register it). Of course these are only workarounds, the real solution is to modify Qt's MySql driver directly.

[m3rlin]

I have dug deeper into plugins and "all" there is to do to make it work... seems to me like a nice project for Nokia, or for me in a somewhat quite week. You are right, QSqlDriver should be programmed for more flexibility. Is Nokia reading this?
Thanks for all the help.

[aleyer]

Hi!
I looked into the code of mysql driver and QSqlDatabase to make sure that mysql_real_connect() is executed only in the driver->open() function and not before.
So, I used this code to connect to the DB:

QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL");
QVariant v = db.driver()->handle();
if (v.isValid() && qstrcmp(v.typeName(), "MYSQL*")==0)
{
MYSQL *handle = static_cast<MYSQL *>(v.data());
if (handle != NULL)
{
mysql_ssl_set(handle, "client-key.pem",
"client-cert.pem", "ca-cert.pem",
NULL, "DHE-RSA-AES256-SHA");
}
}

db.setHostName(settings.value("database/host").toString());
db.setDatabaseName(settings.value("database/databaseName").toString());
db.setUserName(settings.value("database/userName").toString());
db.setPassword(crypto.decryptToString(settings.val ue("database/password").toString()));
db.setConnectOptions("CLIENT_SSL=1;CLIENT_IGNORE_SPACE=1");
db.open();
Well, it works somehow. First, I had problems with certs on the side of the server (at first, apparmor blocked the access to the cert files, then this (http://forums.mysql.com/read.php?11,400856,401127) problem occured. So, the last time I generated the new certs by tinyca, exported it to zip (key+cert), then removed the pass from the key, and only then it finally works.
The user is set up with REQUIRE SSL option, for some reason the server doesn't accept the user when REQUIRE X509 is selected. The strange thing - it connects even when i change all the parameters to NULL. If I call "mysql --ssl -u test -h 192.168.1.8 -p" it does not authorize.
Server runs on Ubuntu Server 11.10, client - on Mint 12, Qt version 4.7.4, MySQL - 5.1.58-1ubuntu1, OpenSSL 1.0.0e.