SSL handshaking without certificates

[palas]

I'm trying to develop a IM program for LANs using broadcast detection and SSL conections with handshaking based on the public key system. (like amule does)
But when looking at implementations of SSL like the one in QT or the one in QCA I found that they are oriented to CA signed certificates. Which is useless for this application. But even thinking in using self-signed certificates I can't found methods to create those certificates. So I'm stuck.
Is SSL good for this? Is there something more apropiated? How can I create self-signed certificates from a QT application? Is it possible to use SSL handshake without certificates (only keys)?
Thank you for your time.

[wysota]

Certificates are one thing and keys are a different thing. You can use OpenSSL to create both if you want. Certificates can be bogus as long as you don't enforce checking them although at least one side (the server) should have the certificate checked for validity to make man-in-the-middle attacks more difficult to perform. Notice that even ssh before performing session key exchange using the public key does check the fingerprint on the certificate of the server.

[palas]

Certificates can be created with OpenSSL but I can't find the way with QSslCertificate. And I don't want to have to tell the users of my program:
"Please get a self-signed certificate and write here its path...
- If you are in linux write...
- If you are in windows..."
Because windows users like the thinks easier. So if the certificate is needed the program should create it itself or have that option.
But I think that using public key encryption, man in the middle attacks would be useless as long as the public keys are shared in a secure manner (but it isn't a problem in a LAN network).
The other problem is that QSslSocket requires at least certificate in the server side so it won't work without them. (If I'm not wrong)

P.P: What? Does ssh use certificates? I don't think so... Maybe it can. But that's no the usual way to work with it. Is it?

[wysota]

Certificates can be created with OpenSSL but I can't find the way with QSslCertificate. And I don't want to have to tell the users of my program:
"Please get a self-signed certificate and write here its path...
- If you are in linux write...
- If you are in windows..."
To use SSL you are linking against OpenSSL, so you can call OpenSSL functions directly from within your application. There is no need for any external commands.


But I think that using public key encryption, man in the middle attacks would be useless as long as the public keys are shared in a secure manner (but it isn't a problem in a LAN network).
What "secure manner"? :) PKI is the "secure manner" to transfer session keys, so here you'd only move the responsibility elsewhere but still have the problem :) The certificate contains the fingerprint of your key which is part of the mechanism that makes it safe. So the simplest way of "sharing keys in a secure manner" is... issuing a certificate.


P.P: What? Does ssh use certificates? I don't think so... Maybe it can. But that's no the usual way to work with it. Is it?

Sure it is. The server has to authenticate to the client the same way as the client authorizes itself to the server. The fingerprint you have in yout .ssh/known_hosts is the crucial part of the certificate (even if it is not called so directly).

[palas]

You mean calling from the application the program openssl or openssl.exe? It is an option. But I don't think it is a good idea for this project. It will sure work if you compile the application. But I'm thinking in distributing also a windows installer which may be linked statically with openssl for example to avoid installing extra things. (more self contained)
Is there a function of a library or something I can link with for that? It's safer...

And I know how PKI works... But certificates are useless if they aren't signed by a CA because you can't check them. And you will have to transfer them by hand to be sure they are trustworthy. And if you can transfer the certificate you can transfer the key. So why you need them? You can also get the fingerprint straight from the public key. Can't you?

[wysota]

You mean calling from the application the program openssl or openssl.exe?
No, I mean calling functions from the crypto library.

[palas]

Ok. I will check the crypto library. Thank you for your attention and patience.