SCP/SFTP with libqxt

[raven-worx]

Hi,

i was wondering if it would be possible to implement SCP and SFTP filetransfer with libqtx, since libqxt uses libssh?!
A short example would also be very helpful

thanks in advance!

[ars]

Hello raven-worx,

I've implemented scp/sftp file transfer using directly libssh2 (see http://www.libssh2.org/). I started examining their examples and it was easy to transfer their C code examples into a class usable in my Qt based environment.
Sorry, I can't help you with libqxt. But if you are interested in getting scp/sftp into your application, you might give libssh2 a try. Maybe their examples can help you finding a way to work with libqxt.

Best regards
ars

[raven-worx]

yes, i was looking at libssh2, but i had some troubles compiling it with qmake and mingw32, so i wanted to give libqxt a try.

[ars]

I use version 1.2.7 with mingw. For compiling libssh2, there's a win32 sub dir containing Makefile for mingw. You need to edit file Makefile.win32 to adjust it according your installation: Compile with zlib, path, path to openssl and so on. For compiling libssh2 there's no need for qmake. Just run your make utility in win32 directory.
After successful compilation copy libraries, dlls and include files to your compiler installation and then you can use it in your Qt projects as you would do with any other external library.

[raven-worx]

i tried that, but i get the following error (after all object files have been sucessfully created):

.......
gcc -O2 -DNDEBUG -DWIN32 -DLIBSSH2_WIN32 -fno-strict-aliasing -Wall -I. -I../i
nclude -I../../openssl-1.0.0c/include -I../../openssl-1.0.0c/include/openssl -I.
./../zlib-1.2.5 -c ../src/global.c -o release/global.o
.......
Creating libssh2.a
mingw32-make: *** No rule to make target `release/libssh2.res', needed by `libss
h2.dll'. Stop.

edit: also only a empty version.inc-File is created --> same error like above with version.inc, it also seems that awk isnt working properly.

[ars]

Oh yes,

sorry, I forgot to mention: I got the same error message. To get rid of it, I just removed the dependency of the library from the resource file. Just replace line

OBJL = $(OBJS) $(OBJDIR)/$(TARGET).res

in Makefile.win32 with

OBJL = $(OBJS)

Then the library & dll should build. Ok, you don't have version information in the library.

[raven-worx]

the libssh-Makefile seems to work now... but now i have troubles compiling openssl with mingw. The call of openssl's mingw32.bat end up in:

......
perl crypto\sha\asm\sha1-586.pl -DL_ENDIAN -DDSO_WIN32 -fomit-frame-pointer -O3
-mcpu=i486 -Wall -DBN_ASM -DMD5_ASM -DSHA1_ASM -DOPENSSL_BN_ASM_PART_WORDS -DOP
ENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE -DOPENSSL_NO_
DYNAMIC_ENGINE >tmp\sha1-586.asm
Pick one target type from
elf - Linux, FreeBSD, Solaris x86, etc.
a.out - DJGPP, elder OpenBSD, etc.
coff - GAS/COFF such as Win32 targets
win32n - Windows 95/Windows NT NASM format
nw-nasm - NetWare NASM format
macosx - Mac OS X
mingw32-make: *** [tmp\sha1-586.asm] Error 1

for zlib i just downloaded the precompiled binaries from the website.

[raven-worx]

ok... it seems that this is a bug of openssl1.0.0+ with mingw. I used 0.9.8 and it compiles without erros now (with a little code-adjustment).

so openssl and zlib should be fine by now. The libssh2-compilation just has two last errors ... i hope:

.....
Creating libssh2.a
.\libssh2-1.2.7\win32\libssh2.a --> could not be found?!
Linking libssh2.dll
\libssh2-1.2.7\win32\libssh2.dll --> could not be found?!
Creating library file: libssh2dll.a

it looks very strange, because it say "creating libssh2.a" and in the nex line the file libssh2.a could not be found ..

[ars]

Hello,

I didn't observe this behavior. Looking into the makefile I got an idea what it could be (but it's just guessing):

$(TARGET).$(LIBEXT): $(OBJS)
        @echo Creating $@
        @-$(RM) $@
        @$(AR) $(ARFLAGS) $@ $^
ifdef RANLIB
        @$(RANLIB) $@
endif
 
$(TARGET).dll $(TARGET)dll.a: $(OBJL)
        @echo Linking $@
        @-$(RM) $@
        @$(LD) $(LDFLAGS) $^ -o $@ $(LIBPATH) $(LDLIBS)

$(TARGET).$(LIBEXT): $(OBJS)
        @echo Creating $@
        @-$(RM) $@
        @$(AR) $(ARFLAGS) $@ $^
ifdef RANLIB
        @$(RANLIB) $@
endif

$(TARGET).dll $(TARGET)dll.a: $(OBJL)
        @echo Linking $@
        @-$(RM) $@
        @$(LD) $(LDFLAGS) $^ -o $@ $(LIBPATH) $(LDLIBS)

To copy to clipboard, switch view to plain text mode 

When you remove the @ in front of the commands, you get the following:

$(TARGET).$(LIBEXT): $(OBJS)
        echo Creating $@
        -$(RM) $@
        $(AR) $(ARFLAGS) $@ $^
ifdef RANLIB
        $(RANLIB) $@
endif
 
$(TARGET).dll $(TARGET)dll.a: $(OBJL)
        echo Linking $@
        -$(RM) $@
        $(LD) $(LDFLAGS) $^ -o $@ $(LIBPATH) $(LDLIBS)

$(TARGET).$(LIBEXT): $(OBJS)
        echo Creating $@
        -$(RM) $@
        $(AR) $(ARFLAGS) $@ $^
ifdef RANLIB
        $(RANLIB) $@
endif

$(TARGET).dll $(TARGET)dll.a: $(OBJL)
        echo Linking $@
        -$(RM) $@
        $(LD) $(LDFLAGS) $^ -o $@ $(LIBPATH) $(LDLIBS)

To copy to clipboard, switch view to plain text mode 

I suggest to make clean and then run the modified makefile again. In my environment this shows

echo Linking libssh2.dll
Linking libssh2.dll
rm -f libssh2.dll
gcc -s -shared -Wl,--out-impl......................

echo Linking libssh2.dll
Linking libssh2.dll
rm -f libssh2.dll
gcc -s -shared -Wl,--out-impl......................

To copy to clipboard, switch view to plain text mode 

So I see the output Linking ... and that make tries to remove a possibly existing libssh2.dll. But as we did make clean before, there is no dll to be removed. So I guess that rm command complains about non existing file. Option -f should suppress this. So I suggest to try rm non_existing_file and rm -f non_existing_file on your shell and have a look at the output. The last one should silently do nothing (but I assume it will complain).

In the Makefile, the rm command is prefixed with -, so make will continue building even if rm returns an error code.

If my assumptions are true, you should get the libs and dlls anyway.

Just 2 more hints:
1. In case you want to use scp:
The default configuration for mingw has set type long long and C function strtoll() as not available. With these settings, downloading a file of size >= 2GB=2^31 byte will fail as libssh gets a negative file size!

In the configuration file libssh2_config.h edit section

#ifdef __MINGW32__
#define HAVE_UNISTD_H
#define HAVE_INTTYPES_H
#define HAVE_SYS_TIME_H
#endif

#ifdef __MINGW32__
#define HAVE_UNISTD_H
#define HAVE_INTTYPES_H
#define HAVE_SYS_TIME_H
#endif

To copy to clipboard, switch view to plain text mode 

to become the following:

#ifdef __MINGW32__
#define HAVE_UNISTD_H
#define HAVE_INTTYPES_H
#define HAVE_SYS_TIME_H
/* begin defines for scp added by ars */
#define HAVE_LONGLONG
#define HAVE_STRTOLL
/* end defines for scp added by ars */
#endif

#ifdef __MINGW32__
#define HAVE_UNISTD_H
#define HAVE_INTTYPES_H
#define HAVE_SYS_TIME_H
/* begin defines for scp added by ars */
#define HAVE_LONGLONG
#define HAVE_STRTOLL
/* end defines for scp added by ars */
#endif

To copy to clipboard, switch view to plain text mode 

Rebuild everything after the changes.
Note:
When using function libssh2_scp_recv(), you can pass the address of a

struct stat fileinfo;

struct stat fileinfo;

To copy to clipboard, switch view to plain text mode 

for getting information (file size) of the file to get downloaded. When using fileinfo.st_size, you should cast it to unsigned.

2. If you try to use libssh2 in a 64 bit environment and you want to get information (file size) about files >= 2GB, you should use one of the daily libssh2 snapshots. Release 1.2.7 has a bug that gives false file size (only in 64 bit environment).

[raven-worx]

thank you very much!!! now the makefile produces sucessfully libssh2.dll and libssh2.a files!
I just removed the @ as you suggested... and it worked.

Since i'm compiling in a windows environment there is no rm-command... i replaced the RM-variable in the makefile with the adequate windows-command: RM = del /f
I also made a make-clean everytime before i started another try to compile, so "del /f" is called everytime on the dll, etc. What i want to say is, it tries to delete the file silently with the force-parameter even if the file dont exists.

thanks for all your help!
kind regards

[ars]

The @ at the beginning of commands in the Makefile just prevent the make utility showing the command. It is typically used like

@echo "Output some text"

@echo "Output some text"

To copy to clipboard, switch view to plain text mode 

With @echo, make out is

Output some text

Output some text

To copy to clipboard, switch view to plain text mode 

If the makefile contains

echo "Output some text"

echo "Output some text"

To copy to clipboard, switch view to plain text mode 

make output is

echo "Output some text"
Output some text

echo "Output some text"
Output some text

To copy to clipboard, switch view to plain text mode 

So you can safely add the @ again to the makefile. I suggested removing them for debugging reasons.

Have you tried to use the cygwin shell instead of windows shell? Using cygwin shell and some other cygwin tools (or Msys) simplifies building Unix/Linux sources under Windows. I have cygwin shell installed and could compile openssl-1.0.0c (just did it this afternoon). Before successful compile, I had to run dos2unix on the *.sh files in util. Running

perl Configure mingw
make

perl Configure mingw
make

To copy to clipboard, switch view to plain text mode 

then produced static link libraries. Running

perl Configure mingw shared
make

perl Configure mingw shared
make

To copy to clipboard, switch view to plain text mode 

build the dlls.

[raven-worx]

thanks for all!
One last question: after a first look in the api, i noticed that there are no methods to get a directory listing with SCP in libssh2?!

[ars]

For getting directory listings use libssh2_sftp_opendir() to open a directory. Then call libssh2_sftp_readdir_ex() (or libssh2_sftp_readdir()) until no more data are available. Finally call libssh2_sftp_closedir() to close the directory handle received with libssh2_sftp_opendir().

The readdir() function calls fill a structure providing file information in binary form.
For the parameters to be passed see the api documentation of libssh2. Also look through the examples, e.g. examples/sftpdir.c.

[Suvarna]

Hi,

I have seen this list of API's for libssh2 - http://libssh2.sourceforge.net/doc/

For any download of a file from server ... I didnt find anything like get/download ..

Atleast example is telling that we need to read and write .. is it so ???

[raven-worx]

what about: http://libssh2.sourceforge.net/doc/#libssh2sftpread

i think you'll have to take care to create a file yourself. You can read the file contents into a QByteArray and save it as a file. It's up to your implementation how you're calling the method if it's blocking or not.

regards.