LibNcFTP: FTP library for UNIX

  • Installing the library
  • Compiling with the library
  • Tutorial
  • Library Data Structures
  • Function reference
  • Questions & Answers
    •

    Installing the library

    If you have the binary distribution, the package will include some library files (libncftp.a and libStrn.a) and some header files (ncftp.h, ncftp_errno.h, and Strn.h).

    Copy the library files to your system's library directory, most likely /usr/lib, and the header files to the header directory, most likely /usr/include. You may of course install those in other directories as long as you know how to tell your compiler how to find them.

    If you have the source distribution, you will need to compile the libraries before you can install them.

    First, unpack the archive. If the package arrived as a ".tar.gz" or ".tgz" file, you need to run gunzip, and then tar to extract the source files. You can do that in one shot by doing "gunzip -c libncftp.tgz | tar xvf -". If the package you have is a ".tar" file you can use "tar xvf libncftp.tar". If the package you have is a ".zip" file you can use "unzip libncftp.zip" or "pkunzip libncftp.zip".

    Now go to the "libncftp" directory you just made. There is a script you must run which will checks your system for certain features, so that the library can be compiled on a variety of UNIX systems. Run this script by typing "sh ./configure" in that directory. After that, you can look at the Makefile it made if you like, and then you run "make" to create the libncftp.a and libStrn.a library files.

    Finally, install the libraries and headers. You can manually copy the files as stated above for the binary distribution, or you can run "make install" to copy the files for you. If you choose to "make install" you may want to edit the Makefile if you do not want to install to the /usr/local tree.

    Compiling with the library

    After compiling both the LibNcFTP library and its support library, LibStrn, you are ready to build your applications.

    Add "-lncftp -lStrn" to your list of libraries in your Makefile. If you did not install the libraries in the same place as the system libraries (i.e. /usr/lib), you will need to use -L with your C compiler to tell it which directory you put them in.

    Similarly, if you did not install the library headers in /usr/include, you will need to use -I, like -I/usr/local/include. Here is a sample command line compile to illustrate: 

            cc -I/usr/local/include minincftp.c -o minincftp -L/usr/local/lib -lncftp -lStrn
    For your applications, you will need to add #include <ncftp.h> to every relevant source file that references a LibNcFTP function.

    Tutorial

    The library avoids using global variables. However you give each library routine a pointer to a structure with the library's state data. This state data must be maintained between calls, and you must pass a pointer to the same structure for each library function. For simplicity, this example puts the two library structures you pass around in two global variables: 
        FTPLibraryInfo li;
    FTPConnectionInfo ci;
    Somewhere early in your program, before you try any FTP, you must initialize the library. Do that like this: 
        FTPInitLibrary(&li);
    Then, when you want to open a host, you initialize a session structure. Do that by calling FTPInitConnectionInfo() to initialize to the default values, and then you set the fields that define the session: 
        FTPInitConnectionInfo(&li, &ci, kDefaultFTPBufSize);
    For this example, we will try a non-anonymous login to a server named ftp.cs.unl.edu with a username of mgleason and a password of au29X-b7. 
        ci.debugLog = myDebugFile;
    strcpy(ci.user, "mgleason");
    strcpy(ci.pass, "au29X-b7");
    strcpy(ci.host, "ftp.cs.unl.edu");
    I recommend that you use the optional debugLog parameter while you are testing. That is a FILE * pointer, and you are responsible for opening and closing it. You may want to use just stdout.

    Then open the host: 

        if ((result = FTPOpenHost(&ci)) < 0) {
        fprintf(stderr, "Cannot open host: %s.\n", FTPStrError(result));
        exit(1);
    }
    Now try some downloads: 
        FTPChdir(&ci, "/pub/foobar");
    globPattern = "log.????96";
    dstDir = "/usr/log/junk";               /* local directory */
    result = FTPGetFiles(&ci, globPattern, dstDir, kRecursiveNo, kGlobYes);
    Make a few directories, setting the recursive parameter to kRecursiveYes to simulate "mkdir -p": 
        newDir = "/pub/foobar/uploads/May96";
    FTPMkdir(&ci, newDir, kRecursiveYes);
    Upload some files: 
        result = FTPPutFiles3(&ci, "/usr/logs/*.05??96", newDir,
                    kRecursiveNo, kGlobYes, kTypeBinary, kAppendNo,
                    NULL, NULL, kResumeNo, kDeleteNo,
                    NoConfirmResumeUploadProc, 0
                    );
    Finally, close the connection. If your FTPOpenHost succeeded, you should make a good effort to make sure you close it: 
        FTPCloseHost(&ci);
    To see a simple example in entirety, see the simpleget.c source file included with the sample code. You should be able to compile that program and run it, to see how things work.

    Library Data Structures

    FTPLibraryInfo

    Each program that uses the library should have one of these structures. This needs to be referenced by all parts of your code that call a FTP library routine, so declare this a global variable or local variable in scope of all subroutines that use FTP. 
    typedef struct FTPLibraryInfo {
        char magic[16];
        int init;
        int socksInit;
        unsigned int defaultPort;
        char ourHostName[64];
        int hresult;
        int htried;
        char defaultAnonPassword[80];
} FTPLibraryInfo, *FTPLIPtr;
    The magic field is used internally by the library to make sure the programmer (that would be you :-) isn't using an invalid structure with the library. The library routines check this field against a well-known value to ensure validity.

    The defaultAnonPassword field can be changed after the library has been initialized. You may need to set this manually if the library does not calculate a valid password for use with anonymous logins.

    The other fields are used internally, but may be useful information to you while debugging (such as ourHostName).

    FTPConnectionInfo

    Each program that uses the library may have one or more of these structures in use at a time. Like the FTPLibraryInfo structure, you should declare these as global variables or local variables that maintain scope throughout use of your FTP operations, because you need to pass a pointer to this structure to each library function.

    This structure is a mixture of mostly internal-use variables and configurable options. 

    typedef struct FTPConnectionInfo {
        char magic[16];
        char host[64];
        char user[64];
        char pass[64];
        char acct[64];
        unsigned int port;
        unsigned int xferTimeout;
        unsigned int connTimeout;
        unsigned int ctrlTimeout;
        unsigned int abortTimeout;
        FILE *debugLog;
        FILE *errLog;
        FTPLogProc debugLogProc;
        FTPLogProc errLogProc;
        FTPLIPtr lip;
        int maxDials;
        int redialDelay;
        int dataPortMode;
        char actualHost[64];
        char ip[32];
        int connected;
        int loggedIn;
        int curTransferType;
        char *startingWorkingDirectory;
        longest_int startPoint;
        int hasPASV;
        int hasSIZE;
        int hasMDTM;
        int hasREST;
        int hasNLST_d;
        int hasUTIME;
        int hasFEAT;
        int hasMLSD;
        int hasMLST;
        int usedMLS;
        int hasCLNT;
        int mlsFeatures;
        int STATfileParamWorks;
        int NLSTfileParamWorks;
        struct sockaddr_in servCtlAddr;
        struct sockaddr_in servDataAddr;
        struct sockaddr_in ourCtlAddr;
        struct sockaddr_in ourDataAddr;
        int netMode;
        char *buf;
        size_t bufSize;
        FILE *cin;
        FILE *cout;
        int ctrlSocketR;
        int ctrlSocketW;
        int dataSocket;
        int errNo;
        unsigned short ephemLo;
        unsigned short ephemHi;
        int cancelXfer;
        longest_int bytesTransferred;
        FTPProgressMeterProc progress;
        int useProgressMeter;
        int leavePass;
        double sec;
        double secLeft;
        double kBytesPerSec;
        double percentCompleted;
        longest_int expectedSize;
        time_t mdtm;
        time_t nextProgressUpdate;
        const char *rname;
        const char *lname;
        struct timeval t0;
        int stalled;
        int eofOkay;
        char lastFTPCmdResultStr[128];
        LineList lastFTPCmdResultLL;
        int lastFTPCmdResultNum;
        char firewallHost[64];
        char firewallUser[64];
        char firewallPass[64];
        unsigned int firewallPort;
        int firewallType;
        int require20;
        int usingTAR;
        FTPConnectMessageProc onConnectMsgProc;
        FTPRedialStatusProc redialStatusProc;
        FTPPrintResponseProc printResponseProc;
        FTPLoginMessageProc onLoginMsgProc;
        size_t ctrlSocketRBufSize;
        size_t ctrlSocketSBufSize;
        size_t dataSocketRBufSize;
        size_t dataSocketSBufSize;
        int serverType;
        int ietfCompatLevel;
        int reserved[32];
} FTPConnectionInfo;
    Before connecting, you will always set the host field, which is the hostname or IP address of the remote FTP server to connect to. If you are logging in non-anonymously, you will also need to set the user and pass fields (and very rarely, the acct field). If the server is running on a non-standard port number (not 21), you may set the port field.

    The library can "redial" automatically, so if the connection or login attempt failed, it can retry. To do this, you set the maxDials field to 2 or more. The related field redialDelay can be set to the number of seconds to wait between each attempt.

    The most interesting field is the errNo field. If you an error occurs within a library function, a negative result code is returned and the errNo field is set to the error number, which you can find listed in <ncftp_errno.h>.

    If you get an error, you may also want to inspect the lastFTPCmdResultStr field. This will contain the first line of the most recent reply from the FTP server. Similarly, the lastFTPCmdResultNum contains the numeric code that came along with it, and the lastFTPCmdResultLL is the LineList containing the complete reply.

    For debugging purposes you may want to use the debugLog and errLog fields, which you can set to stdio FILE * pointers. The debugLog writes the whole conversation that took place on the control connection, and would be what you would get had you done an FTP manually. The errLog file gets written to whenever an error occurs, so this should be a lot smaller than the debugLog.

    Instead of having the library write to files directly, you may wish to use your own logging function. You can use the errLogProc and debugLogProc fields to point to your custom logging functions. The function should be of this type: 

        typedef void (*FTPLogProc)(const FTPCIPtr, char *);
    The first argument is a pointer to the FTPConnectionInfo structure, and the second is the string produced for logging.

    There are several timeout fields you can set if you want certain FTP operations to abort after a period of time. These fields are xferTimeout, connTimeout, ctrlTimeout, and abortTimeout, and each value is in seconds. If you set a timeout field, you also need to have a signal handler for SIGALRM, because alarm is used for this, so when a timeout expires, a SIGALRM signal is generated.

    The xferTimeout refers to the amount of time to wait on an data transfer read or write; The connTimeout refers to the amount of time to wait before establishing a successful control (FTP conversation) connection; The ctrlTimeout refers to the amount of time to wait for a response from the server on an established control connection; And the abortTimeout refers to a special case of the above when an abort transfer has been requested. (Usually that is used when the user hits ^C and is intending to quit the program as soon as possible, so the time is much less than a regular control connection response.)

    The dataPortMode may be set to one of kSendPortMode, kPassiveMode, or kFallBackToSendPortMode. If you want to try passive FTP, you can use kPassiveMode. You can also use kFallBackToSendPortMode which means the library will try passive FTP first, and if the server does not support it, it will then try regular port FTP.

    One of the first things the library does after successfully logging in to a remote server is determine the current working directory. The startingWorkingDirectory is set as a result. It may be NULL if the login failed.

    The library computes statistics for each data transfer. If you are interested in those, you can inspect the bytesTransferred, sec, and kBytesPerSec fields for the results. Those correspond to the size of the file, how long it took in seconds to transfer, and how fast it was in kilobytes per second.

    You may also want to implement a progress meter, which updates while the transfer is in progress. To do that you can pass a pointer to a function of type: 

        typedef void (*FTPProgressMeterProc)(const FTPCIPtr, int);
    The second argument tells you which state the transfer is in. You will get a kPrInitMsg message before the first block of data transferred, and an kPrEndMsg at the end. In addition, you get a kPrUpdateMsgfor each block transferred. The sample source code included with the package shows how to use progress meters.

    Working with LineLists

    Some of the library routines work with a LineList structure. This is simply a linked-list of dynamically allocated C-strings. 
    typedef struct Line *LinePtr;
typedef struct Line {
        LinePtr prev, next;
        char *line;
} Line;

typedef struct LineList {
        LinePtr first, last;
        int nLines;
} LineList, *LineListPtr;
    Here is an example use that shows how to print the contents of a remote wildcard match: 
    LineList fileList;
LinePtr lp;
int i, err;

err = FTPRemoteGlob(cip, &fileList, "*.c", &fileList, kGlobYes);
if (err == kNoErr) {
    for (lp = fileList.first, i=0; lp != NULL; lp = lp->next) {
        ++i;
        printf("item #%d: %s\n", i, lp->line);
    }
}

DisposeLineListContents(&list);

    Function reference

    FTP functions

    FTPAbortDataTransfer

    FTPChdir

    FTPChdirAndGetCWD

    FTPChmod

    FTPCloseHost

    FTPCmd

    FTPDecodeURL

    FTPDelete

    FTPFileExists

    FTPFileModificationTime

    FTPFileSize

    FTPFileSizeAndModificationTime

    FTPFileType

    int FTPFileType(const FTPCIPtr cip, const char *const file, int *const ftype);

    This function can be used to determine if a particular pathname is a directory or regular file.  It returns 'd' in the ftype parameter if the item was a directory, or '-' if it was a regular file.  The ftype parameter return result should only be used if the function returns 0 (kNoErr).  If the item exists but the type could not be determined, the error code kErrFileExistsButCannotDetermineType is returned.  Other types of errors are returned as negative error codes.

    This function calls FTPFileExists, which may be an expensive call.

    Example: 

        FTPConnectionInfo ci;
    int ftype, result;

    if ((result = FTPFileType(&ci, "/pub/linux", &ftype)) == kNoErr) {
        if (ftype == 'd')
            printf("It was a directory.\n");
        else
            printf("It was a file.\n");
    } else {
            printf("An error occurred (%d).\n", result);
    }

    FTPGetCWD

    FTPGetFiles, FTPGetFiles2

    FTPGetFiles3

    FTPGetOneFile, FTPGetOneFile2

    FTPGetOneFile3

    FTPInitConnectionInfo

    FTPInitLibrary

    FTPIsDir

    FTPIsRegularFile

    FTPList

    FTPListToMemory

    FTPLocalGlob

    FTPLoginHost

    FTPMkdir

    FTPOpenHost

    FTPOpenHostNoLogin

    FTPPerror

    FTPPutFiles, FTPPutFiles2

    FTPPutFiles3

    FTPPutOneFile, FTPPutOneFile2

    FTPPutOneFile3

    FTPRemoteGlob

    FTPRename

    FTPRmdir

    FTPShutdownHost

    FTPStrError

    FTPSymlink

    FTPUmask

    FTPUtime

    LineList functions

    CopyLineList

    DisposeLineListContents

    InitLineList

    RemoveLine

    AddLine

    Questions & Answers

    1. How do I get the library to use passive FTP?
     
    The dataPortMode field of your FTPConnectionInfo structure may be set to one of kSendPortMode, kPassiveMode, or kFallBackToSendPortMode. If you want to try passive FTP, you can use kPassiveMode. You can also use kFallBackToSendPortMode which means the library will try passive FTP first, and if the server does not support it, it will then try regular port FTP.
     
    2. How can I do nonblocking FTP library calls?

     
    Technically, you can't. The library isn't coded for nonblocking I/O and it would be ugly it was. But you can have an operation timeout after a number of seconds so you don't hang forever on a slow or nonresponsive server.

    To do that, the xferTimeout field of your FTPConnectionInfo structure can be set to a positive integer. You will also need to have a signal handler for SIGALRM, because alarm is used for this, and when the timer expires you should jump to your signal handler instead of exiting your program.

    If you jump out of a library function, the FTP connection is in an undefined state and cannot be used any further. You should then call FTPShutdownHost to free up system resources before continuing.
     

    3. How do I debug an application using the library?

     
    The easiest is to take advantage of the debugLog field in your FTPConnectionInfo structure, then look at the log. You should be able to see the whole FTP conversation, and what errors the remote server reported, if any. You can then repeat that same conversation using an FTP client to investigate possible problems with the server you are communicating with.
     
    4. How can I contact the library's author?
     
    Send e-mail to Mike Gleason at mgleason@ncftp.com
     
    5. Is SOCKS supported?
     
    Possibly. I built the library from my NcFTP code, which supposedly supports the SOCKS library. I am not sure whether it is implemented correctly because I don't have a SOCKS server to test with.

    When the library is built, the configure script must be told to look for SOCKS, as in ./configure --enable-socks5.