FTP File Compression

Own a personal computer for any amount of time, and you will quickly realize that the amount of storage space on your computer is limited. One way to deal with this problem is to use a compression software package that "squishes" unused programs into small "boxes", thus freeing up a little more of your disk space for other programs.

It turns out that storage space problems are not limited solely to personal computers. As the numbers of files that are available through ftp increases daily, ftp sites are actively looking for ways to squeeze more files into a limited amount of space.

The ftp sites accomplish this by using file compression.

The good news is that a compressed file takes up a lot less space on the ftp site's computer. The bad news is that a compressed file is absolutely useless until you uncompress it.

Wait ... it gets worse. Before you can uncompress a file, you have to know what compression method was used to compress the file in the first place. Unfortunately, there is no one standard ftp file compression method -- there are HUNDREDS of different file compression methods in use today :(

If you have to know what compression method was used before you can uncompress a file, how are you ever going to figure out which method was used? Well, it is actually pretty easy:

  1. Most ftp directories have a READ.ME file that shows an index of all the files that are in that directory. Some really nice ftp sites have expanded READ.ME files that include a mention about what compression method was used and where you can get a free copy of the software needed to uncompress the files.

  2. Look at the files' extensions. By looking at the extensions and comparing them to the chart below, you will be able to determine what compression method was used and what particular software is needed to uncompress the file.
Fortunately, most uncompression software is either public domain (meaning that it is completely free) or shareware (meaning that you can get a copy of it for free, but the author expects you to send him some money for the program if you decide to keep it and use it). Best of all, most uncompression software is available through ftp! :)

The list below shows some of the most popular extensions that you are bound to encounter during your visits to ftp sites around the world. It also shows transfer modes needed to retrieve files with these extensions, what uncompress software package you need to to uncompress the files after you retrieve them, and it even gives some additional comments about each of the extensions.

Paraphrasing something I said in MAP01: Welcome, I want you to be aware that the one compression method that isn't listed below is going to be the one compression package that you ADORE. Please do not take this personally. There are literally HUNDREDS of compression methods in use today, and there is no way that I can list all of them.

Suggestion: Save the following list, and use it as a reference tool for when you encounter an extension that you have never seen before :) Also, please notice that the following list talks about "archie". Archie is an FTP search tool that we will discuss in the next lesson.

(the following list was adapted from "The EFF's Guide to the Internet" by Patrick Crispen)
------------ ------ ---------- -----------------------------------

.txt or .TXT ASCII By itself, this means the file is
a document rather than a program,
and does not need to be uncompressed

.ps or .PS ASCII A PostScript document (in Adobe's
page description language). You can
print this file on any PostScript
capable printer or use a previewer,
like GNU project's GhostScript.

.doc or .DOC ASCII Another common extension for text
documents. (Be careful, though: .doc
and .DOC extensions are also used for
Microsoft Word documents (which are
Binary files). The duck theory will
help you determine the difference)
No decompression is needed, unless it
is followed by:

.Z Binary uncompress This indicates a Unix compression
method. To uncompress type

uncompress filename.Z

and hit enter at your host system's
command line.

u16.zip is an MS-DOS program that
will let you download .Z files and
uncompress them on your own computer.
The Macintosh equivalent program is
called MacCompress (use archie to
find these).

.zip or .ZIP Binary PKZip or This indicates the file has been
Zip/Unzip compressed with a common MS-DOS
compression program, known as PKZIP
(use archie to find PKZIP204.EXE).
Many Unix systems will let you un-ZIP
a file with a program called unzip.

.gz Binary gunzip A Unix version of ZIP. To uncompress,

gunzip filename.gz

at your host system's command line.

.zoo or .ZOO Binary zoo A Unix and MS-DOS compression
format. Use a program called zoo
to uncompress.

.shar or .Shar Binary unshar Another Unix format. Use unshar
to uncompress.

.tar Binary tar Another Unix format, often used
to compress several related files
into one large file. All Unix
systems will have a program called
tar for "un-tarring" such files.
Often, a "tarred" file will also be
be compressed with the gz method,
so you first have to use uncompress
and then tar.

.sit or .Sit Binary StuffIt A Macintosh format that requires
the StuffIt program.

.ARC Binary ARC or Another MS-DOS format, which
ARCE requires the use of the ARC
or ARCE programs.

.LHZ Binary LHARC Another MS-DOS format; requires
the use of LHARC.
"A few last words of caution: check the size of a file before you get it. The Net moves data at phenomenal rates of speed. But that 500,000-byte file that gets transferred to your host system in a few seconds could take more than an hour or two to download to your computer if you're using a 2400-baud modem. Your host system may also have limits on the amount of bytes you can store online at any one time. Also, although it is really extremely unlikely you will ever get a file infected with a virus, if you plan to do much downloading over the Net, you'd be wise to invest in a good anti-viral program, just in case." (1)


Yesterday, I showed you that it is possible to get ftp files using e-mail by sending an e-mail letter to one of the following addresses
          Australia      ftpmail@cs.uow.edu.au
France ftpmail@grasp.insa-lyon.fr
Germany ftpmail@ftp.uni-stuttgart.de
Great Britain ftpmail@doc.ic.ac.uk
Ireland ftpmail@ieunet.ie
Sweden ftpmail@lth.se
USA ftpmail@sunsite.unc.edu
USA ftpmail@ftp.uu.net
USA ftpmail@decwrl.dec.com
with the following commands in the body of your e-mail letter

Before I introduce you to the new stuff, there are a couple of things that I want to review with you.

First, the

command tells the FTPmail address where you want the file sent. If you use the example that I gave you yesterday
          reply pcrispe1@ua1vm.ua.edu
connect ftp.sura.net
chdir /pub/articles
get fall91.issue
without changing the reply address, FTPmail is going to send the file to me, not to you :)

Please remember to change the reply line to include your Internet e-mail address.

Also, I did not mention this in the last lesson, but FTPmail limits you to only one CHDIR command per letter.

Finally, in the last lesson I asked you to contact you local Internet service provider to see if they placed any size limits on file transfers. If they do, there is an additional command that you need to add to your list of commands

This command will break the files into chunks that your system can handle. If your system has a 50,000 character limit on messages from the Internet, your chunksize command should be
    chunksize 49000
(you want to make sure that you set your chunksize below what your system's limits are). This command will break your file into 49,000 character chunks, and will then send the chunks to you :)

You already now how to retrieve ASCII text files using FTPmail. In this lesson I am going to show you how to retrieve Binary files using FTPmail.

Binary file transfers using FTPmail aren't difficult ... they just require a few additional steps. Because all e-mail has to be in ASCII form, FTPmail has to encode your Binary file in ASCII before it can e-mail the file to you. Once you get the file, you can then decode the file back into Binary :)

Fortunately, there are two ways that FTPmail can encode Binary files into ASCII. The first way it can do this is through something called "uuencode." As long as you have a uudecode program -- and uudecode programs are all over the place (chances are your site has uudecode stored on its system) -- the whole process is simple.

The second encoding type that you can use is called "btoa" (binary to ascii). Your local Internet service provider will be able to tell you a little more about btoa.

So, to get ASCII files using FTPmail, you would use the following commands in the body of your letter to the FTPmail address:

and to get Binary files using FTPmail, you would use the following commands in the body of your letter to the FTPmail address: