UFT/SIFT: Internet SENDFILE

UFT is Unsolicited File Transfer; SIFT is Sender-Initiated File Transfer.
They're the two sides of Internet SENDFILE. The word "unsolicited" carries the
unfortunate implication that files sent with UFT are unwanted. But this is not the case.
Files are unsolicited only in that the recipient did not initiate the transfer.
The one receiving the file did not "solicit" its delivery directly.

The UFT protocol ...

automatically negotiates optimum data burst size
supports file/metafile pair concept, thus supports extensible file attributes
can be sent via other transports in addition to TCP
can transfer multiple files per session
does not require file size to be known prior to transfer
allows for negociation of proprietary additions
has unique reponse numbers which can be
mapped to unique national language messages

What's the difference between UFT and SIFT?
When UFT operates "on the wire" (that is, over TCP/IP) it is referred to as just UFT.
When the same transaction is carried over mail, it may be called SIFT. Otherwise, they are the same.
In fact, a major goal of implementors is that a transaction be directly translatable between the two modes.

UFT protocol is suitable for transferring a wide range of object types, including electronic mail and print.
Most often, UFT is used to send source code, data files, large files, or any non-correspondence.

Design Objectives

UFT provides at least the same level of file transfer function as is found in
IBM NJE software sucn as RSCS. Additionally, UFT is a "good Internet protocol". A list of most of the objectives of UFT would include:

Plain Text Protocol:
You should be able to 'telnet somehost.somedomain 608' and perform a manual transaction.
This facilitates debugging as well as VHLL implementation. (see below)
Common Translation Formats:
Whatever format a given file is stored in locally, it is "canonicalized" to an easily understood format for transport on the net, and re-converted (if necessary) at the receiving end. In other words, plain text, binary, record-oriented, and any other local formats are all mapped to/from well-known or clearly definable forms.
Transport Options:
A transfer can be carried by a stream (such as TCP, APPC, or even dial-up) or by store-and-forward (such as "mail", "message oriented middleware", or similar methods) and survives conversion into either form, even multiple back and forth conversions. Thus, if a given system administrator doesn't want to run a UFT server, fine; let his users get everything as MIME (by way of mail). This puts the choice into the hands of the local administration. But some of us will want "non-mail" sent on a different channel.
This leads to another point:
Single-Stream:
A UFT transaction is carried out over a single TCP stream.
Using a single stream facilitates UFT being gatewayed or carried over other protocols besides TCP, such as APPC, dial-up, etc.
Extensibility:
New attributes and file types can be added as needed without causing incompatibility with existing clients and servers. Even such essential attributes as the NAME of the file may be omitted when they're not available on one end or the other.
High Level:
Ease of implementation in a VHLL such as Tcl, REXX, VisualBASIC, Perl, etc.
Not only is the protocol plain text, but the files being transferred are represented as file/metafile pairs.

Why not MIME?

MIME effectively enhances e-mail in several ways, particularly in getting non-text through the network. Why not just use MIME to send files?

Files are not Correspondence:
The "mail" infrastructure is tuned for correspondence.
Quite a few sites have trouble with attachments, and especially with large files.
UFT is direct, not queued:
A direct protocol allows for immediate confirmation (or not) that the file arrived.
But again, with wide deployment of UFT/SIFT, the local administration decides which way they prefer things sent.
Attributes:
There's more "meta info" carried in SIFT/UFT than is presently carried in MIME.
The now defunct Content Type: Multipart/Headerset was a good attempt at coping with arbitrary file attribute sets, but it didn't make it in the MIME working group.
Canonicalization:
Finally, UFT/SIFT places stronger emphasis on "canonicalization". A Content-Type (ala MIME) indicates what kind of processing should be applied to the file once it has been received (or even AS it is BEING received), but it doesn't always indicate HOW the file is to be received. Specifically, the Content-Type header line doesn't directly say if an object is plain text, binary, or something else, and these are very different things on different systems. While canonicalization should be associated with each MIME Content Type, it's not always done when the CT is registered.
Worse, some MIME Content Types can vary in how they should be delivered; that is, they're textual one time and binary the next. For example, Application/PGP might need to be sent as plain text, or it might need to be sent as binary, even though it's always processed by the same agent on both ends of the transaction.
Delivery:
Mail gets redirected by aliases, MX records in DNS, and/or forwarding controls on the users account. This is intentional and desireable, but it leaves a niche for SENDFILE to fill. There are times when you might need to send a file to a host on which you do not receive mail.

Make no mistake about it: We're not knocking MIME. We're not knocking the worldwide e-mail infrastructure. We're just saying that sometimes one needs a freight delivery service rather than a letter carrier.

Why not NJE over IP?

UFT is intended to work like IBM's NJE networking, and NJE can be carried over IP.
So why not just use NJE over IP?

It's Proprietary:
It's an IBM protocol that is too mainframe-centric to be used on all platforms. NJE can certainly be, and has been for years, used on UNIX, VMS, and other non-mainframe and non-IBM platforms, but it is never a small proposiion. UFT, however, is simple, open, and more general in nature.
NJE Doubles the Work:
You'd have two networks to maintain, your TCP/IP network and the NJE network (which may or may not be layered on top of the IP network).

Also, NJE is store-and-forward. UFT is direct.

Why not FTP?

Why not just use FTP to send files?
Because any time you use FTP to send files anonymously,
you run into a number of significant administrative "problems".

Is there a writable directory? (/incoming)
Can the intended recipient access the files?
How can the recipient be identified?
What if two files are written with the same name?

Please understand that UFT/SIFT isn't expected to replace MIME, FTP, or NJE.
It is just hoped that it will augment the standard TCP/IP suite of tools and protocols. As the variety of protocols used on the web allow for achievable "anonymous get", SIFT/UFT provides "anonymous put".

Implementations

UFT has been implemented in C for UNIX (POSIX) and in REXX with CMS Pipelines for VM.
UFT has also been implemented in Tcl 7.5 as proof of concept. (The Tcl version doesn't handle binary files without help from the C version, but this is Tcl's fault, not UFT's.)

To illustrate a simple one-shot file transfer: a UFT client connects to a UFT server, sends a series of commands waiting for acknowledgements, and closes the connection. UFT transactions are carried out over TCP with the server listening on port 608. Most of the commands costitute a "meta file" of attributes of the real file. The real file, referred to as the "body" or "data file", is sent as a stream of octets. The octets are canonicalized from local convention on the sending side to an agreed format for transport and then restored to local convention on the receiving side (where representation may be different than for the sender).

Related Information

Copyright and Licensing

Author

Rick Troth <troth@casita.houston.tx.us>

Inspired by BITNET, which carried non-mail distinctly from mail, and by FRED, which was never developed though it was well conceived, and by LPR/LPD, FTP, and SMTP.