Carl Malamud: Internet Talk Radio, flame of the Internet.
Malamud: This is Geek of the Week and we’re talking to Ned Freed, who is a cofounder and Chief Development Officer with Innosoft International. Ned, I have two questions to start off with, what’s a Chief Development Officer, and what’s Innosoft?
Freed: Innosoft is a small software development company that was started by three students who’d just graduated from Harvey Mudd College, myself, and Kevin Carosso, and Dan Newman were the principal founders. And we initially started out by selling of all things mathematical modeling software. Which turned out to be fun but not particularly…profitable, shall we say. And we as sort of a sideline we had developed some electronic messaging facilities for VMS systems. And the development of that turned out to be a much more reasonable business proposition and so we started concentrating on that. And Innosoft currently… I believe we have fourteen employees at the present time? And I am the nominal Chief of Development, but my strange title comes from the fact that I’m also an officer of the company.
Malamud: Ah, so you’re not in human resources or anything like that.
Freed: No, afraid not.
Malamud: You make basically VMS-specific message handling software?
Freed: Correct. PMDF is our primary product. It is actually sort of a generic messaging switch that was developed originally on VMS but it has been ported to other platforms. However, our interest thus far has been solely in operating it on VMS systems, where the relative lack of Internet messaging technology makes it you know, a good fit, a particularly good fit. And—
Malamud: It’s a good fit in terms of a market niche, but do you find that the Internet standards are a good fit for the VMS operating system, or are they Unix-specific?
Freed: There’s more Unix-specificity in the standards that I would like. And there’s far more Unix-specificity in certain implementation aspects of various Internet things. Messaging happens very fortunately one of things is relatively immune to that. But there are other things that you run into from time to time. For instance things that go around looking at anonymous FTP servers, as an example, and don’t understand the VMS path structure, and you know, that kind of thing presents a problem. Those crop up relatively regularly and present difficulties to us.
But since we write all of our own code in our particular environment, we don’t really see a significant problem with the incompatibility of the standards versus the environment that we’re in. You know, it helps that the VAX is—and the Alpha—are you know, fairly natural machine structures. I mean just because we’re on a different operating system doesn’t mean that the underlying hardware has any fundamentally incompatible notions. I mean, we have roughly the same byte sizes and things like that, so we don’t run into significant problems in that area. Which you do run into on other weirder kinds of hardware and software.
Malamud: You’ve been heavily involved in the effort to develop MIME, the multimedia messaging in the Internet standards. Tell us a little bit about MIME and what it does.
Freed: Okay. MIME, which stands for Multipurpose Internet Messaging Extensions—some people say multimedia, some people say multipart. It doesn’t really matter, it is what it is. The MIME work started out oddly enough as an effort to develop a mechanism to handle international character sets in the Internet messaging environment. That was the original charter of the working group that was originally formed to deal with these issues.
However, messaging in the Internet, despite being probably the largest single application that’s used on the Internet today, had received relatively little attention in terms of the standards process subsequent to the development of RFC 822, which is a 10 year-old document. Actually it’s older than that now. But it was just reproaching its tenth birthday as MIME came to fruition. The development of MIME started as an effort to deal with international character sets, but because of the lack of attention it grew into much more, and we wanted to deal with a lot of the needs that had not been addressed in the messaging world in a clean and unified kind of way instead of trying to solve each of the individual problems piecemeal.
The other very important thing about MIME that—it never hurts to emphasize this and you really can’t emphasize it enough, is that MIME is a culmination of work based on three previous attempts to do various types of multimedia and multipart Internet messaging. Various other people had tried all sorts of other schemes in the past.
Malamud: What are some of those schemes?
Freed: Some of those schemes are the RFC 934 defines a message encapsulation format, a way of encapsulating one message inside of another and having multiple parks in a message, with boundary markers and things. RFC 1049 defines a content type or a labeling scheme so that you can tell what is in a given message. The Privacy-Enhanced Mail work had defined an encoding format so that binary material could be sent through existing Internet messaging. RFC 1154 had defined a labeling and boundary scheme for putting multiple things into a single message. And there were various vendor-proprietary schemes. NeXT has a scheme that’s popular on that platform. And there are several others from other different vendors. The NeXT one is probably the best-known of those.
Another one that’s actually important because of the influence it had on one of the coauthors of MIME, Nathaniel Borenstein, is the Andrew messaging system which had done multipart mail in its own peculiar way for many years. And Nathaniel was involved in the development of that.
All of these schemes worked to a certain extent. They had various problems, of course. Everything has problems. But none of them had achieved wide acceptance and wide usage across the Internet. And part of the initial MIME effort focused on trying to identify why these efforts had not reached fruition and wide deployment. In some cases it was simply perhaps more of a factor of these had not really been standards-blessed efforts, and hence were just experimental things that people had put together at one point or another. A lot of the vendor-proprietary protocols used formats which were not backwards-compatible in any real way with existing applications. And since we felt that that was one of the real hindrances to acceptance we were very careful to stay away from using formats that had significant impact in terms of backwards compatibility with existing applications. And that philosophy, that one single point, you can see influences of that throughout the development of MIME.
And so what we eventually… You take all these influences and what we decided to do…and all this stuff, and you wrap it all up and basically what we came up with was a standard that basically really does three things.
First of all is it establishes a labeling scheme, so that you can label material for what it actually is rather than relying on some kind of a embedding in text and people being able to say, “Gee, this is a sound file. Why don’t you extract it and run it through this and that and the other and then play it.” We wanted the labeling to be natural and machine-oriented yet readable to human beings, simultaneously.
Second we wanted an encoding format so that existing Internet infrastructure could be used to send multimedia mail. Multimedia tends to be binary in nature. The existing Internet infrastructure 7‑bit ASCII, so encoding is necessary in order to use that. We felt with an installed base of…what is it, a million-three systems currently, that anyone who didn’t try to capitalize on that existing infrastructure was basically going to come up with something that never would play.
Malamud: Well but as I understand it a lot of the existing systems already handle binary data within their messaging. Why didn’t you just accept that fact and go with it? I understand that the standards talk about 7‑bit ASCII, but many people have kind of said well, that’s nice but you know, if you’re within this environment you can handle binary data.
Freed: Well, first of all, within this env— The key phrase there is “within this environment.” We actually use the phrase “enclave.” Within certain enclaves binary data can be handled, and actually it’s relatively rare to find places that can really handle binary data. It’s much more common to find people that can handle 8‑bit text. But the problem is you have no idea when you’re going to run into someone that does it wrong. And environment can change from day to day. And in the absence of any standards that clearly say “Thou shalt do this and support it,” a vendor is perfectly justified in saying, “This is not my problem. You’re sending me something that I was never supposed to get.” And then what do you do? There’s no— You know, you have a finger-pointing exercise with no one really being in the wrong, and therefore there’s no clear justification for any one person or vendor to clean up the problem.
Malamud: But what about making all the vendors upgrade their software and let’s say you know, upgrade SMTP to specify binary data is one of the ways of transferring things.
Freed: Well that’s sort of… That of course is in fact what has happened is an extended SMTP has come out of this work in addition to the MIME standard. However events proved us correct in saying that that effort would take a long time to come to fruition. And it is still unclear as to how widespread adoption of the SMTP extensions were.
But the fundamental problem here is that we’re flying in the face of Internet philosophy. I mean, this is more or less a philosophical issue which basically says…you know, what we like to say is that the standards are the standards, and you’re supposed to follow them. And if you don’t, well…you know, that’s fine as long as—and what you can get away with is what you can get away with. But don’t expect other people to conform to that. I mean, it’s a much looser model than say, strict OSI compliance or strict ANSI compliance or something. But it is nevertheless a model, and it is real, and it is there, and we would like to stick with the philosophy. And the philosophical point of declaring some existing infrastructure as just being outmoded and illegal…you can’t just summarily do that. You have to have a transition period. And how are you going to enter in on a transition period where you’re not going to be able to use any of the transitional stuff without you know— You’re asking people to upgrade to something they cannot use. And they’re gonna sit there with something they cannot use, that they presumably have paid money for, until everyone in the entire network has upgraded equivalently. And then you have a flag day where you can switch over to the new thing. That may work for Switzerland when they decide what side of the road to change driving on. Apparently they changed over tonight. But the Internet is a little bit of a looser organization and it just doesn’t fly, I’m afraid.
Malamud: You mentioned labeling as one of the three functions of MIME. What’re the other two?
Freed: The other two are the encoding facilities, and the the third one is structuring, the ability to send more than one object inside of a single message. And this is important for applications where you want to for example send an annotation to an existing document, or you want to send some audio and some video at the same time, and you don’t necessarily want to use an integrated multimedia format for that purpose.
In no way does MIME’s structuring attempt to usurp the territory and the prerogatives of very sophisticated and powerful multimedia formats that allow precise control over placement. That is both temporal placement as to what you play when and what you show when, as well as you know, placement on the screen and everything like that. But there’s a lot that can be done with imprecise control of placement. And by using the tools in MIME you can combine relatively simple formats and get very powerful results. And the bottom line here is that not everybody can be expected to upgrade to the same you know—to some very sophisticated and powerful multimedia tool simultaneously.
In fact it’s not even clear that we want to be in the business of specifying what multimedia tools people should in fact use. There’s a lot of vendors out there with a lot of competing products that implement a lot of actual competing standards for multimedia. And we wanted to avoid dealing with that. We wanted to simply have a labeling scheme, and a structuring scheme, and an encoding scheme, and couple the three, people can label it as what it is, they can structure however they want if it needs to be. And they can encode it so it can be transmitted over the existing network. And then it gets to wherever it’s going, and people can undo all of that because it’s actually fairly simple and obvious how it’s all pieced together. And they can deal with the results.
Malamud: One of the clever functions in MIME is the pointer to external information. I can send you a mail message and that mail message says “There happens to be an archive out there and if you FTP this file we’ll go get it for you.” Could you explain that function and how it works?
Freed: Basically the idea is— This was put in primarily as a means of saving network bandwidth for mailing lists that have a large number of recipients mailing out large documents to all the recipients when it’s probably true that a significant fraction of the recipients don’t really care about the contents of all the documents. But that can be a very time- and space-consuming operation to try to do that.
So what we attempted to do, in recognition of the fact that any use of multimedia mail on the Internet was going to increase bandwidth utilization, we attempted to try to solve that problem by letting people instead of sending the entire object, send a pointer to the object. And the way that works is— We identified several pointer mechanisms that could be used. One of them is a mail server where you read the message and basically the reading agent will compose a request to the mail server to get the actual document back. That’s a relatively simple scheme. Another scheme is to use FTP to FTP the document over—that has turned out to be the most popular scheme, I think. The third is to use something like NFS, or AFS, or some equivalent file-level access protocol to do it. And owing to the inherent security problems associated with that I don’t believe it has proven to be very popular just yet. That may change in the future.
It is actually rather surprising to me that this idea, which was sort of a last-minute kind of “gee, it would be nice to do that” sort of thing has proved to be rather incredibly popular, and one of the more widely-used pieces of MIME functionality. I suppose in hindsight it was obvious that it would be that way. Because no matter how simple a multimedia format is, be it a GIF picture or an audio/basic you know…8‑bit mu-law sound type or something like that, there’s still a significant fraction of people who can’t deal with that kind of data on the network. And so you basically have to have private arrangements really to know to exchange that kind of mail in any real way. Whereas the external reference type is something that basically everybody can use. Because what you’re sending a reference to isn’t necessarily a multimedia document. It can be a large, or even a small, plain text document. That’s perfectly acceptable.
Malamud: One of the buzzwords in the 90s appears to be mail-enabled applications. And it sounds like the external reference function in MIME is one of the tools that a mail-enabled application might want to use. Are there other tools that MIME or underlying transport mechanisms like SMTP oughta have?
Freed: Well, the thing that I think is the real stumbling block to mail-enabled applications right now is the fact that we are only just now beginning to put a security infrastructure in place for electronic mail. It is fine to talk about mail-enabled applications at a very simple level where rather than leaving the spreadsheet you send a mail message right then and there and clip out a piece of it. That’s a mail-enabled application.
On the other hand that’s not dramatically different than hopping to another window on your workstation and doing the operation there. So, I don’t regard that as innovative new use of mail-enabling of anything because cut and paste across windows on workstations is an equivalent functionality there. Mail-enabled applications will come into their own only when applications are exchanging and interpreting data on their own, using mail as a transport infrastructure. And unfortunately most of the applications where it’s really interesting to do that require some degree of confidence in the authenticity of the message. An obvious example is you do what’s called electronic routing and authorization, where you take a document—let’s say it’s a travel voucher—and you send it to a mailing list, and the mailing list is accommodated by a server which routes the travel voucher to all the people who have to approve that voucher in the company. And they sign off on it in an authenticated way.
But we’re talking about handling real dollars here, in a real situation. And you absolutely have to have some degree of confidence in the integrity of that kind of a set up in order to really deploy it widely and make effective use of it. Especially then if you’re going across the Internet, where as everyone mostly knows, mail is not terribly secure.
Malamud: Okay, well let’s turn our attention to security, then. I’ve noticed that there are actually two electronic mail efforts in the Internet. There’s Privacy-Enhanced Mail, and there’s MIME, and they appear to have been parallel, separate activities. Why are there two separate standards looking at mail, and do they interoperate?
Freed: This is largely a result of timeframes and the way that the process put things together, and it has led to a rather unfortunate state of affairs which we are attempting to rectify. I personally believe that both MIME and PEM will come into their own only when they can avail themselves of their counterparts’ services. MIME, which has no integrity checking, is great for sending around structured information and doing all these fancy applications. But when you can’t trust it it’s not useful.
PEM on the other hand has all of the security capabilities but no structure. And so you can send the data around but you don’t know what it is. So you really need both of them, working hand in hand and in concert, in order to get things to function correctly.
Malamud: Well if they do different things then isn’t it just a matter of stapling the two standards together and reissuing ’em?
Freed: Um…to a certain extent that is actually true. Unfortunately there were design choices made that cause a certain degree of conflict between the two. The two standards were developed— MIME is a much younger effort. The Privacy-Enhanced effort has been going on for a lot longer. And it may appear that they cover less territory, but you also have to remember that the debates and the discussions around what constitutes security and all are much more difficult questions to answer than how you come up with a structuring scheme. So, the amount of time it took them to solve a lot of these problems is probably largely due to the difficulty of the problems. And yes they look like relatively isolated problems and small problems in scope, but they’re difficult problems nevertheless.
The issue then came up that the PEM work had largely reached completion shortly after the MIME work was issued. And the choice was, do we want to hold up the PEM work which people are anxiously waiting for and have been for years, do we want to hold it up even more while we unify the two things? Or do we want to get it out there and get people started experimenting with the message integrity infrastructure that they’ve been wanting for so long. And the decision, which was not mine to make, but it was nevertheless taken to go ahead and push PEM through and worry about the interoperability issues later. And what then immediately happened, as is often the case in the Internet, is a document was immediately composed and circulated almost at the same time as the PEM documents were being issued as proposed standards. A MIME/PEM interaction document was composed by myself, and Marshall Rose, and Steve Crocker.
Malamud: What are examples of some of the incompatibilities between the two standards?
Freed: Well, a good example is let’s say I use Privacy-Enhanced Mail on top of some 8‑bit material and I send it and I’m doing authentication only. So I’m not encrypting. So I can basically send the stuff in clear text. Okay, so now what I have is I have a privacy-enhanced header on the material—actually, it’s in the body of the message. But we have the header and then we have the material that’s been privacy-enhanced. And that material let’s just say for the purposes of discussion contains some 8‑bit characters. And this goes zipping around the Internet.
Now, it hits a MIME-compliant gateway which detects the 8‑bit material and says “I really can’t transmit that on without checking to see how somebody else…” you know, whether the next hop can accommodate it or not. It checks the next hop cannot accommodate it; this is using let’s say the SMTP extensions. What it then does is it MIME encodes the material so that it will pass through. So it goes hop hop hop, and then it hits the final destination on the network. At which point the PEM agent, which again we’ll say for purposes of argument is installed there, isn’t going to know about the encoding that was done in order to make the thing pass through the network.
The best result you could hope for is that the message integrity check is going to fail, at which point you don’t know whether the thing is authentic or not. The worst case would be that the PEM implementation becomes terribly confused and won’t I won’t do anything right. Which is a possibility depending on the robustness of the software.
Malamud: So PEM is making sure that your message wasn’t tampered with en route, and MIME explicitly tampered with the message in order to get it to its destination.
Freed: Exactly. And so what you need to do is you need to have the PEM material initially encapsulated inside of MIME so that the agents can be aware of what transformations occurred as the result of transport.
The way that the MIME/PEM document basically addresses this issue is we take material and we we take all the encodings off of it and go back to a canonical format. And then we apply the privacy enhancement technology to that. And then we re bundle it up in MIME all over again. And we send it off. And this has all the advantages that now the MIME transformations are done on the outer side of the privacy-enhanced interior, so that there are no problems or issues with what can happen to it in situations like what I described.
Malamud: Well now the X.400 committee when they were looking at designing a messaging system they built security in from the beginning.
Freed: Well… I would take some issue with that statement. They built security in in the context that they have some stubs for where you could send around encrypted body parts. To my knowledge, that work has never been fully fleshed out. There are also other ways of doing security extensions to X.400 which involve basically taking the entire message and encoding the whole thing and sending it across the wire. And there also are transport-level authentication mechanisms that you can use in various ways.
The X.400 community, in dealing with security, attempted to address a broader range of problems with security issues. As a result they have ended up with a solution which addresses some of the smaller problems and possibly the more important problems, depending on your needs, less well. Privacy-Enhanced Mail with MIME allows you to do things like encrypt parts of a message but not others; or authenticate parts of a message but not others; or mix and match all these kinds of things in fairly complicated ways, so that you can have a document that has sections that only some people can read. You could have a document in two versions, one of which contains the classified material, one of which does not. The one that does not has authentication applied, the one that contains the classified material is also encrypted. There are all kinds of clever applications that you can use in mixing these things. The X.400 work really doesn’t address a lot of those kinds of applications because they’ve never really fleshed out what it means to have an encrypted single body part.
Now, the flip side of that is that when you do these kinds of fancy things someone can tamper with the message en route and perhaps remove a body part from the message, and you will be unable, unless you have applied authentication around the whole message. You may not be able to detect that. The X.400 solutions solve that problem.
The X.400 solutions also solve a problem which is of concern to some high-security sites, where they are less concerned with the material being tampered with but with the leakage of material out of the site.
Malamud: Do you think X.400 has a future? Or is MIME gonna take it over?
Freed: Ha. Um… I have a cheap answer to that, which is I wouldn’t’ve spent a significant amount of time in the last year building an X.400 gateway to MIME if I didn’t think there was at least some future to X.400.
Will MIME take over completely, no. Will X.400 take over completely, no. We are doomed to a future of two standards, in my opinion. There are enough government requirements and other things that are essentially administrative fiats that have occurred that even if X.400 was a genuinely dreadful travesty of justice in every sense of the word, which perhaps some people believe but I don’t. Even if that were true, and everyone agreed on it, there would still be sites that were compelled to use it. And that compulsion is not going away anytime soon, I don’t believe.
Malamud: But even without the compulsion, is there room in this world for two sets of mail standards? In an ideal world, would—
Freed: In an ideal world would there wouldn’t be but in this world there’s got to be room. We are not going to be able to go with one. There are sites, there are systems that simply are incapable of using the X.400 facilities, and there are things missing from X.400 that people need. Equivalently, there are sites that are not going to be able to use MIME, and there are—we hope nothing missing from MIME that they need, but there probably are things that we haven’t addressed.
Malamud: So in a messy world we need a messy set of solutions.
Freed: That’s exactly right.
Malamud: Well there you have it.
Malamud: This is Internet Talk Radio, flame of the Internet. You’ve been listening to Geek of the Week. You may copy this program to any medium and change the encoding, but may not alter the data or sell the contents. To purchase an audio cassette of this program, send mail to radio@ora.com.
Support for Geek of the Week comes from Sun Microsystems. Sun, The Network is the Computer. Support for Geek of the Week also comes from O’Reilly & Associates, publishers of the Global Network Navigator, your online hypertext magazine. For more information, send email to info@gnn.com. Network connectivity for the Internet Multicasting Service is provided by MFS DataNet and by UUNET Technologies.
Executive producer for Geek of the Week is Martin Lucas. Production Manager is James Roland. Rick Dunbar and Curtis Generous are the sysadmins. This is Carl Malamud for the Internet Multicasting Service, town crier to the global village.