``Workers in the U.S. receive an average of 200 e-mail messages per day, ...''--BBC News Online 06/01/99
``No member of the community should engage in acts that waste or prevent others from using electronic resources, for instance, forwarding chain mail, sending mass mailings, or knowingly engaging in other activities that degrade service.''--Rensselaer Policy on Electronic Citizenship, 04/13/98
``The administration needs to step up and ensure that it leads by example.''Umair Aly Hoodbhoy, CSYS '99, Polytechnic, 04/21/99
This is an unofficial document written in response to recent mass mailings at Rensselaer Polytechnic Institute. It is intended as a warning and set of caveats for those considering sending ``email to everybody,'' or even a significant subset of everybody. CIS does not support mass mailings, but the reality is many users can and do make use of email for this purpose. It is a good idea to read these notes if you intend to send an unsolicited mass mailing to the Rensselaer community or elsewhere. --mds.
We have had several requests to help with ``mass mailings'' this past year. That is, email sent to all or a large segment of the Rensselaer population. There have also been numerous Rensselaer Computing Systems users who have sent their own mass mailings, sometimes with unintended consequences. This is a brief review of some of the technical and social consequences of a mass mailing that you might not have considered. This is also a guide to alternatives (such as list servers) which may better serve you in the long run.
First and foremost it must be emphasized that CIS does not support or encourage the use of unsolicited mass email. Often an unsolicited mass mailing is sent because the sender has an incomplete idea of how email operates, and is not aware of the effect of a mass mailing on the server, or the response a mass mailing is likely to bring from the recipient community. However, changes in email technology and increasing use of email for communication has led to more people having the technology on their desktop to send a mass mailings. This has led to more people considering doing so, and some who do it heedless of the consequences. If you think you have a need to send out a mass mailing, please read this document carefully.
Electronic mail operates on a ``store and forward'' client-server technology. The client machine, typically your desktop PC or workstation or one of our remote access machines, requests the server machine to deliver a message to one or more recipients. The server machine then takes that message and either places a copy of it in the local mailbox of each recipient, or forwards it to another machine for delivery.
Usually an @rpi.edu address is delivered locally. This involves the computer appending the message to the end of a file called ``the local mailbox.'' The message can then later be read by the recipient when he or she next checks for new email. Note the lack of connection between the mail server and the recipients desktop machine. The message is not sent to the recipient, it is left on the server to be read later, maybe much later and maybe never. In fact, of the 13,000 accounts on the Rensselaer computing system, about 4,000 to 5,000 never read their email in any given week. A smaller but still significant number never read their email at all.
When the recipient is not local, that is, does not have an @rpi.edu address or has mail to their @rpi.edu address automatically forwarded, the mail server contacts one or more additional machines for delivery. Because of transient network problems and other delays a failure to deliver a message will result in retries for 5 days. Only if the recipient is rejected by the destination machine, or if delivery fails for 5 days, will the message be returned undeliverable. Currently, as a convenience to the sender, messages that have transient delivery errors send a notice to the sender after sitting in the outgoing queue for 4 hours. This is standard practice on most mail servers.
When reading mail, the recipient's desktop machine must copy it from the mailbox on the server to its own hard disk. This might be done over the RPI network, or increasingly it might be done from a home account. In the latter case, the time to transfer a message may be long compared to reading a message on campus. If you don't regularly read email via a dialup modem connection it is difficult to appreciate just how fast the local network is. Common problems faced by people reading mail are client timeouts because of slow networks, and running out of disk quota when reading email on an RCS workstation. The latter is usually associated with receiving a large, often unsolicited, mailing from a friend or associate, but this has also happened as a result of incorrectly sent unsolicited mass mailings.
For purposes of this article, a mass mailing will be defined as email destined to more than 100 recipients on the CC and BCC lines. Under this definition, email to a mailing list with more than 100 members is not a mass mailing, since the sender sends to one recipient---the list. Why the distinction between a mailing list and 100 individually specified recipients? We will cover this in more detail below, but the difference comes down to choice---mailing list members join by choice, and can postpone their mail or unsubscribe from the list when they choose. Recipients on a CC or BCC list have no such choice. If you send email that is truly of interest to 100 people consider getting a mailing list. Otherwise, think hard about how many of those 100 people are likely to be interested in the email, and how many others are likely to delete the message unread.
The total number of recipients fails as a definition of mass mail, however, when considering system resources---again, covered in detail below. A 10 megabyte message to 50 people will result in 500 megabytes of email when delivery is finished. This is an excessive use of system resources for one message, and the sender should ask him or herself if all 50 recipients need a copy of the large message. Would a short message, along with a pointer to an ftp or web site where the entire larger message can be retrieved, be better? How many of those 50 people will have trouble downloading the large message?
We will use the 100 specified recipients definition for most of this document. As with all rules-of-thumb, however, the user should apply common sense and good judgment when applying this rule in a specific situation.
The problems with unsolicited mass mailings can be divided into the technical and the social. The technical problems are those that affect the mail server, the networks, and the functioning of the multiple recipient's personal machines. The social problems have to do with the values of the emerging network culture, and the increasing number of laws that stem from those values.
The technical problems are all based on the simple fact that mass mailings do not scale. A single message may occupy only 1,000 bytes of disk space---about one page of text. After delivery that message will be expanded to about 1,500 bytes, since the delivered message will contain ``envelope'' information. Envelope information includes the sender, the recipient, the date and subject lines, and information about the delivery route taken by the message. As a result, even a one line message saying ``hello,'' takes a minimum of 500 bytes of disk space.
If a minimal message were sent to 1,000 recipients, however, it would occupy 500,000 bytes of disk space. If sent to 10,000 recipients (there are 13,000 user accounts on the RCS system), it would consume 5,000,000 bytes of disk space. That's just one message. If 10 such messages are sent a day, 50 megabytes of disk space would be consumed by the messages. Since most people want to say more than just ``hello,'' it is more likely the final delivered message would consume three times that space (or more), or 15 megabytes for one message, 150 megabytes for 10 messages.
As noted above, at the end of one week's time upwards of 5,000 RCS users would still not have read this message. During that week, the disk space would still be occupied by the message on the server. If a single one-page message were sent each day, that would be 5,000 unread messages, times 1,500 bytes per message, times 7 days or 52.5 megabytes. If 10 messages were sent per day, the number would be 525 megabytes, or about 1/8 of the total disk space on the current mail server---enough extra space to bring down all mail service during a typical mid-semester week. When sending mass email, small numbers add up quickly.
This example, however, is artificial for two reasons. First, modern mail clients can be configured to retain messages on the server after they have been read. This is a convenience to those users who read email from more than one client machine (e.g., a desktop machine and a laptop machine, or a machine at work and a machine at home). As a result, at the end of one week, more than the stated space would be occupied by the read and unread messages still on the server.
Second, the example assumes a simple text message. Many messages these days are not simple text, and instead consist of multi-media formats such as sound, video and programs. Even a simple text message might be sent in Microsoft Word format. In Microsoft Word format, the one line ``hello'' message is 20,000 bytes long---40 times the size of the same message sent as simple text. (When I ran this test, I was surprised at the resulting file size!)
A second problem is scale on the recipient end. I gave my example with both one message, such as you might contemplate sending, and 10 messages such as you and nine others like you might contemplate sending. The truth of the matter is, there are over 10,000 of us on the RPI mail system and a good number of us believe we have a good reason to send a mass mailing. Applying the same math, it is clear that if even a small portion of us follow through with that thought, not only would mail service be disrupted when the disk space fills, but we would spend a large portion of our work day reading (or more likely deleting) email messages. Those of us who already do most of our communicating via email do not look forward to the day when all memos are sent electronically. In reality, the result will be individual users creating larger and larger ``filter'' files to automatically delete messages unseen (except for the long download time) and unread. In the end, email would be a less useful communication method.
Finally, the delivery of a large-scale mass mailing (e.g., mail to everybody) stresses the operation of the mail server machine in other non-obvious ways. Remember that the mail server must append the message to the local mailbox of each local recipient. This operation results in ``locks'' being placed on the mailbox, and on the file system in which it is contained. While the mailbox and file system are locked other messages are delayed. Under normal operations this is not a problem, but mass mailings (and large email messages, for that matter) are not normal operations. During past delivery of mail to everybody at RPI, some messages were delayed for upwards of two hours because of failure to aquire a ``lock.'' We hope to address this problem on the new mail machine, but any solution will itself have other, hopefully more distant, limitations. Again, mass mailing does not scale.
Other problem with unsolicited mass email have more to do with how they are perceived by the email using community. You may have heard of the problem with ``SPAM,'' that is commercial email sent to seemingly random recipients, usually advertising a get-rich-quick scheme or adult web sites. While your mailing may not be SPAM in this sense, a significant portion of your recipients will treat it (and you) as though it were SPAM. That is, it was a mass-mailing to a group of people most of whom (regardless of the importance of the message) are not interested. They may have just finished reading 20 or more other equally uninteresting (to them) messages. The end result might be a complaint sent back to you, or a complaint forwarded to RPI's postmaster (see below), or adding you to their SPAM filters. If a complaint is sent, we must followup on it, and, depending on the circumstances, the result may be a suspension of your email service. At the very least, an unsolicited mass mailing will degrade the quality of your important message in the estimation of some recipients.
So many people are increasingly upset over unsolicited mass mailings (including not only commercial mailings, but chain mail, fake-virus warnings, urban legends about sending email to Disney World to raise money for charity, and so on) that a network consensus on if, when, and how it should be sent is emerging. If you do decide that a mass mailing is for you, it is important that you at least understand what this consensus is and follow the guidelines. This is no different from understanding the etiquette for the more traditional forms of person-to-person communication such as telephone calls or door-to-door solicitation.
Although opinions still vary, the consensus is that so-called ``opt-in'' strategies are preferred over ``opt-out'' strategies. That is, people should be allowed to choose to ``join'' your list of email recipients, instead of being added to the list unasked. This is common sense, since presumably people not interested in the topic messages will not join. And, if they do join, they have only themselves to blame.
Opt-in, however, is not always possible. How do you ask people to opt in, if you cannot send them email? Well, there are ways that should be considered (see next section), but sometimes a mass mailing may be the best way to accomplish this initial task. When this is the case, net-etiquette is adamant that an ``opt-out'' must be offered. That is, if you put people on a mailing list, they must be given a quick and simple way to remove themselves from the list and not receive any more email. Increasingly common are web sites or professional societies that offer members a ``trial edition'' of a new newsletter, telling them that they must subscribe (opt-in) to receive additional copies.
In addition to opt-in and opt-out lists, the consensus is that mass mailings should include valid envelope information. That is, the message headers should be accurate regarding subject, date, relay machines and, most importantly, the sender. Most SPAM has forged return addresses which prevent people from responding to the sender. Forging email is vigorously reviled by the network community, and under no circumstances does RPI support forging any header information in email. Forging email will result in a suspension of the forger's email privileges while the issue is being sorted out, possibly longer.
Finally, the message should be short, with a pointer to additional sources of information such as a web page or printed material. Many people read email by modem connections that they pay for. Multiple unsolicited messages are not appreciated for this reason, and long unsolicited messages are appreciated even less.
The above guidelines are so strongly held by enough people that laws have been proposed, and in some states passed, making them legal requirements. That is, valid return address and envelope information, opt-out strategies and even subject lines stipulating that advertisements must be labeled advertisements for easy filtering. Since some of RPI's users may read email from a state which has passed such a law (perhaps via a forwarded address), and because Rensselaer is a member of the network community, care should be taken to ensure any message is in compliance.
Before sending an unsolicited mass mailing, consider the alternatives. The Rensselaer Computing System offers web pages, local Usenet news groups and mailing lists which anybody officially associated with RPI can use to provide information. We are also examining web-casting technologies, and integrated services for possible deployment in the fall of 1999. This is in addition to the traditional methods of interdepartmental memos and campus newspapers. The variety of methods gives the individual user control over how they choose to receive information. For some email is more convenient, while others would prefer to read news or web pages. It is better for all if we honor those choices.
For example, if you are a department or union sponsored club, you can request a listproc mailing list. Listproc is a program optimized to send email to the list members. The RENSSERV-L list, for example, is for general RPI-wide announcements. The advantage of a listproc mailing list is that it offers built-in opt-in and opt-out methods, as well as ways to suspend messages while on vacation, and control who can send messages to the list.
Individuals can also post to any number of RPI news groups which have an active readership. New groups can be created if required and even cross-linked with listproc lists. RENSSERV-L, for example, is cross-linked with rpi.talk.rensserv allowing people to read and post to RENSSERV-L by email, or by Usenet. The listproc software also has a web interface, currently in beta testing, from which list subscribers can control their subscriptions, and read postings.
Anybody at RPI can make a web page, and departmental pages are linked in a common directory. This is a very popular method of distributing information, and many people have made a habit of visiting certain pages for regular updates. The web has no practical size limitations, is opt-in by definition and can be easily read from nearly anyplace. Even if you still feel the need send a mass mailing using listproc or some other method it is usually better to refer people to a web page for details. This reduces the size of the message, and makes it possible to make use of non-text formats in an efficient way. That is, the simple email text message can point to a color-multimedia-java-enabled-dynamic document, updated daily with the latest news.
Posters, bulletin boards, mail rooms, and newspapers. We sometimes forget that these methods have been around for so long because they are so effective. These (along with news and web pages) are good ways to advertise a new mailing list about an important Rensselaer program or topic.
Rensselaer Polytechnic Institute is an educational institution. As such, all Rensselaer employees must comply with the Family Education Records Privacy Act (FERPA) which forbids us from releasing information about students. This means the message cannot contain personal information about individual students that others can read---including a student's email address.
For example, putting student email addresses on a list of recipients that includes people who are off-campus, or who should not have that information for other reasons, is a likely violation of FERPA. For this reason, mass mailings that go to students should have no more than one recipient per message, even if this means sending, for example, 1,000 separate messages instead of one message to 1,000 recipients.
Not only is this a good idea, but forging return addresses is a violation of other RCS policies. All email must be sent in a way that identifies the sender, and allows the recipient to respond. In the case of unsolicited mass mailing, this is also a requirement under anti-Spam statutes, is considered good etiquette, and it provides a way for a recipient to inquire for further information or ask to be removed from the mailing list.
A valid return address also means that somebody is reading mail from that address. A dead-letter box or filter to an unread mailbox does not count as ``valid.'' If it is important to send the mass mailing, it is important to read the responses.
This is partly to be in compliance with FERPA, and also to prevent ``mail storms.'' If a message is sent to 1,000 people all on the messages CC list, there will be two unanticipated side-effects. First, the message size will balloon from a small text message to a small text message with a very long recipient list. For 1,000 recipients, this ballooning will result in a message approximately 18,000 bytes in length---nearly as bad as sending a Microsoft Word attachment. The result will be a message consuming 18 megabytes of server disk space, instead of one consuming 1 megabytes of server disk space.
The second result will be the mail storm. Some of the recipients of the mass mailing will respond to all members of the list, and not just the sender. This may happen accidentally, or be done intentionally, usually to protest the message and any other responses to the list. When this happens, it is easy to consume all remaining space on the server, bringing down mail service for the community. At the least, it will generate hundreds of complaints to the RPI postmaster and the original sender. To comply with both FERPA and this rule, use the BCC (blind carbon copy) instead of the CC (visible carbon copy) commands to list the recipients. If your mailer does not support BCC, or you can't find it, send one message per recipient.
All mass mailings should offer a way for people to avoid receiving future mailings. This can be as simple as ending the message with ``to be removed from this mailing list, send email to <your-email-address>, and then taking steps to remove those who ask. Or, it could be information on how to un-subscribe from a mailing list, a web page with a program providing an un-subscribe service, or a phone number to call. Regardless, the method should be there and be honored. This is to allow our community of users to structure their own message streams according to their own preferences, and to comply with emerging net standards and state laws.
If you are concerned that those opting out will miss an important message, remember that, first and foremost, the Rensselaer community is composed of adults who can make their own decisions. You can always provide a web address in the original message where interested parties can check for updates.
By now the reason for this should be clear. Short email is always better than long email, but it is a necessity in the case of mass mailings. Better than even a one-page document describing all the details is a short description along with a pointer to a web page. This also means that all forms of multimedia and complex file types are not to be sent as mass mailings. Instead, include a pointer or offer to email more information to those who request it.
In general, it is a good idea to avoid sending complex document types to unknown recipients. There are about 7,000 people who read their email at least once a week at RPI using a wide variety of mail readers and desktop machines. By sending complex documents, you limit the number of people who are able to read your message. If, for some extraordinary reason, a complex document must be sent by mass mailings, spread the delivery out over a long time period---perhaps days---to avoid filling the mail spool.
If, for some reason, you think it is necessary to send a large message to 1,000 people (or even 100 people), contact our postmaster first! Even a few people sending a small number of large messages has caused interruptions in mail service. This usually happens over the holidays when people forward animations and other entertaining files. In each case, nobody sent a mass-mailing, just a couple of dozen people sent the large files to just a couple of their friends. The result was nearly a doubling of our normal mail load, interuptions in service, and steps had to be taken (in one case, on Christmas day) to continue normal service.
In any large list of recipients, some of the addresses will be wrong. People move, change jobs, graduate, take leaves of absence, enter incorrect forwarding addresses and so on. Any significantly long list generated in the morning will have incorrect addresses by the evening---it may even be generated incorrectly. There are only a few databases on campus that can provide accurate information about who is in what category of Rensselaer user, and they are not generally available for other uses. It is easy to say ``I need a list of all freshmen chemistry majors who are planning to take CHEM301 next fall.'' It is impossible for CIS to provide you with that list, and it is debatable whether we should.
In short, you need to come up with your own list of recipients, and vouch for it's accuracy. The more people on the list who do not belong there, the more complaints you will receive. This is another good reason for using the listproc server and advertising the list---people can select themselves for inclusion. It is also a good reason for having a simple ``opt-out'' method, so incorrect recipients can get off the list before accumulating too much irrelevant (to them) email.
Avoid broad ``shotgun'' lists. Emailing a message to all members of a class, when it applies to only a well-defined subset, should be avoided. Remember, the more extra people sent your message, the more the message will be perceived as an annoyance by the recipients. You may think of it as just one message, but it will be only one of dozens (or hundreds) received that day by each reader.
Email is not instantaneous. For a mass mailing, it may take several hours for all of the messages to be delivered, longer if done during peak hours. Further, not everybody reads email constantly during the day. Some read it mornings or evenings only, others once or twice a week and others never at all. If the information is time-sensitive consider other methods such as phone trees, campus-mail or radio announcements. Or, establish a web site with updates of important information that people can refer to if the information applies to them.
If, after reading this, you are worried about sending a mass mailing, good---you should be. Mass mailings are the single largest source of email server problems and user complaints. But, if you plan to go ahead anyway (there are some legitimate reasons for sending email to lots of people), and still have questions send email to postmaster. If you are not sure if you need help, you probably need help.
If you have more general questions about mass mailings and the Internet community's response to them, the book, Stopping Spam, by Schwartz and Garfinkel (O'Reilly Associates, 1998) is a good general source containing useful information and references. A good online source of information is the Coalition Against Unsolicited Commercial Email. More resources can be found in You Have Spam! Remember, you may not consider your message to be SPAM and most likely it isn't. But, in the case of unsolicited mass mailings, many recipients will react and respond as though it were. It is important to understand this when making a wise choice in using email.