Use Delta-Files to Optimize Email Storage (19-Aug-2009)

Use Delta-

-Files to Optimize Email Storage

Files to Optimize Email Storage

"Reply with History", is a common function of EmailApplications. If the responder selects "reply withhistory", all previous messages will be included into the newly generated one as history data. Considering the situation thatone email is replied with history multiple times (such as a discussion thread), the history data will be stored many times according to the current implementation of most Email Applications (IBM Lotus Notes, Microsoft Outlook, Foxmail). Especially,

when there are large

attachments in the original email and it is interchanged among many parties, the storage cost becomes inevitable to be considered.

The similar situation also happens for "Forward" function etc.

In general, a typical user mailbox contains hundreds of conversations. Here are several emails extracted from one email conversation.

To optimize email storage, some email applications and standalone compression tools prompt a choice of"compress history data" to the user when they select "reply with history". One staple implementation is that the applications generate a file of the history data, compress the file, and then attach it to the email. Such solution introduces additional effort to the email users, at least one more task: the user need to decompress the file in order to read the history data. To avoid the inconvenience, people may not choose to compress data before they "reply with history".

When receiving an email, the system will detect if there are any referenced emails (the history data) in it and determine which one is the latest one. And then, the comparison process will be kicked off to compare the received email and the referenced one . At the third step, the comparison result will be used to generate delta file . There is a possibility although slim thatthe delta file size is larger or not enough smaller than the received email . So, we compare the two files size , and see if the ratio of storage saving is no smaller than the threshold (whose value can be configured ). If yes, the delta file will be used to replace the received email copy . Or else, the generated delta file will just be ignored . See the workflow in Figure 1.

1

Figure 2: Basic Workflow of Using Delta File to Save Email

1) Conversation Detection in Email mailbox

One conversation detection approach is to detect email threads . The Internet message format , RFC 2822 is a common standard that specified a syntax for text messages within the framework of email systems . It does not provide a precise definition of an email thread , but instead defines how an email thread can be detected using structural attributes embedded in email messages , such as "In-Reply-To", "References" and "Message-ID".

2) Email Comparison

We can treat email as HTML document and utilize existe...