With the new IBM Lotus Domino 8.5.x features you are ready to dramatically reduce your data volume which saves resources, backup time and storage and thus money!
Which features am I talking about?
- Design note compression
- Document compression
- De-duplication of all data (archiving)
- De-duplication of attachments (DAOS)
- LZ1 compression of attachments
- Optimize pasted bitmap
Design compression is one of the simplest actions. You enable the feature on your templates and the new databases created from that template will be using about 50% less disk space. Since even an empty database can be multiple MB in size (mail85.ntf is 30 MB!) and the reduction can be multiplied by the amount of databases related to that design the savings can be quite reasonable.
You can enable the design compression for existing databases on your servers with a simple compact option and your users will not notice the difference even if they use clients with lower release than Notes 8.
Data volume reduction is relative to how large a database is compared to the design size. If all your mail files are more than 500 MB in size, the gain will not be much. But if you have 5,000 mail databases on your server, 15 MB x 5,000 = 75 GB, might seem significant to some.
ODS: This feature is available since version 8.5 and requires ODS 51.
Document compression is enabled in the same way as Design compression. The data volume reduction differs a lot depending on which kind of data is stored, but according to IBM can be up to 60% for a document. We have not seen numbers quite as large like this in mail file document compression.
ODS: This feature is available since Domino 8.0 and requires ODS 48
Archiving is a way of moving Notes data out of active mail files to an archive copy. You enable archive policies which are applied via organizational or explicit policies. You can only archive personal mail files and based on time specific criteria, not size. It is recommended to archive to a separate server to have the best effect. If you have multiple servers hosting mail files, clusters for instance, archiving is in fact de-duplication of all data.
Server based, policy based archiving has not been very reliable, like one of the following blog items will show. IBM Lotus Domino 8.5.1 promises to remedy most of the problems we have encountered in previous releases. Apart from fixes, no major things have changed so the archiving works with any ODS and mail template on your 8.5.x servers.
ODS: this feature is available since R6.5 with ODS 43, but possibly any ODS.
De-duplication of attachments (DAOS)
We have shown that the promise of data reduction is actually delivered. This new feature of Domino 8.5 is quickly proving to be the major cost reducing and performance enhancing feature. With DAOS attachments which are the same across DAOS enabled database on a server are stored only once and outside the NSF. This reduces to the NSF data and the de-duplication of attachments in a read-only store facilitates fast backups and reduces IO. In 8.5.1 more work has been done for communication between DAOS enabled servers and communication between clients and DAOS enabled servers. How cool is it, that a mail with a large attachment sent from your PC will not have to send this attachment if it is already on the server. This also happens when servers replicate data. On the fly attachments are compared to the DAOS catalog and determined if they still need to be transferred. All this lowers storage, disk IO and network bandwidth.
The DAOS estimator is very usefull in determining the estimated amount of DOAS and NSF data results.
ODS: This feature is available with Domino 8.5.x and requires ODS 51 at least.
LZ1 compression of attachments
LZ1 compression of attachments is not new. This has been available since R6 together with network compression. LZ1 is a fast compression method which vastly reduces the size of files without needing much resources on client or server. To correctly implement this, you have to enable it in the advanced database properties on mail.box-es and mail files. Even local mail.box-es need to be enabled for LZ1 compression. The local mailbox.ntf is already enabled for LZ1 compression, so the resulting mail.box is as well.
The mentioned DAOS estimator is also useful to show that you have a lot of attachments either not compressed at all, or compressed with Huffman only (pre-LZ1 compression).
See the stats below:
Compression Statistics:
None: 425173
Huffman: 139633
LZ1: 158006The reason for this is that although you enabled all mail.box-es and all mail files with LZ1, the router is not compressing incoming SMTP.
If you combine this LZ1 information with DAOS and know that 2 attachments which are the same, but stored with different compression techniques are NOT the same for DAOS. This means that DAOS is less efficient than when all attachments are stored as LZ1. Hence it is very important to make sure all attachments are compressed with LZ1. You should re-compress the attachments (compact option) for current data (see important information below) especially when using DAOS. The ability to compress incoming SMTP with LZ1 is available in Domino 8.5, but should only be used with 8.5.1. I will explain how this is done later.
ODS: This feature is available in Release 6.x and requires ODS 43 to enable it.
Optimize pasted bitmapMany users know that Print-Screen is very useful to grab information from your screen (in Windows) and are able to send this via e-mail to support or other users. However, the clipboard is storing this screen-print in BMP format and it is pasted as BMP as well in your Notes client. BMP is not the most efficient way to store images because they are very large. Users can work around this issue of sending large mails with screen shots in it by importing other formats like JPG or GIF or transforming the BMP to JPG (for instance in MS Paint) and pasting that information. Notes can do this for you on the fly with a simple client side Notes.ini parameter:
OptimizeImagePasteSize=1ODS: This feature is available in Notes 6.5.5 and up and has no requirements for ODS, but 43 is expected
CombinedBefore you ask : "This is all very nice, but can I combine all these methods?" my answer is "Yes, you CAN!"
Let us know what your experiences are.
References:LZ1http://www.ibm.com/support/docview.wss?rs=899&uid=swg21090049Recompresshttp://blog.nashcom.de/nashcomblog.nsf/dx/design-compression-issue.htm?opendocument&commentsand
IBM support: http://www-01.ibm.com/support/docview.wss?rs=899&uid=swg21370344Image pastehttp://www-01.ibm.com/support/docview.wss?rs=899&uid=swg21136853