Here is the real-life story of DAOSifying far over 4TB of email data storage.
Situation
- Lotus Notes and Webmail users: 7.000 in Benelux
- Primary mailservers running Domino 7: 4 (2 clusters)
- Archive servers running Domino8.5: 2 (1 for each cluster)
- Storage and data: All servers have 2 TB of Data LUN on SAN
- Data growth: Mail data grows roughly 10 GB per day (no quota allowed)
- Operating system: Windows 2003/64 with 4 GB of RAM
- DAOS data storage: same LUN as the Data (will be moved to separate disk)
- DAOS style: agressively set at 256 KB
Since all storage systems where full with data the customer was not able so archive anymore, so we upgraded the Archive servers to Domino 8.5 and started using DAOS and compression.
Current status
This is what happend so far......the processes are still running since we choose to control per directory going step by step.
- Arch01 50% DAOSIFIED
- Arch02 25% DAOSIFIED
- Start compression of Design
- Data and recompress attachments to LZ1, on subset of the Archived files and then enable them with DAOS
We where able to reduce the storage on the archive servers with around 750GB in total, this cleared the way to archive mailservers again an perform the same DAOS action again on newly archived files and added storage making 400GB of space in the mailservers.
- Arch01 uses 950 GB for NSF data and 325 GB for DAOS, while the logical size of all NSF files is 1,7 TB
- Arch02 uses 1430 GB for NSF data and 275 GB for DAOS, while the logical size of all NSF files is 2,0 TB
The numbers are not 100% correct but they are really close, we should have measured a bit more closely. Since the storage keeps growing and we continue archiving and DAOS'ifying this is a moving target at the moment.
Conclusion
It's hard to calculate exacly how much is reduced but it show significant impact in reduction of storage and back-up time.
Posts
4 responses:
Very important step was to reduce the data with full compression via >compact -c -v -n -ZU [folder] before enabling DAOS. This already reduced the databases a lot. It also compensates for the feature that not all attachments in mail is LZ1 compressed although you enabled LZ1 compression throughout the domain on all servers mail files and mail.box-es. With the large database at this customer it took a while, but it was very worth it because the initial reduction was higher than expected, but also DAOS is more effective when all attachments are compressed with the same algorithm.
Interesting indeed.. I wonder if your customer isn't by any chance Eniac's parent company called Imtech... It's just a wild guess
how many .nlo files did it produce?
Did your file system support such ammount of files?
Adam, NTFS is quite capable of handling milions of files, so there is no issue there. To be able to tell you how many there actually are one would have to wait several minutes for Windows to actually count the amount of items and used storage. It could be a problem restoring all these small files when rebuilding the server, but restoring the NSF files would be first priority anyway.
Post a Comment