Domino Transaction Logging

Friday 28th January, 2005
Using transaction logs with Lotus Domino is becoming more commonplace and gets better with each release, with significant added value from Domino 5 to Domino 6.

As backup windows get shorter companies are investigating archival transaction logging with specific backup solutions. But what is the best way to implement this?

This recently came up on an internal email I received from a sister company. It raises many good questions.

If you have any thoughts/experience on this, I'd appreciate you share it here.

  1. 1) Neil Thomas Said: (01/02/2005 17:45:02 GMT) Gravatar Image
    Domino Transaction Logging

    OK Ben I will try to do some answers on this one since nobody else has.

    I have only seen this on iSeries with BRMS as the backup tool and have not used the Tivoli offering, but the principles are the same.

    I am probably teaching you to suck eggs with this but I will go through it anyway.

    In archival mode the Tlogging engine will create shards and fill them up with data continuously until the backup engine cleans out the shard and marks it as ready for use again. The data cleaned out by the backup engine is again in turn stored until the next backup session, when it is cleared out.

    So you need enough space for the shards to grow, depending on the performance of the backup engine, and also enough space for the backup program to store it■s Tlog data before the backup kicks in.

    With any enterprise server build you should not be running the Tlogging on the same Controller or Drive Array as the Domino server data. This can cause sever performance issues which we will go into later.

    For backup and DBIID issues, your backup software will need to force a clean backup the night of a Compact change. So for instance if a database is compact ■C or compact ■D during the day, the backup program should do a full backup of that database and then backup any shards stored since the compact was completed.

    When restoring, you will restore to a dummy server instance, which will allow the restore of full databases then the roll forward to the required time. Obviously you are slave to the quality of your backup solution. You should be able to identify the date to which you want it restored. Also if you request a date after an intermediate full backup (new DBIID during the working day), then the backup software should skip forward to the latest full backup, restore it and then roll forward from there.

    Obviously if your full backup fails, you have to take another one, otherwise your backup strategy is blown as with any other kind of backup. Clustering will make this a less painful issue.

    When the domino server is running, a server task clears out the shards and marks them for deletion. However I have seen an instance where the server was so stressed (multiple partitions using the same disk controller with Tlogging and backup happening at the same time on the same logical disk), that the agent marked a shard as ready for use when it had not released the lock. Transaction logging immediately crashed the server. Not pretty and the server was unstable until we disabled Tlogging.

    As part of the fault finding for this I switched off the BRMS agent to see what would happen. At 26 Gig of extents I decided that nothing was going to happen and loaded the BRMS agent up again. It took a while but it cleaned out the extents. However it left all 26 Gig of empty extents taking up disk space. This can be resolved by running a backup with the server down then deleting all extents before starting the server again.

    So it is the same with all things. Plan the implementation. Architect the solution. TEST the thing to destruction. Then deploy the solution and pray.

    Good luck. The site I was on had more than one incremental fix from IBM before it worked correctly.

  2. 2) Richard Thomsen Said: (14/02/2005 17:33:13 GMT) Gravatar Image
    Domino Transaction Logging

    I use Tivoli Data Protection for Domino where I work, and it is great for reducing the time windows required for storage. The trick is balancing how often to do the 'selective (full)' backups. Too many and you might as well be doing a full backup every night...too few, and you will spend hours restoring transaction logs to get to the desired day. Twice a week works ok for us.

    One other caveat...when you enable transaction logging, the compact button in database properties no longer reclaims space. I wrote an article on how to work around this with some event monitoring and agents in The View. If you are a quota-based organization, that compact issue needs to be carefully considered prior to enabling transaction logging (since users could potentially go over quota and not get back under without administrative action to do a compact -B).

  3. 3) Philip Storry Said: (20/02/2005 20:32:24 GMT) Gravatar Image
    Domino Transaction Logging

    I echo what others have said here, except I'd like to add one more tidbit:

    Richard says that the compact function no longer works, etc. - and he's right.

    But on my R6.5 servers, I have an option on the Transaction Logging tab of the server document which allows me to choose how quota enforcement is done. There are two file size options - check the file size when adding a note, and when extending the file. Both of these will behave as Richard says.

    But the third option, which I think was added in R6, is "check space used in the database when adding a note". Or something to that effect.

    It means that it you go below your quota by deleting stuff, you won't need to compact. It'll spot that even though the file is over quota size, the space isn't used - and it'll not bother sending a quota warning. Your users will like that, because it means that they go below quota immediately - otherwise, they'll keep getting the quota warnings until compact runs.

    I think that this option adds a little load to the server, but not much. I've noticed no problems so far on some pretty busy servers...

  4. 4) David Grasso Said: (22/02/2005 10:55:19 GMT) Gravatar Image
    Domino Transaction Logging

    Philip,

    The correct option is "Check space used in file when adding a note"

Add Comment
 
Subject:
   
Name:
E-mail:
Web Site:
 
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?