The Great updall argument...

Wednesday 16th August, 2006
On this thread, on edbrill.com, NeilT writes:

"However take that same server, issue a "load updall -r -c" and you will see the space utilisation grow in a spectacular way. That is because the new files will have no indexes. As soon as the indexes are built, you can kiss goodbye to a large chunk of space which will never come back again until you do a compact -d. But the first thing that will happen is that the indexes will come back again as you can't use the databases without them."

I point out in a response that this is fundamentally wrong:

NeilT,

I have never, ever seen anyone issue a "load updall -r -c" on a production server...it's non-sensical.

Either you have indexes and they need rebuilding and you use -r or you don't and you need to create with a -c. These switches just don't have much purpose being issued together.

"Updall -c" alone should be issued sparingly. Yes, on a huge database with thousands of documents you may want to pre-create indexes for that specific database but as a general command...never.

View indexes can be set to be discarded over time for a reason...they aren't in use. There's no point in indexing, and re-indexing, views that a user doesn't use regularly if at all. On my user mailfiles, I estimate that only 25% of the views are in use in the production environment. All the others have either never been built or have long since been discarded. Issuing an "updall -c" will just waste space building an index that isn't required.

"updall -r" only rebuilds indexes that already exist so won't actually use up any more space at all. In fact, as the views get rebuilt, they may even get smaller as they are optimised to take into account deleted documents etc.

So yes, 'issue a "load updall -r -c" and you will see the space utilisation grow in a spectacular way', but who the hell would ever do that?

Neil replies:

Well Ben,

Updall -r updates all indexes used once or more. Updall -c updates all indexes never used and all FT indexes. So, if you want to force a full update of ALL indexes after a rebuid/Updgrade, you will ALWAYS use updall -r -c. Well if you don't want to be doing it real time during user log-in at peak periods and jamming your server up.

As you have said, indexes which are unused are discarded after 45 days. So you create a new database/do a full compact, front load it with all indexes then let lack of use remove those which are not used.

Which part of proactive maintenance and server load levelling did I miss?

Remember that I'm on record as saying the disk usage of indexes are not an issue, so long as you manage it.

What I left you to infer was that claiming this index usage doesn't exist by comparing unindexed databases leads customers to believe they don't need to expand their disk capacity. Disk is cheap, accept and plan for it is what I was saying. Doing the other leads to dissatisfaction and defectors for completely avoidable reasons.

I'm not sure if he hasn't read my reply, or it's hard to teach an old dog new tricks but I followed up:

On the technical side you're wrong, but I don't see any value in a public argument; by all means contact me off-line to discuss further if you wish.

Neil:

Ahh technically wrong. I shall remember that :-)

Bill says he wants to slap me when he's on my side. The last guy I publicly humiliated because he was wrong (and tyring to make me accept it), made my life a misery for 2 years. I shall instead refer you to "Updall options" in the Domino Administrators Guide and leave it at that.

However if you are challenging the processes I use to design/support Domino servers, you have every right to do that and I won't have a public fight over it.

Which naturally called me out in public which I didn't want to do but, seeing as Neil won't take it offline, then here we are.

Yes, I am challenging you - but this isn't the place. I've created a new thread...just for you...I hope Bill is watching :O)

So, we're here Neil...do you want this? Do you want me to tell me you why you are wrong?


  1. 1) Declan Lynch Said: (16/08/2006 15:10:26 GMT) Gravatar Image
    The Great updall argument...

    Just because he does his admin a different way from how you do your does not mean that he is wrong.

    Updall -r -c works perfectly and does what HE wants it to do, rebuild existing indexes and create ones that don't exist.

    If I'm doing a server upgrade I always down the server, run a fixup and a compact -d. do the upgrade and then run an updall -c ( no need for the -r as there are no indexes because aI dropped them all earlier ) and then I start the server up again.

    Am I wrong also ?????

  2. 2) Mike Wissinger Said: (16/08/2006 15:23:22 GMT) Gravatar Image
    The Great updall argument...

    I don't want to get into right/wrong, but I've decided that compact -d followed by updall -c wasn't worth my time. It was my standard upgrade proces through last year, but I have a couple clients with big custom applications that just took forever to build views for.

    When I upgraded the cluster mate I left the indexes in tact during compact and changed the updall to a -r -x. It saved a few hours, and did what was needed.The disk space saved was cheap, but the time necessary to get it installed may not be if server hardware is controlled by another department.

    When rolling out a new applciation, I can see updall -c as being very important to keeping performance up while users form first impressions. For an upgrade of existing servers, though, I've discontinued it. No right/wrong, just not how I want to spend my time on a saturday night.

  3. 3) Bruce Elgort Said: (16/08/2006 16:17:25 GMT) Gravatar Image
    Ummm..gents....

    Read the tag line for Ben's blog: "The ego has landed". :-) We love ya Ben you ASW.

  4. 4) Ben Poole Said: (16/08/2006 16:51:03 GMT) Gravatar Image
    The Great updall argument...

    Sheesh, you crazy admin guys... :o)

  5. 5) Ben Rose Said: (16/08/2006 17:45:04 GMT) Gravatar Image
    The Great updall argument...

    @1 - Declan - If you've deleted all the indexes already with a "compact -d", that's exactly what you'd have to do as starting the server on monday morning without any indexes could be painful to say the least. As you say, there's no need for a "-r" as there isn't any indexes to rebuild.

    Interestingly, the Domino admin guide to which Neil referred me states that "compact -c" must specify a database name...is he saying that he issues it once for each database?

    It's not exactly what's right or wrong when it comes to running updall, of course all the options work without a specific error, my point was that Neil's first statement was just plain wrong in several parts.

    @2 - Mike - Exactly right, unless you're moving to a new ODS, then there's little benefit in the compact except maybe _before_ the upgrade as it may make the rest of the process a little smoother after the DBs are optimised to remove whitespace etc. I personally don't bother with the -x either as the full text indexes will generally have been updated by the time the users get to site anyway.

    @3 Bruce - Dammit, busted...you sussed out my cunning plan. Please don't tell everybody. ;O)

  6. 6) Notes Bucket Said: (17/08/2006 08:25:21 GMT) Gravatar Image
    The Great updall argument...

    Well, I know for a fact (from years of Enterprise customer support) that FIXUP follwed by UPDALL -r solves everything ! :-)

  7. 7) Ben Rose Said: (17/08/2006 14:29:48 GMT) Gravatar Image
    The Great updall argument...

    Previous comment deleted, invalid email address supplied.

  8. 8) NeilT Said: (17/08/2006 21:49:25 GMT) Gravatar Image
    The Great updall argument...

    Sorry been very busy sorting out connector issues and doing cross Exchange fidelity checking so I haven't had much time to look.

    Ok so to the documentation. Yes it says "must specify a database". So when I do a compact -r -c on brand new databases, I should see basically nothing. Because there are no used indexes (well a few because the system gets it's hands on the database before you do, amgr, indexes etc.

    So why does it take up to several hours when I issue an updall -r -c with the server offline?

    Why do all the databases bloat up?

    Why do all the fti files update?

    Why when I just did a simple test with one of my client directories did it shrink down to 680 Mbytes when I did a compact -d and bloat up to 960 Mbytes when I did a compact -r -c? After all, according to you it should to zip, nada, nothing!

    Why when I go in with the Admin client is every single view (with a single document or more), indexed?

    Why am I following an old, but relevant, practise I learned from old documentation whilst working for Lotus?

    Why do I ALWAYS do an updall -r -c?

    It appears to me that if you just do an updall -c, you need to specify the database. If you do an updall -r -c, the -r selects and actions the database, the -c is tacked on as an additional action with the database already in process. Well that's what I have observed anyway over the last 12.5 years.

    As to your questions about "Do I specify databases by name?" Yes actually I do for the system databases. I do it by script and run them during system maintenance windows.

    So Ben, I take your challenge and I throw it back at you. Updall -r -c works. I don't known what updall -c does because I have never used it, not once, ever and was not what I said.

    That's the problem with doing exams instead of spending valuable time in tedious boredom watching old, creaky, slow servers fully index databases. There is no doubt it works when you have just spent several hours watching it work.

    If you want to rip me up Ben, please have at it. I have my faults and I make my mistakes. However when you impugn my knowledge and I am right, I become Very tetchy indeed and find it somewhat difficult to be pleasant. Yes, yes I know you all think I am a miserable cantankerous old git, but my grandchildren and children love me so I can't be all bad.....

    I am not an idiot and I’m not stupid. I don’t work on charity and I don’t lie to my customers. I tell them the truth and live with the consequences. Every Customer I have ever met respects that. Pity I can’t say the same about all the consultancies.

    So let’s not drag this out any longer. It’s gone on long enough and I am way too busy right now to fight about it.

  9. 9) Ben Rose Said: (18/08/2006 00:14:53 GMT) Gravatar Image
    The Great updall argument...

    @8 - NeilT - Sorry, a friend of mind told me you were easy to wind up, he was right...and so was Bruce

    You had the last public word so many times and you still kept going. It doesn't matter, just yanking your chain. Enjoy your updall -r -c, it's never gonna do me no harm :-P

  10. 10) Bruce Elgort Said: (18/08/2006 16:16:54 GMT) Gravatar Image
    The Great updall argument...

    @9,

    I said nothing of the sort about Neil. All I know is that you are easy to wind up but, everybody knows that. :-)

  11. 11) NeilT Said: (18/08/2006 16:27:28 GMT) Gravatar Image
    The Great updall argument...

    @9,@10

    Yup, you're both right. I get seriously ticked off especially when I have a raging migraine.

    I really do have to get out of this rat race. Just don't wind me up too much I might go against my better judgement and recommend an MS product out of spite........ :-D

    Ok so it wasn't my last word..... but almost.

Add Comment
 
Subject:
   
Name:
E-mail:
Web Site:
 
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?