wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

Minimizing your server downtime when upgrading


We all love upgrades. They steal the nights, the weekends and the public holidays. But it doesn't have to be that way. For this post let us presume you are running Domino 6.x and want to upgrade to Domino 8.5. While Domino 8.5 can very well run on the same hardware specification as a Domino 6.x server, your server is showing its age and anyway is to small for the data growth you experienced during the last 5 years. So you bought a new box. With that you can get away with a downtime of 10 seconds in 6 easy steps (or phases). Here you go:
  1. Phase 0
    This is where you are now. That phase had a duration of 5 years and ends with the management approval to buy a new box and its delivery to your closet data center.
  2. Phase 1
    You install and configure a brand new Domino R8.5 server with a new server.id and a new IP address. In our sample that server is called tmpServer/YourOrg at IP 10.34.21.88. Make sure to check the following:
    • You make that new R8.5 server the directory server of your Domino Directory
    • You have added CREATE_R85_DATABASES=1 to the notes.ini and made the server a member of LocalDomainServers and/or your server group. You have checked all databases to include the group as manager in the ACL
    • You made sure that the new server has the same access (ACL level, roles) to your databases like the existing one
    • You replicate all databases from your existing server to the new server
    • You verified (using document counts) that all documents indeed have been replicated (You could have an agent checking unids if you want to be fully save)
    • Ensure that in your database the properties for LZ1, data and design compression are set. It has no effect on the existing database if you have to set it until you do a compact -c or create a new replica
    • Update (thanks Matt): Verify that your replication formulas made it. Also check your Unread marks.
    • The server document allows only a group "UpgradeProject" access to the new R8.5 server. This group contains LocalDomainServers and the team that does the upgrade
    • You copy both server ids to each of the servers
    Phase 1: the new server goes on stream
    Duration: Anything between a few hours and a few days during regular working hours.
  3. Phase 2
    You unplug the network cable from the new server, shut down the Domino server. You edit the IP configuration of the server to use the existing server's IP address (this is why you unplugged the cable in the first place). You edit the server's notes.ini and point to the existing server.id to be used. Check the notes.ini if you need to set more parameters (e.g. network port addresses for partitioned servers etc.). Restart your server but keep it unplugged.
    Phase 2: the new server gets unplugged
    Duration: 5-10 minutes (depending on how long your server needs to boot), best during off-peak hours since Phase 3 needs to follow immediately.
  4. Phase 3 (The Downtime)
    Best you do that together unless the boxes are close to each other. You type in the Domino server console: Set Config Server_Restricted=1 (thx Thomas) and Drop ALL (If you are a nice guy you warned your users with a broadcast message before you do that. Remember: a (!) sends the broadcast to a dialog box). Your server is now clean of users and no user session can be opened. Pull out the network cable and plug in the network cable of your new server. Since the server name and the IP address is the same as the old server before no other configuration needs to be done. You are back in business.
    Phase 3: The new server goes online
    Duration: 10 sec (If you type and plug fast even shorter).
  5. Phase 4
    You edit the notes.ini of your old server and point to the temporary server.id. You edit the IP address of the server to point to the temporary IP address. You remove the Server_Restriced line from the notes.ini. You shutdown the server, plug the network cable back in and reboot it. A final replication with the new server makes sure that anything created in Phase 2 is back where it belongs. You have full working access to the "old" databases now.
    Phase 4:check that everything works
    Duration: Take your time. Check everything.
  6. Phase 5
    Everything worked out, you reformat your old server and give it a new lease of life.
  7. As usual: YMMV. Update Elaborated Phase 1 bases on the comments below.

Posted by on 08 July 2009 | Comments (a) | categories: Show-N-Tell Thursday

Comments

  1. posted by goran on Thursday 09 July 2009 AD:
    There are some more considerations to be done:

    1. There is a problem moving from 6 directly to 8.5. Check SPR# JCIK7PFGCT

    2. The signature function is changed in 8.5 and all the users will loose their signatures and will be unable to use this function.

    So you have to plan moving your users to new client as well.
  2. posted by Andy Pedisich on Thursday 09 July 2009 AD:
    Nice! This is pretty much exactly how I move to a bigger, faster, better server platform. It works 100% of the time.
    - Andy
  3. posted by Keith Brooks on Thursday 09 July 2009 AD:
    just did this the other day, walked someone through it. Nice visuals, have forwarded along.
    By the way, great SNTT graphic!
  4. posted by Sean Cull on Thursday 09 July 2009 AD:
    I usually clear the replication histories before I switch the server ID. Not sure if it is required but I like the idea of keeping everything as clean as I can
    Sean
  5. posted by George Paglia on Thursday 09 July 2009 AD:
    This is great stuff!!!

    My only comment would be to check if any databases that have Roles in their ACLs are also assigned to this new server (or server group) BEFORE beginning the replication to duplicate the data. Both servers' must have matching access and roles; if they don't, the document counts will be different.

    Unfortunately, the function to create replicas does not allow you to tailor access "like another server" when you are doing the creation. We find it prudent to use the catalogs to check document counts before throwing the switch.

  6. posted by Bruce Olschewski on Thursday 09 July 2009 AD:
    I like this procedure! One follow-up question: I have a Domino based web application that accesses the Domino Directory for authentication. If I leave the old server up for a few months until it goes away, will I have any problems if I put the new server.id and IP address on the old server? I would like to avoid putting it on the new server because of some of its requirements.
  7. posted by Dave Harris on Tuesday 21 July 2009 AD:
    I thought a server restart removed Server_Restricted from the notes.ini, or is that not the case anymore?
  8. posted by Colin Williams on Saturday 08 August 2009 AD:
    I used this technique a couple of weeks ago to swap out an aging server and upgrade it from 8 to 8.5. The entire operation took 20 minutes -- most of which was taken up by the physical swap out of the servers themselves. A huge thanks for posting this!
  9. posted by Anthony Suson on Friday 14 August 2009 AD:
    Hi,

    I agree, Domino upgrade and server move to new machine then replicate all databases is very easy and seemless to the end-user (if you have small DATA files).

    However, our environment is that DATA MAIL files resides in SAN Server. We will only replace the Phyisical MAIL Server and will still used the SAN. Once the NEW Mail server is completed, we'll then connect the 900GB DATA MAIL files.

    In this case, what would be the best approach to Upgrade the server?

    Also, if we'll perform the fixup, compact and updall task Offline BEFORE and AFTER the upgrade, i guess it would take us days to finish it all together...


    Please advice,
    Thanks
  10. posted by Frank on Wednesday 08 January 2014 AD:

    System database, such as log.nsf, admin4.nsf, busy.nsf or clubusy.nsf, etc... are needed to be replaced in tempoary server in step 4. Also need to ensure pending mail in mail.box are cleaned in old server. Some extra planning things, such as DAOS, transaction logging, add-on, antivirus, backup/restore