The draft version of the Redbook Self Assessment and Strategy Guide for Migrating from Domino Document Manager has been published. I contributed to the book mainly in Chapter 11. Analyzing Customizations. Part of the effort is an upgrade to DXLMagic which now sports a GUI (build with Thinlet and a module that can compare two Notes database designs, not limited to DDM templates. The comparison report is a start but (for my taste) far from perfect. Give it a spin and compare some databases and let me know in which sequence I should improve what.

Categorized views are kind of a trademark of Lotus Notes (Client) applications. We like them, we build them, we love them. We also want to see them on the web. There is only one small issue: That display of information is pretty unique to Notes. You do find this tree/table combination in other applications only to display files (like Nautilus, Finder or KDE) but not data. So a categorized view is kind of odd on the web. I played around to find alternate displays for single and multiple category data. Here is what I came up with.

• Single Category view with Listbox
  The category isn't part of the view itself but a picklist on the left (which might be filled by a @DbColumn). The table shows the current selection matching the selection. Works out of the box already today. Interesting extension challenge: allow selection of multiple entries in the listbox.
  
• Single Category view with Combobox
  A variation of the first theme. Useful if you have a lot of view columns and need the real estate on the left. Variation: Instead of the dropdown: show a link that uses a popup to allow selection of the category to show.
  
• Multiple Categories with Combobox
  This actually works also with a sorted view since the limit key takes an array as entry. As variation similar to the previous example could be to show a breadcrumb link list that allow to click or hover to show the selection. Challenge: How to make it obvious that you need to select/narrow from left to right. Extra challenge: when you change box 2 and the value in box 3 is no longer an available value
  
• Multiple Categories in tree/table combination
  Looks like a file dialogue, so users should be familiar. You trade horizontal space for vertical space. Makes navigation in categories more accessible since all categories are available any time
  
• Pivot view on 3 categories
  Category 2 becomes the columns of the table, Category 3 the rows. Adds sums to rows and columns. Good base material for graphs. Challenge: decide on the aggregation mode: sum/average/percentage -or- optional display of such a column
  
• Pivot view on 3 categories with data rows
  Similar concept like the previous but with individual data rows displayed. Might show additional aggregation rows or columns
  

Now someone needs to build all these custom controls. All the images have been build using Balsamip Mockups

The new DXLMagic UI is making progress. One of the features I'm working on are reports included into the JAR file, so you can pick them from the internal list easily. I'm planning to have reports for @Formula, Fields, HideWhen and actions. What reports would you like to see in that list?

We get this question quite often lately obviously driven by who-does-not-need-to-be-named claiming lesser bandwidth requirement. Of course that is utter nonsense. There is no instrinct bandwidth requirement in Domino. After all Notes servers were happily running on 4800 Baud modem connections. Bandwidth is the speed, so when you rephrase the question, you see it is missing half of it: "How fast should [insert-your-favorite-land-transport-vehicle] be?" The logical reply: to do _what_? So when looking for bandwidth requirements you need to know: how much data do I have and in what amount of time do I need (or want) this data to be delivered. So step 1 is to compute these values:

• Average requirements:
  [Average/Median number of messages per hour] *[Average/Median size of message] / (360* [Acceptable average delivery time in seconds])
  The 360 is to adjust hour to seconds.
• Peak requirements: [Peak number of messages per hour] *[Peak size of message] / (360* [Expected maximum delivery time in seconds])

These formulas are product independent. Now you can apply additional factors. E.g. when messages are transmitted using SMTP/MIME attachments swell by 34% due to the mime encoding. Notes compresses documents, data and network communication and can save 5-70% of transmission size. Why this big spread? Well when you transmit a compressed archive file, there is little you can squeeze out it, a old MS-Office document on the other hand can loose 80% of its size when compressed. There are a few caveats of course:

• Corporate habit: we see very often that 80-90% of messages are retrieved in just 2 of 24 hours. so when you calculate 24000 messages/day you mis- calculate your average to be 1000 messages/hour while your true average is 9600 in the relevant hours.
• You underestimate your growth. What might have been enough 3 month ago might not be good enough one year in the future. (IBM internally seems to be a big difference. Using Lotus Quickr and Lotus Connections we actually see a decline in message volume)
• Management (or user) expectation: They would expect prompt delivery event for the biggest messages at peak time (ties a little toward the first point)
• Bandwidth availability. This is mostly an issue on VPN connections. The nominal speed my ISP bills me is far higher that what I ever able to get.

What's your bandwidth experience?

When looking for data in a Notes database using LotusScript you have 3 possibilities: NotesView.getDocumentByKey (and its cousing NotesView.getEntryByKey), NotesDatabase.dbSearch and [NotesView|NotesDatabase].FTSearch. Use each of them wisely (when to use what might make another nice post). I want to focus on FTSearch for now. One would expect that a fulltext search against a view is substantial faster than against a whole database. However the results are quite different. The fulltext index is build once for the entire database. So even a FTSearch against a view will use the Index for the whole database. Searches are actually very fast. What makes a difference is how the results are processed and how you intend to use the result. If you plan to list them all out, there is no real difference. If you only want to show a subset continue reading. Very often a search returns 1000 documents but you only want to show 20 or 50 at a time. The core search returns an document collection that contains the UNIDs but not the document objects. When you use db.FTSearch the document objects are initialised when you loop through the collection as do a NotesDocumentCollection.getNextDocument(doc). So if you only use a fraction (50 of 1000) you would only initialize document objects 50 times. On a NotesView.FTSearch on the other hand the collection gets fully initialised since Domino needs to check if the document meets the view's selection criteria. If it matches the document stays in the result collection, if not it gets removed. So even if you would only use a few documents you would need to bear the waiting time of all document object initialisation calls. Unfortunately if you need a very specific sorting sequence you need to stick with a view. Andre gives more advice. In summary: in cases where your expected results are much bigger than what you want to show NotesDatabase.FTSearch beats NotesView.FTSearch

We have scheduled 3 more XPages training classes in AP:

• 28/29 July, Jakarta - Indonesia (contact Gunawan for details
• 31Aug/01Sep, Seoul - Korea (contact Kim Ky Young for details)
• 22/23 Sep, Hanoi - Vietnam (contact Nguyen Hoai Nam for details)

And yes. Class delivery is by yours truly.

We all heard the stories about DAOS space savings. In case you missed the party, go download the estimator and run it against your server. But there is much more to the R8.5 release. I'm using the new ODS structure (ODS51) also locally (simply add CREATE_R85_DATABASES=1 to your notes.ini and run a little script). My mail file as of just now has about 6000 documents, almost no attachments (I'm using MyAttachments) and design and data compression enabled. I have tons of folders that I use regularly, so there is quite some view indexes in that NSF.

Out of curiosity I replicated the mail file back to a test server running Domino 7.x. Domino 7.x runs ODS43 and doesn't support document or design compression. That same database swell to 273 MB. This is an increase of 50% (or if you walk the other way around: Moving from ODS43 to ODS51 saved me 33.3%). That's a very compelling reason to act.
As usual YMMV

It has been a while since the last update. There are a number of interesting enhancements coming soon to DXLMagic. While the command line interface is still available for your automation projects there will be a GUI and other enhancements:

• GUI for Export, Import, Injection, Reports (Transformation) and Server Documentation
• Graphical Installer for Windows (Mac/Linux maybe later)
• A list of build in reports
• New Function: Compare 2 databases
• New Function: Brute Force View Tuning (article coming soon)

Stay tuned.

XPages is a constant topic when talking to Domino developers. One of my favorite questions to them is: "What (custom) controls are you expecting to use". The controls might exist, being build or being acquired. This is the unsorted list of what I heard so far:

• Names fields with address picker
• Custom address dialogs
• Numeric spinners
• Graphic controls connected to a data source
• Custom data sources control
• Controls supporting encryption/decryption
• Workflow controls
• Tagging input/Tag cloud from Lotus Connections
• Controls that are Sametime aware
• Chat controls
• Data filter control that looks like the Excel filter mechanism
• Tree control to render an outline
• Dialogbox control
• Login/User account control
• Pivot table linked to a data source
• Picture gallery controls in various forms and shapes including cover flow
• Task list controls (add this to a task in Notes or Activities)
• Notification control (send a message to notify others: eMail, Chat, Twitter)
• Document history controls
• Access control control (reader/author field management)
• Translation control (translate the DATA)
• Rating control
• Feedback control
• Shopping cart control
• Lotus Connections Business Card control
• Attachment control. Here I got multiple ideas around attachment management including Quickr integration
• "See also" control: add a link to the page
• Application Help control
• Comment control
• Expiry control (from just adding a date to sophisticated lookups in corporate policies databases)
• Media view control (Flash, Media players etc.)
• Inline edit control (edit one field without switching to edit mode)
• Spreadsheet control (with Export - thx John)
• Search control (from search in a view to Enterprise Search to Google search)
• Poll control (ask a question and render the results)
• ToolTip control
• "Print This" control
• PDF control
• Gannt chart control
• Scheduling control (using Quarz) for tasks
• Balsamiq Mockup control
• iLog Rule engine control
• Update: New control requests/ideas
  • Query Building control (Formula/FullText), thx Erik
  • Datasource merge control: Have 2 identical structured datasources and merge their content into a single grid/table
  • Login/Credential for SSO, thx Kevin
  • Transformation/Export control: transform the content of the pages into other formats. A superset of PDF/Spreadsheet export.
  • Paypal control: there are new more open APIs available now
  • Actionbar / Dropdown menu control
  • Right Mouseclick menu
  • Document lock/unlock control
  • Timer control: execute JavaScript and/or Ajax calls on time
  • Slilder control for views, data tables and repeat control

Interesting insight: We are seeing "small" controls like the names field as well as "big" ones like an entire Workflow piece. To be clear: that's what customers and partner said, that's * not* taken from any IBM ToDo list (or at least none I'm aware of). Some of these controls are already in the wild.
What controls are you looking to use in your application?

An interesting question landed on my desk: " In a web application you use extensive agents to prepare or render content. When the user is authenticated you want that agent to run in the user's context. For anonymous users you want the agent to run in the agent signers security context, so you don't need to expose your resources to anonymous browsing. How to do that?"
Answer:

1. Write all your business code in functions in a script library. Have one function as entry point. It is a matter of taste to use the documentContext as parameter or to retrieve that inside the function.
2. Write two agents that both call that function: "agAnonymous" and "agUser". Configure the first one to run with the signer access, the second one to run as web user.
3. In your WebQuer[Open|Close] use this formula:
   agentToRun := @if(@UserName = "Anonymous";"agAnonymous";"agUser");
   @Command([ToolsRunMacro];agentToRun);

As usual: YMMV.

HTML has widely used and more exotic tags. Fieldset and Legend being of the later type. Theo reminded me, that they can be useful and asked how to incorporate them into a XPage. The best approach is to create a custom control with custom parameters and an editable area inside and simply type the html controls into the source code. The legend is rendered by a computed field, so translation will be able to pick it up.

The source code of the XPage looks like this (note: the custom property doesn't show up here since it is stored in a file hidden from the Domino Designer perspective):

< xp: view xmlns:xp=" http://www.ibm.com/xsp/core">
    < fieldset>
        < legend>
            < xp: text escape=" false" id=" legendText" value=" #{javascript:compositeData.legendText}">
            </ xp: text>
        </ legend>
        < xp: callback facetName=" facetFieldSetContent" id=" callbackFieldControlSet" />
    </ fieldset>
</ xp: view>

When using the custom control in a XPage or another control it looks like this (note the content inside the "this.facets") is what you put in there. can be a panel or table with fields in it or another control.
>

< xp: view xmlns:xp=" http://www.ibm.com/xsp/core" xmlns:xc=" http://www.ibm.com/xsp/custom">
    < xc: FieldSetControl id=" MyFieldSet">
        < xc: this.legendText> This is the &lt;b&gt;legend&lt;/b&gt; text</ xc: this.legendText>
        < xp: this.facets>
            < xp: panel xp:key=" facetFieldSetContent" id=" stuffInsideTheFieldSet">
                < xp: label id=" label1" for=" inputText1" value=" Test Field">
                </ xp: label>
                < xp: inputText id=" inputText1" />
            </ xp: panel>
        </ xp: this.facets>
    </ xc: FieldSetControl>
</ xp: view>

As usual: YMMV

Domino 8.5.1 went into controlled beta recently. While others do performance testing provide us with and performance tips I poked around detail improvements. One of the early tips to integrate dojo components into XPages is to use a DIV wrapper. In 8.5.1 this is no longer necessary, since you now can specify dojoType and dojoAttributes directly on a panel (and some other custom controls). A nice step to cleaner code

We all love upgrades. They steal the nights, the weekends and the public holidays. But it doesn't have to be that way. For this post let us presume you are running Domino 6.x and want to upgrade to Domino 8.5. While Domino 8.5 can very well run on the same hardware specification as a Domino 6.x server, your server is showing its age and anyway is to small for the data growth you experienced during the last 5 years. So you bought a new box. With that you can get away with a downtime of 10 seconds in 6 easy steps (or phases). Here you go:

1. Phase 0
   This is where you are now. That phase had a duration of 5 years and ends with the management approval to buy a new box and its delivery to your closet data center.
2. Phase 1
   You install and configure a brand new Domino R8.5 server with a new server.id and a new IP address. In our sample that server is called tmpServer/YourOrg at IP 10.34.21.88. Make sure to check the following:
   • You make that new R8.5 server the directory server of your Domino Directory
   • You have added CREATE_R85_DATABASES=1 to the notes.ini and made the server a member of LocalDomainServers and/or your server group. You have checked all databases to include the group as manager in the ACL
   • You made sure that the new server has the same access (ACL level, roles) to your databases like the existing one
   • You replicate all databases from your existing server to the new server
   • You verified (using document counts) that all documents indeed have been replicated (You could have an agent checking unids if you want to be fully save)
   • Ensure that in your database the properties for LZ1, data and design compression are set. It has no effect on the existing database if you have to set it until you do a compact -c or create a new replica
   • Update (thanks Matt): Verify that your replication formulas made it. Also check your Unread marks.
   • The server document allows only a group "UpgradeProject" access to the new R8.5 server. This group contains LocalDomainServers and the team that does the upgrade
   • You copy both server ids to each of the servers
   
   Duration: Anything between a few hours and a few days during regular working hours.
3. Phase 2
   You unplug the network cable from the new server, shut down the Domino server. You edit the IP configuration of the server to use the existing server's IP address (this is why you unplugged the cable in the first place). You edit the server's notes.ini and point to the existing server.id to be used. Check the notes.ini if you need to set more parameters (e.g. network port addresses for partitioned servers etc.). Restart your server but keep it unplugged.
   
   Duration: 5-10 minutes (depending on how long your server needs to boot), best during off-peak hours since Phase 3 needs to follow immediately.
4. Phase 3 (The Downtime)
   Best you do that together unless the boxes are close to each other. You type in the Domino server console: Set Config Server_Restricted=1 (thx Thomas) and Drop ALL (If you are a nice guy you warned your users with a broadcast message before you do that. Remember: a (!) sends the broadcast to a dialog box). Your server is now clean of users and no user session can be opened. Pull out the network cable and plug in the network cable of your new server. Since the server name and the IP address is the same as the old server before no other configuration needs to be done. You are back in business.
   
   Duration: 10 sec (If you type and plug fast even shorter).
5. Phase 4
   You edit the notes.ini of your old server and point to the temporary server.id. You edit the IP address of the server to point to the temporary IP address. You remove the Server_Restriced line from the notes.ini. You shutdown the server, plug the network cable back in and reboot it. A final replication with the new server makes sure that anything created in Phase 2 is back where it belongs. You have full working access to the "old" databases now.
   
   Duration: Take your time. Check everything.
6. Phase 5
   Everything worked out, you reformat your old server and give it a new lease of life.
As usual: YMMV. Update Elaborated Phase 1 bases on the comments below.

I like movies. My family and I go to the cinema quite often (a movie is only half the fun without the coke and the popcorn) and we own quite a number of VCDs and DVDs. What p\*\*\*\*s me off more and more: in every movie I get a advertisement about how bad "burning movies" is. Every DVD starts with an FBI warning telling me: "Hey you spend your money with us, but we tell you you probably are a thug". Grab any DVD from a Bangkok street vendor and you are not bothered by advertisement you can't skip or a warning about your obvious criminal intend. I don't mind to pay for media I watch, but I very much mind being called a potential criminal after I actually paid due to greed (?) of the media industry. Economy of abundance anyone?

I'm a customer of a number of banks. HSBC being one of them. I also hardly visit any bank branches. I do my business online. I like HSBC's site. It is easy to use and fully functional (Short of a glitch that doesn't allow to set a date for all type of transactions). In recent years all the banks have added additional security to their online login by requiring a security token besides your user name and password. This token is either generated by a little gadget or send via SMS to your mobile phone. HSBC choose the first option (while some smarter banks actually let you choose what option you like). The token vendor they picked seems to be on the cheap site and with 99% probability the devices' internal clock will get out of sync with the security server (about every 2-3 month), so you can't login. A call to the help-desk fixes that, but it takes 3-4 hours.
In other words: you can't depend on the availability of HSBC Internet banking when you need or want it. I chatted a little with the help desk guy who was very pleasant to talk to and highlighted that this problem dents HSBC's reputation. So I asked if I'm just dumb out of luck or the problem is widespread. He admitted, that _all_ HSBC Internet banking customers will experience that type of problem (Guess that's why there was a specific option in the voice menu just for that). I'm now seriously considering to close my account since I'm not amused. All other bank tokens I use(d) never fail(ed).
Note to HSBC: Fix the problem or loose customers