I had trouble with my ReadyNAS NV+ before. Now it seems like a deja vu. In October the SMART error count of disk 3 went up and the automated messages suggested that the drive might fail any time soon. So I replaced disk3. The ReadyNAS resynced the drive and live seems good. Before I left for winter holiday I shut down all systems. After my return and rebooting my infrastructure everything seemed normal. Only after a few days I got the message that disk 3 had failed and needed replacement. I shut down the unit and checked if any of the connectors had a problem. Then I tried to reboot. The ReadyNAS wouldn't come up claiming disks being defective (on that little LCD in the front panel). Since I had to travel I shut down the unit and left it. So after my return I tried again. Now the ReadyNAS booted and claimed it would resync drive 3 and it might take a while. So I left it. In the morning the unit claims that disk 4 was dead now. So I opened a ticket with Netgear. Having had bad experience with their phone support before I opted for online submission. Now we are playing ping-pong and I get one message after the other that won't bring me closer to resolving my issues. There are a number of things that irk me:

1. I had 5 replies from TechSupport so far. 6 replies from 5 different people (and I start to suspect, that the name is system generated): Erik, Kelvin, Nick, Thomas and Orson
2. Support claimed that the WD10EADS in my system isn't on their compatibility list However there is the WD10EACS, which is just an earlier model.
3. Thomas mentioned to get a data recovery working he needs to escalate the case, but when I asked for that I got another round of check this check that.
4. Every reply starts with "Thank you for choosing Netgear". That's not necessary after the first round
5. The statement We will do our best to help resolve your case in the least possible time. is lacking the <irony> tags

Netgear is moving dangerously close to my No-Fly-Zone (I wonder if DLink does NAS with RAID).

In Tim's and my Lotusphere session AD111 we rendered data from an XPage into a ODT document. Once you get the hang of it and it is easy to implement export to text, spreadsheet or presentations. One nice aspect: no server side OpenOffice or Symphony code is needed. Here is how you do it:

1. In DDE use Window, Show Eclipse Views, Other (Alt+Shift+Q Q) and select "Navigator"
2. Locate the WebContent/WEB-INF folder and add 2 directories: src and lib
3. Download the Java toolkit from ODF Toolkit and add odfdom.jar to your WebContent/WEB-INF/lib directory
4. Right click on your NSF and select Project Properties. Then select Java Build Path. In the source tab add the src folder you created in step 2. In the Libraries tab add the odfdom.jar you copied in step 3
5. Create the class you want to use for rendering the content (rendering logic goes there). It should have a method that takes an OutputStream as parameter (I called it render)
6. Optional: define your new class to be a managed bean. Once you do that it can be used as if it is a native global variable. To do this locate the faces-config.xml in the WEB-INF directory and add a bean definition. Could look like this:
   

   <faces-config>
     <managed-bean>
       <managed-bean-name>stuff</managed-bean-name>
       <managed-bean-class>com.lotusphere2010.ad111.Sample</managed-bean-class>
       <managed-bean-scope>session</managed-bean-scope>
     </managed-bean>
   <faces-config>
   

7. Create a new XPage, go to the page properties, basics and set rendered=false. You can see in the XSP source
   <xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false">
8. Add code (see below) in the beforeRenderResponse event. It is important to use this event, so you can get hand on the OutputStream rather than the writer
9. The ODFToolkit requires one additional setting for security, so locate [NotesInstallDir]/jvm/lib/security/java.policy file (you need to do that on the server too) and add:
   grant { permission java.lang.RuntimePermission "shutdownHooks"; };
10. You are all set. Enjoy. As usual YMMV

XPages and server side JavaScript come quite some classes (and I'm not counting the Java classes you could use). We have:

• Class Library: DOM
  • DOMAttr
  • DOMCDATASection
  • DOMCharacterData
  • DOMComment
  • DOMDocument
  • DOMDocumentFragment
  • DOMDocumentType
  • DOMElement
  • DOMEntity
  • DOMEntityReference
  • DOMException
  • DOMImplementation
  • DOMNamedNodeMap
  • DOMNode
  • DOMNodeList
  • DOMNotation
  • DOMProcessingInstruction
  • DOMText
  • DOMUtil
  • NamespaceContext
  • NamespaceContextImpl
• Class Library: Domino
  • DirectoryNavigator
  • NotesACL
  • NotesACLEntry
  • NotesAdministrationProcess
  • NotesAgent
  • NotesAgentContext
  • NotesBase (recycle() is defined here)
  • NotesColorObject
  • NotesDatabase
  • NotesDateRange
  • NotesDateTime
  • NotesDbDirectory
  • NotesDirectory
  • NotesDocument
  • NotesDocumentCollection
  • NotesDxlExporter
  • NotesDxlImporter
  • NotesEmbeddedObject
  • NotesError
  • NotesException
  • NotesFactory
  • NotesForm
  • NotesInternational
  • NotesItem
  • NotesLog
  • NotesMIMEEntity
  • NotesMIMEHeader
  • NotesName
  • NotesNewsletter
  • NotesNoteCollection
  • NotesOutline
  • NotesOutlineEntry
  • NotesProperty (used in Mashups and composite apps)
  • NotesPropertyBroker
  • NotesRegistration
  • NotesReplication
  • NotesReplicationEntry
  • NotesRichTextDoclink
  • NotesRichTextItem
  • NotesRichTextNavigator
  • NotesRichTextParagraphStyle
  • NotesRichTextRange
  • NotesRichTextSection
  • NotesRichTextStyle
  • NotesRichTextTab
  • NotesRichTextTable
  • NotesSession
  • NotesStream
  • NotesView
  • NotesViewColumn
  • NotesViewEntry
  • NotesViewEntryCollection
  • NotesViewNavigator
  • NotesXSLTResultTarget
• Class Library: Runtime
  • I18n
  • Locale
  • TimeZone
• Class Library: Standard
  • Array
  • Boolean
  • Date
  • Function
  • Math
  • Number
  • Object
  • RegExp
  • String
• Class Library: XSP
  • DirectoryUser
  • NotesXspDocument
  • NotesXspViewEntry
  • XSPContext
  • XSPUrl
  • XSPUserAgent
• Function library: @Functions (with many functions- for the next time)
• Function library: XSP Functions (with many functions- for the next time)

That's it. Read the full list with all properties and methods.

When you want to use DAOS in Domino 8.5.x you need to activate Transaction logging. DAOS allows to store attachments on cheaper disks and eliminate duplicates (Not sure if you want DAOS, use the estimator, it will tell you). BUT DAOS requires transaction logging. Transaction logging requires, to be performant, its own disk. The best practices guide states it very clearly: " It is absolutely essential to place the transaction log directory on a separate physical device devoted solely to transaction logging. Part of the performance improvement you will gain from transaction logging depends on having a separate disk, which allows fast writes to the transaction log files. Putting the transaction log on the same drive as other files — the operating system, the Domino executables, the Domino data, the OS pagefile, etc. — can result in a significant performance degradation even in comparison with an R4 server, since the server must read from and write to these files as well.".

So speak after me:" I shall use a dedicated drive for my Domino transaction log. I won't use a partition, a directory or a SAN destination. I shall use something insanely fast for this drive.".

No ifs or buts or eventuals. So the DAOS savings require some disk investment. While you are on it: make sure you use ODS51 and compression for design and data - and review the design for high performant Domino servers (Raid 10 works best). Others have recommendations too.

In a lot of customer discussions I hear: "Oh we need to go RDBMS since Notes' NSF doesn't scale". Which is an interesting statement. When digging deeper into that statement, I get a number of reasons why they believe that:

• Somebody told them
• They had a problem in R4
• The application got slower and slower over the years (and yes it's that same old server
• The workflow application using reader fields is so slow
• They actually don't know

Then I show them this:

(atually not the graphic, but the live property box using my Notes client). The question quickly arises how the acceptable performance of such a database can be achieved. There are a few pointers to observe:

• Watch your disk fragmentation (troubleshooting tip on the Notes and Domino wiki)
• Be clear about your reader and author fields usage. In case the RDBMS fans insist on their solution, ask them to build the reader field equivalent in RDBMS and measure performance then.
• Watch your view selection formulas carefully. (You don't use @Now, @Today, @Yesterday or @Tomorrow do you?)
• You want to use DAOS to keep attachments out of the NSF (helps with fragmentation) -- don't forget to buy a disk for your transaction log.

As usual YMMV.

If you were following the tweets about Lotusphere or LotusKnows you know everything what is contained in this post. The guest speaker was William Shatner of Star Trek fame. Instead of repeating the chronological sequence of the OGS I'm listing our product and partner announcements sorted by product name. Enjoy.

• General
  This year's Lotusphere theme is "Lotus knows why. Lotus knows how" revolving around three business themes: The collaboration agenda, the Cloud and LotusNext. The collaboration agenda is IBM's industry specific approach to collaboration. It provides a framework approach to identify gaps in collaboration patterns and improve effectiveness and efficiency. It is the official transition from product selling to solution selling, something the more successful sellers have embraced for many years already. Now the power of IBM's internal collaboration will help all to do that. IBM's view on cloud is a natural extension of existing on premise equipment. IBM's cloud vision encompasses a seamless integration and transition between any IT assets whether local or in the cloud.
• General Manager
  Bob Picciano handed over the role of Lotus General Manager to Alistair Rennie.
  
• Lotus Connections
  In H2 2010 Lotus Connections 3.0 will be available it adds, based on customer demand, new capabilities around metrics and audit capabilities and extended mobile access. Social Analytics will allow to automagically find connections hidden in plain view through a recommendation engine to connect to people and communities.
  
• Lotus Foundations
  Xerox is delivering OfficeNow! solutions as Lotus Foundation applications linking to Xerox multifunctional devices.
  
• LotusLive
  Lotus Live has after just one year in the market more than 18 million users. in Q2 IBM will add LotusLive Communities which today are available in Lotus Connections on premise.
  LotusLive Engage
  Engage now is integrated into the Notes client using new sidebar applications. Engage is the perfect tool to extend collaboration safely across corporate boundaries. Why eMail a file around with a version nightmare if you can share them online including tracking who looked at what version.
  LotusLive iNotes
  Light eMail solution starting at $3/User month.
  LotusLive Labs A new feature in LotusLive will allow users to test new capabilities before they become mainstream functions (Google labs anyone ). It will have Slide Library (Slide River), Collaborative Recorded Meetings (Agora) and other. Labs will be available in Q2 to IBM partners.
  LotusLive Notes (hosted Notes) allows for a hybrid model with on- and off-premise mix of Domino servers. Minimum number of users lowered from 500 to 50. Mailbox size is 5GB/user. Sametime chat is included and services for migration and backup are available.
  Domino servers hosted in the cloud for development will be available on developerworks that will take less then 30 minutes to set up.
  
• Lotus Notes and Domino
  Businesses adopting Notes and Domino 8.5.x at a record adoption rate, twice as fast as the previous fastest deployment. To continuously improve product quality there will be the 8.5.2 maintenance release in 2010.
  
• Lotus Quickr
  Lotus Quickr 8.5 Beta for both Domino and J2EE can be expected soon. The J2EE version now runs on top of the current Websphere Portal 6.1.5 making deployment much easier. The release of Quickr 8.5 is expected in Q2. The Domino version will catch up with the J2EE version and directly connect to ECM systems. The available APIs are greatly extended and XPages developers can directly access Quickr services.
  The Quickr plug-in for Lotus Notes now allows to not only store attachments but entire eMails in the open standard EML format. Quickr calendars now can surface as iCal streams to be integrated into your personal calendar. Quickr will support CMIS to connect to all sorts of backends including our competitors.
  
• Lotus Sametime
  Lotus Sametime 8.5 has shipped. Community (chat) and meeting functions are now fully separated with the later running on a Websphere infrastructure (like the Sametime gateway and Sametime advanced before). The new clients (Eclipse, Web, Mobile) allow a very rich user experience. Audio/Video is lacking in the current web clients, but planned as future enhancements. Now users have their fixed meeting room to be used any time. Meetings can be recorded in industry standard formats. (Note on this: there are 2 IBM research project to take the recording further: Agora and Falcon - see them in the innovation lab. I'm a proud member of the Falcon research team).
  
• Lotus Symphony
  Lotus Symphony 3.0 Beta will be available in February and go gold in Q2. It is based on the OpenOffice 3.x code stream and will improve the user experience, functionality and stability of Symphony. One important new feature is the support for Visual Basic Macros in xls files.
• Lotus Traveler and Mobile
  The Lotus Traveler server is coming to Linux. A native iPhone application allows for message encryption and is available in the iTunes appstore today.. Lotus Traveler is bundled with the Nokia E72. A native Android application will support mail and calendar to be delivered in H1. The plan is to initially support the Google phone (a.k.a Nexus One) and Motorola's Droid. RIM produces new clients for the Blackberry for Sametime, Quicker and Lotus Connections, delivering on an old promise. The application will be sold both by IBM and RIM. A lot of detail improvements around directory lookups and scheduling.
  
• Websphere Portal and Mashups
  IBM offers Industry accelerators for key industries to dramatically cut down the time to deploy a workable Portal. Accelerators are initially available for health care, government, banking, insurance, education, manufacturing and transport. Portal and Mashups are fully integrated into Unified Telephony.
  
• Partners
  IBM featured partners on stage: Xerox, iEnterprises, Group Technologies, GoPro and Ascendant.
• Project Vulcan
  IBM's new vision for the future of Lotus Software: continuity (no rip & replace, deeper integration), convergence (unified collaboration, join on-premise and cloud), innovation and new opportunities. Vulcan is based on open standards like HTML5, CSS3, REST, Web Services, Widgets and Mashups. All of this will be available in the Notes client (the premier experience), online and on mobile devices. XPages taking center stage to deliver all that. A new homepage will collate information using social analytics. The homepage looks very much like the universal inbox I advocated before. Searches in the homepage can turn into permanent self updating lists and collections. It will provide full integration of concurrent online co-editing with task and attention management (more on that will be revealed in the mini keynotes during the show). Project Vulcan will be available this year on developerworks.
  

You will find additional and deeper coverage on Planet Lotus. Stay tuned for more entries on this week full of new information.

It is Friday morning 05:00AM. The Taxi is waiting. I'll fly via Tokio and Washington to Orlando on United Airlines. See you on the other side.
Update: Cleared immigration in DC, almost lost my laptop. flight so far was smooth, but it was the first flight in a loooong time without an individual inflight entertainment system in coach. Quite an old 747, but the food was good.
Update 2: In Washington I bumped into Mikkel, which was funny since we met in Frankfurt last year. Door to door travel time:29h 30min.

Should be framed and hand over every product managers desk like a Damocles sword:

Applies to developers too.

I want one:

It is part of the Lego Mindstorms Universe and comes with instructions and plenty of inspirations. Since Robotics is a CCA I have the perfect excuse to buy.

Domino is by nature an I/O intensive application. While R8 drastically improved I/O it is still a lot of writing going on. A lot of Domino servers run on Windows and use NTFS as file system. NTFS isn't exactly designed to run large ever (size) changing files (this is why a RDBMS usually pre allocates the table space to be used - and it is done on new machines). Now just imagine that you would need to ask your admin to pre allocate your 2GB mail file quota for every user (your disk sales guy would send you to heaven for that) regardless of current need. So an NSF reflects the actual size. It's kind being in-between a rock and a hard place. Enter performance improvement strategies. An old myth is that compact -C will improve performance. While it makes the NSF neat and tidy it splatters the file segments all over the place. Adam Osborn explains it nicely in an blog entry. There is a video about it. So what should you do:

1. Learn! Know the Domino Server Performance Troubleshooting Cookbook inside out. Watch Andrew's presentation on Domino performance. Ask Auntie Google
2. Move your temporary filed onto another disk (I would love to hear how that works if you use a RAM disk or a solid state disk). It has 2 effects: I/O is better distributed and the temporary files don't contribute to the data disk's fragmentation. You need to set two notes.ini parameter: NOTES_TEMPDIR and View_Rebuild_Dir
3. Build a high performance Domino Server (remember RAID10 looks good) and tune it well
4. If Windows is your server operating system: defrag, defrag, defrag! There is an excellent presentation (don't mind the fonts) by Albert Buendia of the Spanish Lotus User Group SLUG about it. Realistically you have two choices. The free (and "find-you-own-support") DominoDefrag on OpenNTF (managed by Andrew Luder, Australia) and the commercial DefragNSF from the Australian IBM Business Partner Preemptive Consulting (run by Adam Osborne, Australia) Defrag.exe on Windows isn't an option for executing (only analysing) on your 24x7 box.

It would be interesting to see a fragmentation shootout between NTFS, JFS and EXT4 (anybody listening @ developerworks?)

I want one

Update: fixed the picture, yes comes with vouchers for ambulance, sample wills and divorce lawyers

Sometimes it is good to go back to some basics to explain what Notes and Domino are about. A Notes database is a schema free document store with build in message routing and document level access control mechanism. I'd like to shed some light on the nature of the schema free functionality. A Notes document (technically a note) contains any number of items. There is no need to define possible items. You put an item into a document and it is there. Items have a name, a data type, a sequence (used for replication) and an array of values. That is one of the really strong points in Domino (shared only by Adabas and FileMaker): you don't need to create a master-child table combination, but just store multiple values directly into the document. So a note is conceptually very close to a schema free XML document. The usual way to add an item to a note is to use a form. In a form the developer defines fields (which have a data type, name and rendering properties as well as processing rules: default values, transformation and validation conditions) that get mapped by name onto items. Note the difference: in a form we have fields, in the document (note) we have items A Notes form is basically a piece of RichText with fields thrown in. You can create a working form without the need for any buttons and actions, just throw in some fields and text around them and you are done. The Notes client's default function allow to open, save and close a form. When you open a form and save it a notes document is created with one item for every field in the form. Fields without content result in items with empty values.

In addition some system fields (e.g. for the user saving the form $UpdatedBy) and the field Form containing the form used to create this document is saved. The form field is the only link between the form and the document (If the form has an alias, the alias name is used - handy since different forms can share the same alias. That link can be altered.

I want one:

It flies via your iPhone. Remember that old Bond movie where he drives with a phone as wireless steering? Driving a car with a Nokia 9300i seems lame in comparison.